| elements, which then appear as colored
+blocks ``floating'' over the base map image (in our implementation, the
output looks essentially identical to that shown in
Figure~\ref{fig-image}).
@@ -462,54 +483,59 @@
HTML overlays may be generated at either the server or the client.
Unlike the techniques discussed previously, however, HTML overlays can
be generated at the client without the need for additional software,
-because only HTML (i.e., text) is being generated. This can be easily
-achieved using client-side JavaScript, so HTML overlays can use any of
-the distribution styles discussed in Section~\ref{sec-distribution}
-without violating our requirements. We have therefore adopted two
-representative overlay techniques for our experiments: server-side HTML
-overlays (using the image server distribution style) and Google Maps
-(using the shared environment distribution style). Since Google Maps
-uses embedded \verb|![]()
| elements, we have used embedded \verb|
|
-elements for the our server-side HTML overlays.
+because only HTML (i.e., text) is being generated, not images. This can
+be easily achieved using client-side JavaScript, so HTML overlays can
+use any of the distribution styles discussed in
+Section~\ref{sec-distribution} without violating our requirements. We
+have therefore adopted two representative overlay techniques for our
+experiments: server-side HTML overlays (using the image server
+distribution style) and Google Maps (using the data server distribution
+style). Since Google Maps uses \verb|![]()
| elements, we have used
+\verb|
| elements for the server-side HTML overlay.
-Server-side HTML overlays are actually slightly easier to implement than
-either server-side image generation or image overlays, because we do not
-need any code to generate or manipulate images (the base map image is
-static and thus requires no additional processing). All that is required
-is code to transform latitude/longitude coordinates into projected map
-coordinates and produce corresponding \verb|
| elements.
+Server-side HTML overlays are actually slightly simpler to implement
+than either server-side image generation or image overlays, because we
+do not need to write any code to generate or manipulate images (the base
+map image is static and thus requires no additional processing). All
+that is required is code to transform latitude/longitude coordinates
+into projected map coordinates and produce corresponding \verb|
|
+elements.
Google Maps \cite{Goog-M-2006-maps} is a more complex proposition. This
-technique uses the shared environment distribution style, where
-JavaScript code running within the browser enables the client to control
-the display of the base map and any overlays. Data and map images are
-requested asynchronously from the server as required, using Ajax
-technologies.
+technique uses the data server distribution style, where JavaScript code
+running within the browser enables the client to manipulate the base map
+and its overlays. Data and map images are requested asynchronously from
+the server as required, using Ajax technologies, which seems to imply
+that Google Maps in fact uses the shared environment distribution style.
+However, the server has no involvement beyond simply supplying data to
+the client. In the shared environment distribution style, the server is
+directly involved in manipulating the map, under the control of the
+client. This is clearly not the case with Google Maps.
The primary advantage of Google Maps is the powerful functionality it
provides for generating and interacting with the map. Users may pan the
map in any direction and zoom in and out to many different levels. A
satellite imagery view is also available. In addition, further
information about each point plotted (such as the name of the city, for
-example) can be displayed in a callout next to the point, as shown in
-Figure~\ref{fig-google}.
+example) can be displayed in a callout attached to the point, as shown
+in Figure~\ref{fig-google}.
However, there are also some significant disadvantages to the Google
Maps technique\footnote{Interestingly, the Google Earth application
addresses many of these issues, but since it is not a browser-based
-solution it falls outside the scope of this work. However, for
-interest's sake we will do an informal comparison in
-Section~\ref{sec-results} between Google Earth and the four techniques
-that we have tested.}. First, it is a distributed application, thus
-making it more complex to implement, test and debug [REF]. Second, the
-server must have a registered API key from Google, which is verified
-every time that a page attempts to use the API. Similarly, the client
-must connect to Google's servers in order to to download the API's
-JavaScript source. This means that the technique must have an active
-Internet connection in order to work. Finally, the Google Maps API does
-not currently provide any technique of toggling the visibility of
+solution it falls outside the scope of our consideration. However, for
+interest's sake we did an informal comparison between Google Earth and
+the four techniques that we have tested, and this has been included in
+the results in Section~\ref{sec-results}.}. First, it is a distributed
+application, thus making it more complex to implement, test and debug
+[REF]. Second, the server must have a registered API key from Google,
+which is verified every time that a page attempts to use the API.
+Similarly, the client must connect to Google's servers in order to to
+download the API's JavaScript source. This means that the technique must
+have an active Internet connection in order to work. Finally, the Google
+Maps API does not currently provide any way to toggle the visibility of
markers on the map, so it is not possible to implement the interactive
-``layers'' that are possible with an HTML overlay. (It is possible, of
+``layers'' mentioned at the start of this section. (It is possible, of
course, that Google will implement this feature in a later version of
the API.)
@@ -533,66 +559,79 @@
progressively larger synthetic data sets. The first data set comprised
one point at the South Pole (latitude \(-90^{\circ}\), longitude
\(-180^{\circ}\)). Each successive data set was twice the size of its
-predecessor, comprising a regular grid of latitude/longitude points at
-one degree intervals\footnote{It should be noted that only 64,800 points are
-required to fill the entire grid, so the five largest data sets have
-many duplicate points.}. A total of twenty-one data sets were created in
-this way, with the number of points ranging from one to 1,048,576
-(\(=2^{20}\)).
+predecessor, building up a regular grid of latitude/longitude points at
+one degree intervals\footnote{The entire grid has 64,800 points, so the
+five largest data sets have many duplicate points.}. A total of
+twenty-one data sets were created in this way, with the number of points
+ranging from one to 1,048,576 (\(=2^{20}\)). The result of plotting the
+16,384-point data set is shown in Figure~\ref{fig-grid-points}.
+
+
+\begin{figure}
+ \begin{center}
+ \includegraphics[width=0.95\textwidth,keepaspectratio]{ImageGeneration-full}
+ \end{center}
+ \caption{The 16,384-point data set plotted on the base map.}
+ \label{fig-grid-points}
+\end{figure}
+
The focus on scalability meant that we were primarily interested in
-measuring page load times, memory usage, and the amount of data
+measuring page load times, memory usage and the amount of data
generated (which impacts on both storage and network bandwidth). Page
load time can be further broken down into the time taken to generate the
map data, the time taken to transfer the map data to the client across
the network, and the time taken by the client to display the map.
Unfortunately, the Google Maps technique requires an active Internet
-connection, so we were unable to run the experiments on an isolated
-network. This meant that traffic on the local network could be a
-confounding factor. We therefore decided to eliminate network
-performance from the equation by running both the server and the client
-on the same machine\footnote{A Power Macintosh G5 1.8\,MHz with 1\,GB
-RAM, running Mac OS X 10.4.7, Apache 2.0.55, PHP 4.4 and Perl 5.8.6.}.
-This in turn enabled us to measure the time taken for data generation
-and page display independently of each other, thus simplifying the
-process of data collection and also ensuring that the client and server
-processes did not unduly interfere with each other, despite running on
-the same machine.
+connection (as noted in Section~\ref{sec-overlay}), so we were unable to
+run the experiments on an isolated network. This meant that traffic on
+the local network was a potential confounding factor. We therefore
+decided to eliminate network performance from the equation by running
+both the server and the client on the same machine\footnote{A Power
+Macintosh G5 1.8\,GHz with 1\,GB RAM, running Mac OS X 10.4.7, Apache
+2.0.55, PHP 4.4 and Perl 5.8.6.}. This in turn enabled us to measure the
+time taken for data generation and page display independently, thus
+simplifying the process of data collection and also ensuring that the
+client and server processes did not unduly interfere with each other,
+despite running on the same machine.
It could be argued that network performance would still have a
confounding effect on the Google Maps technique, but this would only be
likely for the initial download of the API (comprising about 235\,kB of
-JavaScript source and images), as the API will be locally cached
-thereafter. The API key verification occurs every time the map is
-loaded, but the amount of data involved is very small, so it is unlikely
-that this would be significantly affected by network performance. Any
-such effect would also be immediately obvious as it would simply block
-the server from proceeding.
+JavaScript source and images), which would be locally cached thereafter.
+The API key verification does occur every time the map is loaded, but
+the amount of data involved is very small, so it is less likely that
+this would be significantly affected by network performance. Any such
+effect would also be immediately obvious as it would simply block the
+server from proceeding.
For each data set generated, we recorded its size, the time taken to
generate it, the time taken to display the resultant map in the browser,
-and the amount of memory used during the test by both the browser and
-the web server. The data set generation time and memory usage were
-measured using the \texttt{time} and \texttt{top} utilities
-respectively. The map display time was measured using the ``page load
-test'' debugging feature of Apple's Safari web browser, which can
-repetitively load a set of pages while recording various statistics, in
-particular the time taken to load the page. Tests were run up to twenty
-times each where feasible, in order to reduce the impact of random
-variations. Some tests were run fewer times because they took a very
-long time (several minutes for a single test run). We typically broke
-off further testing when a single test run took longer than about five
-minutes.
+and the amount of memory used during the test by the browser. We also
+intended to measure the memory usage of the server, but this proved more
+difficult to isolate than we expected, and was thus dropped from the
+experiments. The data set generation time and browser memory usage were
+measured using the \texttt{time} and \texttt{top} utilities respectively
+(the latter was run after each test run to avoid interference). The map
+display time was measured using the ``page load test'' debugging feature
+of Apple's Safari web browser, which can repetitively load a set of
+pages while recording various statistics, in particular the time taken
+to load the page. Tests were run up to twenty times each where feasible,
+in order to reduce the impact of random variations. Some tests were run
+fewer times because they took a very long time (several minutes for a
+single test run). We typically broke off further testing when a single
+test run took longer than about five minutes, as by this stage
+performance had already deteriorated well beyond usable levels.
While it is really beyond the scope of this work, out of interest some
informal tests were also undertaken using the Google Earth application.
A Perl script was used to generate a collection of KML files
corresponding to the data sets described above. Each data set was then
loaded into Google Earth, and a stopwatch was used to measure how long
-it took to load the data set, defined as the period that the dialog box
-``Loading myplaces.kml, including enabled overlays'' was displayed on
-screen.
+it took to load the data set, defined as the period during which the
+dialog box ``\textsf{Loading myplaces.kml, including enabled overlays}''
+was displayed on screen.
\subsection{Technique implementation}
@@ -601,9 +640,10 @@
server-side image generation, server-side image overlay and server-side
HTML overlay techniques were all implemented using the image server
distribution style. A separate dispatcher page was written in PHP for
-each technique, which enabled passing arguments---such as the number of
-points to be plotted---from the client to a corresponding Perl script
-for each technique. The final page was then constructed as follows:
+each technique, which enabled arguments---such as the number of points
+to be plotted---to be passed from the client to a corresponding Perl
+script for each technique. The final page was then constructed as
+follows:
\begin{description}
\item[server-side image generation] The Perl script generated a JPEG
@@ -619,12 +659,12 @@
\item[server-side HTML overlay] The base map (a JPEG image) was
displayed using a CSS-positioned \verb|![]()
| element. The Perl
script then generated a collection of CSS-positioned \verb|
|
- elements, which were included inline into the source of the
- dispatcher page.
+ elements, which were included inline in the source of the dispatcher
+ page.
\end{description}
-The Google Maps technique was implemented using the shared environment
+The Google Maps technique was implemented using the data server
distribution style. Once again, a PHP dispatcher page was used. This
time, however, the page included client-side JavaScript code to load and
initialise the Google Maps API, create the base map, and build the map
@@ -640,10 +680,13 @@
\section{Results}
\label{sec-results}
-It is important to note that the intent of these experiments was not to
+As noted in the introduction, the intent of these experiments was not to
do a full analysis and statistical comparison of the performance of the
-different techniques, but rather to identify broad trends. We have
-therefore not carried any statistical analysis of the data.
+different techniques, but rather to identify broad trends. We have not,
+therefore, carried out any statistical analysis on the results. We will
+now discuss the results for data size, page load time and memory usage.
+Because the data set size increases by powers of two, we have used
+log-log scales for all plots.
@@ -657,19 +700,22 @@
\begin{center}
\includegraphics[scale=0.66]{data_size}
\end{center}
- \caption{Comparison of generated data size for each technique.}
+ \caption{Comparison of generated data size for each technique (log-log scale).}
\label{fig-data-size}
\end{figure}
\subsection{Page load time}
+The size of the data generated for each technique is shown in
+Figure~\ref{fig-data-size}.
+
\begin{figure}
\begin{center}
\includegraphics[scale=0.66]{data_generation_time}
\end{center}
- \caption{Comparison of data generation time for each technique.}
+ \caption{Comparison of data generation time for each technique (log-log scale).}
\label{fig-data-generation-time}
\end{figure}
@@ -678,7 +724,7 @@
\begin{center}
\includegraphics[scale=0.66]{page_load_time}
\end{center}
- \caption{Comparison of map display time for each technique.}
+ \caption{Comparison of map display time for each technique (log-log scale).}
\label{fig-page-load-time}
\end{figure}
@@ -687,19 +733,37 @@
\begin{center}
\includegraphics[scale=0.66]{combined_time}
\end{center}
- \caption{Comparison of combined page load time for each technique.}
+ \caption{Comparison of combined page load time for each technique (log-log scale).}
\label{fig-combined-time}
\end{figure}
\subsection{Memory usage}
+We measured both the real and virtual memory usage of the browser by
+running the \texttt{top} utility after each test run and observing the
+memory usage in each category. This told us the size of both the current
+``working set'' and the total memory footprint of the browser process
+after it had completed a test run.
+
+The real memory data proved somewhat unreliable, however. Real memory
+usage was generally consistent across several test runs, but frequently
+it would fluctuate upwards by a factor of nearly two for no readily
+apparent reason. We can only assume that this was a result of other
+processes running on the test machine interacting with the browser
+process in unexpected ways. We therefore discarded the real memory data.
+
+The virtual memory data proved more consistent, as the virtual memory
+footprint of a process is less likely to be impacted by other running
+processes. The amount of virtual memory used by the browser for each
+technique is shown in Figure~\ref{fig-virtual-memory}.
+
\begin{figure}
\begin{center}
\includegraphics[scale=0.66]{virtual_memory}
\end{center}
- \caption{Comparison of virtual memory usage for each technique.}
+ \caption{Comparison of virtual memory usage for each technique (log-log scale).}
\label{fig-virtual-memory}
\end{figure}