diff --git a/Gleeson_paper.tex b/Gleeson_paper.tex index b12cbc7..3b577ac 100755 --- a/Gleeson_paper.tex +++ b/Gleeson_paper.tex @@ -22,6 +22,7 @@ % The amssymb package provides various useful mathematical symbols \usepackage{amssymb} + \usepackage{units} \usepackage{url} @@ -187,11 +188,11 @@ Text box & \(\unit[63]{mm} \times \unit[11]{mm}\) & \(\unit[55]{mm} \times \unit[8]{mm}\) & \(\unit[47]{mm} \times \unit[6]{mm}\) \\ Combo box & \(\unit[63]{mm} \times \unit[11]{mm}\) & \(\unit[55]{mm} \times \unit[8]{mm}\) & \(\unit[47]{mm} \times \unit[6]{mm}\) \\ Button & \(\unit[28]{mm} \times \unit[13]{mm}\) & \(\unit[24]{mm} \times \unit[9]{mm}\) & \(\unit[17]{mm} \times \unit[6]{mm}\) \\ - Check box\(^{*}\) & \(\unit[9]{mm} \times \unit[9]{mm}\) & \(\unit[6]{mm} \times \unit[6]{mm}\) & \(\unit[4]{mm} \times \unit[4]{mm}\) \\ + Check box\(^{a}\) & \(\unit[9]{mm} \times \unit[9]{mm}\) & \(\unit[6]{mm} \times \unit[6]{mm}\) & \(\unit[4]{mm} \times \unit[4]{mm}\) \\ \hline \end{tabular} - {\footnotesize \(^{*}\)This refers to the size of the check box itself, not the associated text label.} + {\footnotesize \(^{a}\)This refers to the size of the check box itself, not the associated text label.} \end{table} @@ -212,13 +213,14 @@ computing environments. Part 9 of this standard describes different tests that can be used to evaluate one or more pointing devices \citep{ISO-2000-9241-9}. The standard describes a serial point and -select task and recognises a dependent measure used with this test -(\emph{throughput}). The serial test comprises moving the cursor back -and forth between two targets using the pointing device and selecting -each target by pressing and releasing a button on the pointing device. -One disadvantage of this approach is that only two targets are used in -the test and therefore interactions between more than two targets, which -often happen in a typical information system interface, are not studied. +select task and recognises a dependent measure used with this test, +known as \emph{throughput}. The serial test comprises moving the cursor +back and forth between two targets using the pointing device and +selecting each target by pressing and releasing a button on the pointing +device. One disadvantage of this approach is that only two targets are +used in the test and therefore interactions between more than two +targets, which often happen in a typical information system interface, +are not studied. \citet{Mack-IS-2001-EHCI} note that throughput is a very important measure, as it reflects the efficiency of the user @@ -226,18 +228,17 @@ accuracy. Throughput is calculated by the following formula: \begin{equation} \label{eqn-throughput} - \mathit{throughput} = \mathit{ID}_{e} / \mathit{MT} + \mathit{throughput} = \frac{\mathit{ID}_{e}}{\mathit{MT}} \end{equation} where \(\mathit{MT}\) is the movement time in seconds (defined as time -taken to successfully select a target) and \(\mathit{ID}_{e}\) is the -\emph{index of difficulty} measured in bits. Throughput is thus measured -in bits per second (bps). -%!! ref to Fitts here? (given that he defined the concept of IDe) +taken to successfully select a target) and \(\mathit{ID}_{e}\) is Fitts' +\citeyearpar{Fitt-PM-1954-Law} \emph{index of difficulty} measured in +bits. Throughput is thus measured in bits per second (bps). The index of difficulty is calculated by the following formula: \begin{equation} \label{eqn-IDe} - \mathit{ID}_{e} = \log_{2}(D / W_{e} + 1) + \mathit{ID}_{e} = \log_{2}\left(\frac{D}{W_{e}} + 1\right) \end{equation} where \(D\) is the distance to the target and \(W_{e}\) is the \emph{effective width} of the target. @@ -255,7 +256,7 @@ \ISOnine\ does not provide any guidance on the range of index of difficulty values to use in testing. \citet{Doug-SA-1999-CHI} recommend using a range from 2 to 6 bits. They also recommend calculating the -\emph{error rate}, as a separate dependent measure of accuracy. The +\emph{error rate} as a separate dependent measure of accuracy. The error rate is defined as the ratio of incorrect to correct selections made on a target, so an error rate of 100\% implies that there were as many errors made as correct selections. Error rate is not included in @@ -266,18 +267,11 @@ the selection device in question. -%!! move this further down --- somewhere in the analysis? -% -% Due to the nature of making a selection within a drop-down list, the -% throughput for a combo box was adjusted to take into account the extra -% distance to the desired list item. - - \subsection{Comfort} \label{sec-evaluation-comfort} ISO 9241-9 argues that to fully evaluate a selection device requires -assessment of user effort and comfort in addtion to performance +assessment of user effort and comfort in addition to performance measurements. Comfort is subjective and can be assessed by means of questionnaires, while effort can be evaluated objectively by measuring the biomechanical load on users as they use a device. Unfortunately, @@ -290,18 +284,19 @@ questionnaire comprised sixteen questions, eight of which were taken from the ISO ``Independent Questionnaire for Assessment of Comfort'' \citep{Doug-SA-1999-CHI}. The remaining eight questions related -specifically to the targets tested and the size of the targets tested. - -In particular, the questionnaire aimed to assess the participants' -comfort in using the input device, the difficulty in accurately -selecting each of the targets and the preferable size of each target -using the input device. +specifically to the target types and target sizes that were tested. In +particular, the questionnaire aimed to assess the participants' comfort +in using the selection device, the difficulty in accurately selecting each +of the target types and the preferred size of each target type using the +selection device. The responses to twelve of the questions were based on a five point ordinal scale. The remaining four questions referred to the -participant's preferred size for each target and were based on a three -point response corresponding to the sizes tested---small, medium and -large. +participant's preferred size for each target type and were based on a +three point response corresponding to the target sizes tested---small, +medium and large (see Table~\ref{tab-target-sizes}). There was also a +space for participants to provide additional general feedback about the +testing process. \subsection{Other considerations} @@ -316,116 +311,121 @@ for learning; indeed, \citet{Doug-SA-1999-CHI} recommend applying a repeated measures paradigm and testing for learning effects. +One interesting aspect of using typical GUI items as targets is the +variation in selection behaviour for different target types, compared to +earlier studies that used simple rectangular targets. The button, check +box and text box target types can be said to employ a ``one-step'' +selection behaviour, because they require only single action (i.e., the +user clicks on them) in order to be selected. A combo box is different, +however, because it employs a ``two-step'' selection behaviour: first +the combo box must be selected in order to show the list of items, and +then an item must be selected from the displayed list. This behaviour is +illustrated in Figure~\ref{fig-combo-box}. To complicate matters +further, users may execute this two-step behaviour using either a +``one-click'' or a ``two-click'' approach. In the former approach, the +user clicks on the combo box, drags down to the desired list item, then +releases. In the latter approach, the user clicks once on the combo box, +then clicks again on the desired list item. If the list were longer than +what could be displayed on screen, this could even lead to a +``multiple-click'' approach, where the user clicks multiple times on the +downward scroll arrow in the drop-down list. We have, however, not +considered this possibility in our experiment. + + +\begin{figure} + \centering + \includegraphics[scale=0.8]{combobox-step1}\quad + \includegraphics[scale=0.8]{combobox-step2} + \caption{The two-step action required to select a combo box.} + \label{fig-combo-box} +\end{figure} + \section{Method} \label{sec-method} An experiment was carried out to test the effect of size for different -GUI targets with different selection devices. The experiment consisted -of completing a series of simple point and select tasks. Small, medium -and large sizes were tested for a combo box, text box, check box and -button. The selection devices tested were a touch screen overlay and -mouse. The test was multi-directional, meaning the target appeared in -more than one direction to the user. A variety of different sizes, -angles and distances were used for each target position. +GUI target types with different selection devices. The experiment +involved participants completing a series of simple point and select +tasks. Small, medium and large sizes were tested for a combo box, text +box, check box and button, using either a touch screen overlay or a +mouse. The test was multi-directional, meaning the targets appeared in +multiple directions from the initial starting point. A variety of +different sizes, angles and distances were used for each target +position. + +The test itself comprised a screen containing a button in the middle and +a target for the participant to select as illustrated in +Figure~\ref{fig-test-environment}. When a participant clicked on the +centre (``Go'') button, a trial was started and a target appeared on the +screen. The trial ended when the participant successfully clicked the +target, which caused it to disappear. The time taken between clicking +the ``Go'' button and successfully clicking on the target was recorded +as well as the number of errors made during the trial. The final +coordinates of the successful click on the target were recorded in order +to calculate the effective width of the target. + + +\begin{figure} + \centering + \includegraphics[draft]{test-environment} + \caption{Screenshot of the test environment with the target in the + top left of the screen and the ``Go'' button in the centre.} + \label{fig-test-environment} +\end{figure} + \subsection{Participants} \label{sec-method-participants} -A participant sample size of twenty four was used for the experiment. +A participant sample size of twenty-four was used for the experiment. Each participant was allocated to one of two groups with each group using one selection device in testing. The allocation of groups was based upon the results of a questionnaire completed by each participant prior to testing. The purpose of the -questionnaire was to establish the level of computer, mouse and touch -screen experience of each participant. A participant was allocated to a -selection device group depending on what device they had the least -amount of experience with. +pre-test questionnaire was to establish the level of computer, mouse and +touch screen experience of each participant. Each participant was then +allocated to a selection device group depending on which device they had +the least experience with. -Due to the testing being done within a nutrition program environment, -the participants were all nutritionists (typical users of the program). -There were 21 female and 3 male participants with all having a -university level of education. All participants were unpaid volunteers. +Due to the testing being done within the nutrition program environment +mentioned in Section~\ref{sec-introduction}, the participants were all +nutritionists (i.e., typical users of the program). There were +twenty-one female and three male participants, all with a university +level of education. All participants were unpaid volunteers. + \subsection{Apparatus} \label{sec-method-apparatus} -Software written in Visual Basic.Net with Microsoft Studio 2003 was used -to implement the test as illustrated in -Figure~\ref{fig-test-environment}. Each test was connected to a Microsoft -Excel worksheet and the data corresponding to the relevant measures -(movement time, number of errors and selection coordinates) were -captured using the software and written to the Excel worksheet. - -\begin{figure} - \centering -% \includegraphics{test-environment} - \caption{Screenshot of the test environment with the target in the - top left of the screen and the ``go'' button in the centre.} - \label{fig-test-environment} -\end{figure} +The test environment was implemented in Visual Basic.NET using Microsoft +Studio 2003, and is illustrated in Figure~\ref{fig-test-environment}. +During each test, data corresponding to the relevant measures (movement +time, number of errors and selection coordinates) were captured by the +software and automatically written to a Microsoft Excel worksheet. The touch screen used in testing was a 17'' Magic Touch USB overlay -Model KTMT-1700-USB-M. This touch screen uses a lift-off touch strategy. -A touch screen overlay is a piece of equipment external to the monitor. -It sits in front of the monitor and behaves similarly to a touch screen -monitor. Using an overlay results in a gap between the overlay and the -monitor itself, this causes a slight discrepancy between where the user -touches the overlay and where the cursor is positioned on the screen. +Model KTMT-1700-USB-M. This device uses a take-off touch strategy, that +is, a selection is not confirmed until the user's finger is removed from +the screen. An important property of touch screen overlays is that they +are placed over a conventional monitor and the touch surface is thus not +coincident with the display surface. This can cause a slight discrepancy +or parallax effect between where the user touches the overlay and where +the cursor is positioned on the screen. The touch screen overlay was fitted to a Dell 15'' Flat Panel Model E151FPb monitor. A flat panel monitor was chosen because it was noticed -during pre-testing that typical CRT monitors with rounded screens caused -a gap between where the users touched the screen and where the cursor -gets positioned. A flat panel is less likely to suffer this problem. The -device used for testing the mouse was a Dell PS/2 Optical Mouse Model +during pre-testing that typical CRT monitors with curved screens +produced a variable gap between the overlay and the display surface, +thus potentially leading to a greater parallax effect than with a flat +display surface. + +The mouse used in testing was a standard Dell PS/2 Optical Mouse Model M071KC. Both devices were connected to a Dell Inspiron 7500 laptop -computer which ran the testing software. +computer that ran the testing software. -\subsection{Procedure} -\label{sec-method-procedure} - -The participant was initially given an introduction to the test by the -research observer. The introduction included a brief summary of the aims -of the study and what the test involved. The participant was also given -and told to read an instruction sheet which they had access to -throughout the duration of the test. After reading the instruction sheet -the participant had the opportunity to ask questions or raise any -issues. - -Participants were instructed to complete each block of tests as quickly -as possible without losing accuracy. In between blocks of tasks, -participants were given the opportunity to rest for as long as they -wished. It was made clear to the participant that a task was only -complete once the target was successfully selected. Selecting the -button, check box and text box required the participant to simply click -on the target. The strategy required to select a combo box was -different. A combo box is a two-step target compared to the other -targets which were simple one-step targets. First the combo box must be -selected in order to show the list of items and then an item from the -displayed list must be selected. During the testing the participant was -instructed to always select the third item in the list when selecting a -combo box as illustrated in Figure~\ref{fig-combo-box}. - -\begin{figure} - \centering - \includegraphics{combobox-step1}\quad - \includegraphics{combobox-step2} - \caption{The two-step action required to select the combo box.} - \label{fig-combo-box} -\end{figure} - -The participant was then instructed to complete a practice task -involving fifteen random trials of the same point and select tasks used -in the test. This brought all participants to a minimal level of -experience with their selection device. This also meant each participant -knew how to correctly select each device including the combo box. - -At the conclusion of the test the participant was required to fill out a -questionnaire regarding comfort and user satisfaction with the selection -device used. \subsection{Design} \label{sec-method-design} @@ -433,55 +433,111 @@ A mixed design experiment was used with the selection device as a between-subjects factor. The independent (between-subject) variables were: - \begin{itemize} - \item Target Type (text box, combo box, button and check box) + \item Target type (text box, combo box, button and check box) - \item Target Size (large, medium and small) + \item Target size (large, medium and small) - \item Target Distance (\unit[40]{mm}, \unit[80]{mm} and - \unit[160]{mm}) + \item Target distance (\unit[40]{mm}, \unit[80]{mm} and + \unit[160]{mm}---see below) - \item Target Angle (\(45^{\circ}\), \(135^{\circ}\), \(225^{\circ}\) - and \(315^{\circ}\)) + \item Target angle (\(45^{\circ}\), \(135^{\circ}\), \(225^{\circ}\) + and \(315^{\circ}\)---see below) \item Trial (1 to 144) \item Block (1 to 6) \end{itemize} +The dependent variables within the experiment were throughput, movement +time and error rate. The entire test was divided into six blocks. Each block contained every -possible combination of target type (4), size (3), angle from starting -point (4) and distance (3). There were 144 trials in each block and the -entire experiment per participant consisted of a total of 864 trials -(six blocks of 144 trials). +possible combination of target type (four combinations), size (three +combinations), angle from initial starting point (four combinations) and +distance from initial starting point (three combinations). Consequently +there were 144 trials in each block and the entire experiment per +participant comprised a total of 864 trials (six blocks of 144 trials +each). Combinations of target type, distance and angle were presented to +the participant in random sequence with no repetition. Target size was +deliberately set to large for the first forty-eight trials in each block, +followed by medium for the next forty-eight trials, and finally small for +the remaining trials, in order to compensate for learning effects. -The different combinations of target location on the screen are -illustrated in Figure~\ref{fig-target-positions} and are a combination -of distance and the angle from the starting point. +The combination of distance and angle from the initial starting point +yielded twelve possible target positions for each trial, as illustrated +in Figure~\ref{fig-target-positions}. Three distances were used that +represented target positions ranging from close to the initial starting +point to very far away from the initial starting point. Four angles were +chosen so that targets could be tested in ninety degree blocks and +giving a good range of screen positions for the target. The first angle +was set to \(45^{\circ}\) with \(90^{\circ}\) increments thereafter, in +order to mimic real life user interface target selection, where targets +are situated in different areas of the screen and therefore selections +are made in multiple directions that are neither simply horizontal nor +vertical. + \begin{figure} \centering - \includegraphics[scale=0.8]{target-positions} + \includegraphics{target-positions} \caption{Positions of targets tested. The black box represents the - starting point and the rounded rectangles represent the target - positions.} + initial starting point and the rounded rectangles represent the + target positions.} \label{fig-target-positions} \end{figure} -The dependent variables within the experiment were throughput (TH), -movement time (MT) and error rate (ER). +The index of difficulty (\(\mathit{ID}_{e}\)) was ascertained for each +possible task using the combination of distance and non-adjusted target +width. This showed that the test had a range of \(\mathit{ID}_{e}\) +values from \unit[0.7]{bits} (\unit[63]{mm} width and \unit[160]{mm} +distance) to \unit[5.4]{bits} (\unit[4]{mm} width and \unit[40]{mm} +distance). It is important to note that the combo box distance values +were adjusted in these calculations to reflect the two-step selection +behaviour of this target type. That is, we need to consider not just the +distance from the initial starting point to the target, but also the +extra distance from the main combo box to the selected list item. In our +experiment, participants were told to always select the third list item +in combo boxes, so the adjusted distance for a combo box was equal to +the normal distance from the initial starting point to the target, +\emph{plus} the additional distance to the third list item. -The index of difficulty was ascertained for each task using the -combination of distance and width. This showed that the test had a range -of Fitts' Index of Difficulty values from 0.7 bits (\unit[63]{mm} width -and \unit[160]{mm} distance) to 5.4 bits (\unit[4]{mm} width and -\unit[40]{mm} distance). The ordering of the target size presented to -the participant within each block in the experiment was deliberately set -to large, medium and lastly small to compensate for learning. + +\subsection{Procedure} +\label{sec-method-procedure} + +The participant was initially given an introduction to the test by the +research observer. The introduction included a brief summary of the aims +of the study and what the test involved. The participant was also given +an instruction sheet that they had access to throughout the duration of +the test. After reading the instruction sheet the participant had the +opportunity to ask questions or raise any issues. + +Participants were instructed to complete each block of trials as quickly +as possible without losing accuracy. Participants were given the +opportunity to rest for as long as they wished between blocks. It was +made clear to participants that a task was only complete once the target +was successfully selected. Because of the two-step selection behaviour +of the combo box target type, participants were instructed to always +select the third item in the list when selecting a combo box (as +illustrated in Figure~\ref{fig-combo-box}). Additional data about the +selections made on combo boxes were recorded in order to account for the +selection approach of the participant, whether it be a ``one-click'' or +``two-click'' approach. + +Before the test began, participants were instructed to complete a +practice session involving fifteen random trials of the same point and +select tasks used in the test. This brought all participants up to a +minimal level of experience with their selection device. This also meant +that each participant knew how to correctly select each target type +including the combo box. + +At the conclusion of the test the participant was required to fill out a +questionnaire regarding comfort and user satisfaction with the selection +device used. + \section{Analysis} \label{sec-analysis} @@ -490,23 +546,26 @@ and throughput and was used to evaluate selection device performance. A mixed design repeated measures analysis of variance model (MANOVA) was used for movement time and throughput to examine within subject -differences in target and size, as well as between subject differences -in device. A Greenhouse and Geisser correction of the F-ratio was used -whenever the Mauchly's test results showed that assumptions of -sphericity were violated. +differences in target type and size, as well as between subject +differences in selection device. A Greenhouse and Geisser correction of +the F-ratio was used whenever the Mauchly's test results showed that +assumptions of sphericity were violated. Post hoc tests, for multiple comparisons, were made using the Bonferroni -method. Due to the skew observed in the error rate data inter-device +method. Due to the skew observed in the error rate data, inter-device difference in error rates were assessed using the Mann-Whitney U Test. -The comfort questionnaire was based on a five point ordinal scale. In -general five indicated a bad rating. Because of the small data size, a +The comfort questionnaire was based on a five point ordinal scale, where +five generally indicated a bad rating. Because of the small data size, a Mann-Whitney (non-parametric) test was used. All statistical analyses -were performed using SPSS version 11.0. +were performed using SPSS version 11.0. A \emph{p}-value of \(< 0.05\) +was considered statistically significant. + \section{Results and discussion} \label{sec-results} + \subsection{Adjusting for learning} \label{sec-results-learning} @@ -516,12 +575,13 @@ \citep{Doug-SA-1999-CHI}. From analysing the results of movement time and throughput over each -test block, it is clear for the combo box and check box that learning -occurs from the first to second block with the touch screen (as seen in -Figure~\ref{fig-movement-time-learning}). Due to prior experience, no -learning is observed with the mouse. No learning occurs with the text -box or button most likely due to their large size and simple selection -behaviour. +test block, it was clear for the combo box and check box target types +that learning occurred from the first to the second block with the touch +screen, as seen in Figure~\ref{fig-movement-time-learning}. Due to prior +participant experience, no learning was observed with the mouse. No +learning occurred with the text box or button most likely due to their +relatively large size and simple selection behaviour. + \begin{figure} \centering @@ -532,194 +592,212 @@ \label{fig-movement-time-learning} \end{figure} + Statistical analysis using a simple repeated measure ANOVA was carried out on movement time for both the check box and combo box. For movement -time of the combo box, the effect of block * device was significant -\((F(1.549, 1335.219) = 4.373, p < 0.05)\). +time of the combo box, the effect of block \(\times\) device was +significant \((F(1.549, 1335.219) = 4.373, p < 0.05)\). Helmhert +contrasts showed that the differences between blocks became +non-significant after block 1 \((p > 0.05)\). -Helmhert contrasts show that the differences between blocks become -non-significant after block 1 \((p > 0.05)\). For movement time of the -check box, the effect of block * device was significant \((F(1.608, -1385.960) = 4.763, p < 0.05)\). - -Using Helmhert contrasts, the differences between blocks become -non-significant after block 1 \((p > 0.05)\). This again shows that -there was learning involved in block 1. +For movement time of the check box, the effect of block \(\times\) +device was significant \((F(1.608, 1385.960) = 4.763, p < 0.05)\). Using +Helmhert contrasts, the differences between blocks became +non-significant after block 1 \((p > 0.05)\), which again shows that +learning occurred in block 1. To account for learning with the combo and check box, results from block -6 only will be used to calculate the performance measures of measurement -time, throughput and error rate. The results from block 6 alone would -give a good measure of performance. +6 only were used to calculate the movement time and throughput measures, +as the results from block 6 alone gave a good measure of performance. +However, the error rate results were highly skewed for both target +types, with two participants accounting for almost 90\% of the errors. +Error rates for the check box and combo box were therefore calculated +using results from all six test blocks. + \subsection{Movement time} \label{sec-results-movement} The results showed that the mouse had an overall movement time of -\unit[1.3]{s} for all targets compared to \unit[1.6]{s} for the touch -screen. Therefore we can conclude that a mouse is on average 15.2\% -faster than a touch screen overlay. This is interesting as -\citet{Sear-A-1991-IJMMS} found that the movement time between a mouse -and touch screen (monitor) was similar for rectangle targets larger than +\unit[1.3]{s} across all target types compared to \unit[1.6]{s} for the +touch screen overlay. Therefore we can conclude that a mouse is on average +15.2\% faster than a touch screen overlay. This is interesting because +\citet{Sear-A-1991-IJMMS} found that the movement times for mouse and +touch screen monitor were similar for rectangular targets larger than \unit[2]{mm}. Therefore the nature of the two types of touch screen -(overlay and monitor) may affect the movement time associated with the -type of touch screen. It is also likely that due to the loss of accuracy -found with the overlay during testing, the touch screen monitor will -have a lower movement time compared to the touch screen overlay. +(overlay versus monitor) may affect the movement time associated with +the type of touch screen. It is also likely that due to the loss of +accuracy found with the overlay during testing, the touch screen monitor +will have faster movement times compared to the touch screen overlay. -The movement times for each target showed that the text box has the -fastest movement time, followed by the button, the check box and then -the combo box. These results are illustrated in -Figure~\ref{fig-movement-time} and follow Fitts' Law in that the largest -target (the text box) had the fastest movement time. +The movement times for each target type showed that the text box had the +fastest movement time, followed by the button, the check box and the +combo box. These results are illustrated in +Figure~\ref{fig-movement-time} and are consistent with Fitts' Law +\citep{Fitt-PM-1954-Law}, in that the largest target (the text box) had +the fastest movement time. + \begin{figure} \centering \includegraphics{movement-time} - \caption{Movement time for each target type across both devices and - all sizes.} + \caption{Movement time for each target type, averaged across both + devices and all target sizes.} \label{fig-movement-time} \end{figure} + As expected, the combo box had the slowest movement time due to the -two-click behaviour involved in making a selection. The sizes of the +two-step behaviour involved in making a selection. The sizes of the combo box were exactly the same as the text box but movement time was -119\% slower. Thus the extra movement of selecting an item from the drop -down list increases the movement time involved with the combo box -dramatically. As the distance to the list item is relatively short from -the main combo box area, the significant increase in movement time is +119\% slower. Thus the extra movement of selecting an item from the +drop-down list increases the movement time involved with the combo box +dramatically. As the distance from the main combo box area to the list +item is relatively short, the significant increase in movement time is therefore most likely due to users making more errors. -A touch screen has similar movement time to a mouse for medium and large -sized targets. But for the small targets, the touch screen was 67\% -slower than the mouse. The only time where the touch screen was found to -be faster than the mouse was with the largest target type - the large -text box. +A touch screen overlay has similar movement time to a mouse for the +medium and large targets, but for the small targets, the touch screen +overlay was 67\% slower than the mouse. The only time where the touch +screen overlay was found to be faster than the mouse was with the +largest target type---the large text box. -The movement time for the small check box with the touch screen was 69\% -slower than that of the mouse. The small check box was the smallest item -tested having a width of \unit[4]{mm} and height of \unit[4]{mm}. We can -conclude that the touch screen was not efficient for selecting targets -as small as \unit[4]{mm}. \citet{Sear-A-1991-IJMMS} showed a touch -screen has similar movement time to the mouse for targets as small as -\unit[2]{mm}. Although a touch screen monitor can be used with targets -as small as \unit[2]{mm}, a touch screen overlay should only be used for -targets with a size of greater than \unit[4]{mm}. The results from the -error rate analysis also support this +The movement time for the small check box with the touch screen overlay +was 69\% slower than that of the mouse. The small check box was the +smallest item tested, with dimensions of \unit[4]{mm} \(\times\) +\unit[4]{mm}. We can conclude that the touch screen overlay was not +efficient for selecting targets as small as \unit[4]{mm}. Compare this +with \citet{Sear-A-1991-IJMMS}, who showed that a touch screen has +similar movement time to a mouse for targets as small as \unit[2]{mm}. +While a touch screen monitor can be used with targets as small as +\unit[2]{mm}, a touch screen overlay should only be used for targets +with a size of greater than \unit[4]{mm}. The results from the error +rate analysis also support this conclusion (see +Section~\ref{sec-results-error-rate}). + \subsection{Throughput} \label{sec-results-throughput} Throughput for the mouse was \unit[1.238]{bps}, slightly higher than the -1.215 bps throughput for the touch screen. The device by itself was -shown not to have a significant effect on throughput \((F(1, 22) = 0.02, -p > 0.05)\). Throughput did not vary for size but throughput did vary -depending on target type \((F(2.07, 45.55) = 4.77, p < 0.001)\). Check -boxes had the highest throughput rate of \unit[1.967]{bps} (sd = 0.720). -This is interesting as the check box was shown to have the second worst -movement time and the worst error rate (see -Figure~\ref{fig-throughput}). +\unit[1.215]{bps} throughput for the touch screen overlay. The selection +device by itself was shown not to have a significant effect on +throughput \((F(1, 22) = 0.02, p > 0.05)\). Throughput did not vary for +size but throughput did vary depending on target type \((F(2.07, 45.55) += 4.77, p < 0.001)\). Check boxes had the highest throughput rate of +\unit[1.967]{bps} (sd = 0.720). This is interesting as the check box was +shown to have the second worst movement time and the worst error rate +(see Figure~\ref{fig-throughput}). + \begin{figure} \centering \includegraphics{throughput} - \caption{Throughput for each target type across both devices and all - sizes.} + \caption{Throughput for each target type, averaged across both + devices and all target sizes.} \label{fig-throughput} \end{figure} + Upon further investigation it was seen that the movement time for the -check box was in fact in the middle range of all targets and due to its -small size it had a high index of difficulty. Therefore these two -factors are the reason for the check box having such a high throughput -rate. The combo box had the worst throughput of \unit[0.501]{bps} (sd = -0.213). The index of difficulty was not very high for the combo box and -so it was due to its high movement time that the combo box had such a -low throughput rate. +check box was in fact in the middle range of all targets, and due to its +small size it had a high index of difficulty. These two factors are the +most likely reason for the check box having such a high throughput rate. + +The combo box had the worst throughput of \unit[0.501]{bps} (sd = +0.213). The index of difficulty was not very high for the combo box, so +its low throughput rate could be attributed to its high movement time. The overall throughput rate of \unit[1.2]{bps} for the mouse is much -lower compared with previous research. A study by -\citet{Doug-SA-1994-SIGCHI} showed a mouse had a throughput rate of +lower than that found by previous research. A study by +\citet{Doug-SA-1994-SIGCHI} showed that a mouse had a throughput rate of \unit[4.15]{bps}. \citet{Mack-IS-1991} compared three devices (mouse, tablet and trackball) using four target sizes (8, 16, 32 and 64 pixels) over two different types of tasks: pointing and dragging. The throughput -for the mouse in this case was 4.5 bps. This may indicate the level of -difficulty with selection within this experiment is a lot higher than -within previous research. This could be due to the selection of GUI -targets instead of arbitrary rectangle targets. +for the mouse in this case was \unit[4.5]{bps}. This may indicate that +the level of selection difficulty in our experiment is higher than in +previous research. This could be due to the selection of GUI targets +instead of arbitrary rectangular targets. + \subsection{Error rate} -\label{sec-results-errorrate} +\label{sec-results-error-rate} The error rate for the mouse was only 2.7\% which is consistent with -previous studies. The touch screen on the other hand had an error rate -of 60.7\%. \citet{Sear-A-1991-IJMMS} found that the touch screen had -an average error rate of 49\% but this was across much smaller targets. -This suggests there is a loss in accuracy from using a touch screen -overlay compared to a touch screen monitor. +previous studies. The touch screen overlay, on the other hand, had an +error rate of 60.7\%. \citet{Sear-A-1991-IJMMS} found that the touch +screen monitor had an average error rate of 49\% but this was across +much smaller targets. This suggests that there is a loss in accuracy +from using a touch screen overlay compared to a touch screen monitor. -The check box had a significantly high amount of errors; 78.5\% for all -sizes and both devices and in particular, 312.5\% for the small check box -with the touch screen. A 100\% error rate indicates one wrong selection -made for every correct selection. The touch screen incurred the majority -of the errors. With the check box the mouse had an error rate of 4.4\% -and the touch screen had an error rate 152.5\%. The distinguishing factor -of the check box compared to the other targets was its small size. We -can conclude from this that the touch screen overlay has inaccuracy in -selecting small targets (\unit[4]{mm} or less). +The check box had a significantly high error rate: 78.5\% for all sizes +and both devices and in particular, and 312.5\% for the small check box +with the touch screen overlay. The touch screen overlay incurred the +majority of the errors. With the check box the mouse had an error rate +of 4.4\% and the touch screen overlay had an error rate of 152.5\%. The +distinguishing factor of the check box compared to the other target +types was its small size. We can conclude from this that the touch +screen overlay is more inaccurate for selecting small targets +(\unit[4]{mm} or less). Buttons and text boxes had much lower error rates compared to that of the check box and combo box (as seen in Figure~\ref{fig-error-rate}). As buttons and text boxes also had low movement times, we can conclude that these two targets have very good overall performance. + \begin{figure} \centering \includegraphics{error-rate} - \caption{Error rate for each target type across both devices and all - sizes.} + \caption{Error rate for each target type, averaged across both + devices and all target sizes.} \label{fig-error-rate} \end{figure} +Note that the error rate calculation for combo boxes assumes a +``two-click'' selection approach, as only two of the twenty-four +participants used the ``one-click'' approach. Both of the ``one-click'' +participants used the mouse. + + \subsection{Comfort} \label{sec-results-comfort} In terms of accurate pointing the mouse (2.083) was rated easier than -the touch screen (3.000). These differences were statistically +the touch screen overlay (3.000). These differences were statistically significant \((p < 0.01)\). The responses regarding the question on -neck, wrist and arm fatigue showed that the touch screen had a high -rating (4.083), whereas the mouse was rated in the midpoint range +neck, wrist and arm fatigue showed that the touch screen overlay had a +high rating (4.083), whereas the mouse was rated in the midpoint range (3.167). These differences were statistically significant \((p < 0.5)\). The final question rated the overall difficulty in using the selection device. The mouse (4.250) was rated easier to use than the touch screen -(3.333). These differences were statistically significant \((p < +overlay (3.333). These differences were statistically significant \((p < 0.05)\). -For user satisfaction with the touch screen, both the text box and -button were rated easy to accurately select and the preferred size was -both the large size and the medium size. This feedback is consistent -with the data collected in that text boxes and buttons have short -movement time and low error rates (easy to accurately select). +Participants using the touch screen overlay rated both the text box and +button as easy to accurately select, with the large and medium sizes +being most preferred. This feedback is consistent with the data +collected in that text boxes and buttons have short movement time and +low error rates (i.e., they are easy to accurately select). The combo +box was rated in the middle of the range, and the check box was rated as +very hard to select. Three quarters of the touch screen overlay +participants preferred to select large combo boxes and check boxes, +which is consistent with the poor error rates and movement times +associated with these two target types on the touch screen overlay. -The combo box was rated in the midpoint range in terms of ease in -accurately selecting with the touch screen and the check box was rated -very hard to select. Three quarters of the touch screen users preferred -to select the large size combo boxes and check boxes and this reflects -the poor error rates and movement times associated with these two -targets with the touch screen overlay. +Participants using the mouse rated the text box and button easy to +accurately select, with the large and medium sizes being most preferred. +Both the combo box and check box were rated harder to select than the +button and text box with the check box having the worst rating. As with +the touch screen overlay, participants preferred large combo boxes and +check boxes. -The participants using the mouse rated the text box and button easy to -accurately select with the medium and large sizes being the most -preferred. Both the combo box and check box were rated harder to select -than the button and text box with the check box having the worst rating. -Like the touch screen overlay, the preferred size for the combo box and -check box was large. - -One participant noted the lack of arm support for targets at the top of -the screen. This is an interesting comment because the nature of using a -touch screen means the users arm might be raised off the desk and be -self supporting when selecting items towards the top of the desktop -screen. +In the general feedback, one participant noted the lack of arm support +for targets at the top of the screen. This is an interesting comment, +because the nature of using a touch screen means the user's arm might be +raised off the desk, and thus be self-supporting when selecting items +towards the top of the screen. Another suggestion was making the target change colour when the cursor is located above it. This is a similar concept to that of interactive @@ -727,48 +805,121 @@ shown to affect the speed and accuracy when making a selection \citep{Bend-G-1999-PhD}, and so it likely the visual feedback received from GUI targets will affect the selection performance. All the targets -being tested provide some form of immediate visual feedback from the -button being visually depressed it to a tick appearing in the check box. -Future study is needed to assess how visual feedback affects selection -performance and what the most effective method of providing feedback is. +that were tested provide some form of immediate visual feedback, from +the button being visually depressed to a tick appearing in the check +box. Further study is needed to assess how visual feedback affects +selection performance of GUI items and what the most effective method of +providing feedback is. + + +\subsection{Other findings} +\label{sec-results-other} + +Calculating the standard deviation of the final selection coordinates on +the screen revealed two interesting patterns. First, when selecting text +boxes with the mouse, there was greater variation in final selection +coordinates on the right side of the screen (\(45^{\circ}\) and +\(315^{\circ}\) target angles) than on the left side of the screen +(\(90^{\circ}\) and \(225^{\circ}\) target angles). This behaviour was +not apparent when selecting text boxes with the touch screen overlay, as +illustrated in Figure~\ref{fig-variation-textbox}. This would mean that +participants using the mouse were much more careful in making selections +with text boxes on the left side of the screen than on the right. + + +\begin{figure} + \centering + \includegraphics{variation-text-mouse} + \includegraphics{variation-text-touch} + \caption{Standard deviation of final selection coordinates for the + text box, averaged across all target sizes.} + \label{fig-variation-textbox} +\end{figure} + + +Second, and even more interesting, was the observation that while +selections made on the combo box with the mouse exhibited similar +behaviour to the text box, the behaviour on the touch screen overlay was +more or less the opposite, as shown in +Figure~\ref{fig-variation-combobox}. That is, selections of combo boxes +on the left side of the touch screen overlay had greater variation than +selections made on the right. + + +\begin{figure} + \centering + \includegraphics{variation-combo-mouse} + \includegraphics{variation-combo-touch} + \caption{Standard deviation of final selection coordinates for the + combo box, averaged across all target sizes.} + \label{fig-variation-combobox} +\end{figure} + + +These effects were most pronounced for the large size of both the text +box and the combo box. The variation of selection coordinates for the +smaller targets (the button and check box) were consistent across both +the mouse and touch screen overlay and for all target sizes. + +We can only speculate as to the causes of this variation. In the case of +the combo box, one possibility is its asymmetrical appearance compared +to the other target types, which may encourage participants to try to +click specifically on the drop-down arrow of the combo box, rather than +clicking on the combo box as a whole. However, this does not explain the +variation between left and right sides of the screen, nor why the same +behaviour was observed with the completely symmetrical text box. + +Another possibility is the handedness of the participants, which in the +case of a touch screen might affect how difficult it is to select +targets on different sides of the screen. Unfortunately, we did not ask +participants whether they were right- or left-handed and thus can draw no +conclusions on this point. + \section{Conclusions} \label{sec-conclusions} -The goal of the study was to assess the ability of a touch screen -overlay in selecting different targets commonly presented to users in an -information system. The touch screen overlay sits over a normal monitor -and results in a gap between the overlay and monitor itself. This gap -was shown to decrease the accuracy of the touch screen. +The goal of our study was to assess the efficacy of a touch screen +overlay compared to a mouse, when selecting the typical GUI targets +commonly presented to users in desktop information systems. This was +achieved by an experiment measuring movement time, throughput and error +rate for various combinations of target type, size, angle and distance. +Comfort and user satisfaction were assessed by means of a questionnaire. The results showed that the touch screen overlay was both slower and -less accurate than the mouse. The touch screen was found to have -reasonable performance with large GUI items but poor performance with a +less accurate than the mouse. The touch screen overlay was found to have +reasonable performance with large GUI items but poor performance with smaller GUI items. The touch screen overlay did have comparable movement times to the mouse for medium and large sized targets. Throughput did -not vary across device or size but did vary across target. Both -selection devices had the same user preference except with the smallest -target, check boxes, in which the mouse had a higher preference. The -mouse was rated easier to make accurate selections with than the touch -screen. The touch screen overlay also has worse arm, wrist and finger -fatigue compared to the mouse. From these results we can conclude that -the mouse had higher user satisfaction than a touch screen. +not vary across device or size but did vary across target type. Both +selection devices had the same user preference except with respect to +the smallest target type---check boxes---in which the mouse had a higher +preference. The mouse was rated easier to make accurate selections with +than the touch screen overlay. The touch screen overlay also had worse +arm, wrist and finger fatigue compared to the mouse. From these results +we can conclude that the mouse had higher user satisfaction than a touch +screen. + +An unusual variation in final selection coordinates was noted for both +text boxes and combo boxes. Further studies are required to establish +whether this is a consistent phenomenon and if so, to identify why this +variation occurs. In general we can conclude that a touch screen overlay with no external -device (i.e. pen) is not an effective selection device for targets -having a size of \unit[4]{mm} or smaller. When designing interfaces that -will be used with a touch screen overlay, selection within the interface -will be more efficient if the GUI items are larger than \unit[4]{mm}. +device (e.g., a pen) is not an effective selection device for targets +with dimensions of \unit[4]{mm} or smaller. When designing interfaces +that will be used with a touch screen overlay, selection within the +interface will be more efficient if the GUI items are larger than +\unit[4]{mm}. Although the results showed that the touch screen overlay was not -efficient and usable for selecting items with a size of \unit[4]{mm} or -less, this may not be the case when a pen or some external device is +efficient and usable for selecting GUI items with a size of \unit[4]{mm} +or less, this may not be the case when a pen or some external device is used in conjunction with the touch screen overlay. In general there -seems to a lack of research done in device assessment with touch screens -and pens or other external devices. Further testing on touch screens +seems to a lack of research done in device assessment of touch screens +with pens or other external devices. Further testing on touch screens used with an external device such as a pen may well show that a touch -screen overlay is adequate and efficient for selecting small items -(\unit[4]{mm} or less). +screen overlay is adequate and efficient for selecting small GUI items. % The Appendices part is started with the command \appendix;