Preliminary Evaluation of Ambient Visualizations’

This user study evaluated different visualizations being representatives of a design space comprising the two dimensions "degree of abstraction" and "amount of information". Collecting quantitative and qualitative data, a descriptive statistical analysis was performed. Based on the results, the subjectively perceived best visualization will be concluded and thus, deployed it in a field study to provide ubiquitous feedback on a desk worker’s cognitive load.

Figure 6.5: Two-dimensional design space illustrating the degree of abstraction on the x-axis and the amount of information on the y-axis.

The nine following visualizations represent different parts of the spectrum characterized by the two dimensions: (a) Speedometer, (b) Thermometer, (c) Working Head, (d) Question mark-Head, (e)Smoking Head, (f) Personalized Entity, (g) Multiplying Circles, (h) Colored Circle, and (i) Scaling Circle.

Study Design and Measures All visualizations were presented to each participant following a within-subject design. The order of visualizations was randomized according to Latin square and for each visualization the following five dependent variables have been assessed on a Likert-item scale ranging from 1 (=I do not agree at all) to 4 (=I fully agree):

Intuitive Comprehensibility The statement "The visualization is comprehensible." referred to the effort a user has to make to understand what a visualization is showing. This aspect has been derived from Nielsen’s [187]10 Usability Heuristics for User Interface Designwhich phrase recommendations to make a system intuitively comprehensible.

Subjective Distraction Asking about the distraction, the statement was phrased as follows: "The visualization is not distracting." assessing if the user was prevented from concentrating on something. According to Kirkham et al.

[137] this requirement was particularly valuable for deployments in the workspace.

Unobtrusiveness While the latter aspect considered the visualization’s potential to annoy users, obtrusiveness refers to the ambient quality of it. Hence, it was stated "The visualization is unobtrusive". As can be seen in prior work [64, 113, 290], many visualizations aim to be unobtrusive for users.

Aesthetic Pleasantness Since also visually appealing factors play a role in the evaluation of visualizations- even if it is subconsciously judged, the statement "The visualization is aesthetically pleasant." has been included in the assessment. Hereby again one of Nielsen’s [187] 10 Usability Heuristics for User Interface Design, namely theaestics and minimalist designhave been included in the assessment.

Sufficiency of Information Content The last aspect which was rated refers to the amount of information being presented in the visualization. It has been identified as an important criteria in related research [212]. To get a sense for whether the data has been presented without too much detail but with sufficient information to be able to be understood, the final statement was

"The information content of the visualization is sufficient".

Furthermore, the participants were asked to provide qualitative feedback on each visualization. Therefore, ten questions were asked partly reassessing what has been inquired through the Likert-item scales. Accordingly, the first two questions asked what was understood from the visualization and in how far the visualization has been understood intuitively. Then, the participant should explain his or her impression of the visualization and what is being associated with it.

Referring to it’s potential to distract the users, the participant was asked in how far the visualization has not been distracting first, before asking in how far the visualization has been unobtrusive. Afterwards a similarly phrased question considered the aesthetic pleasantness. Lastly, two questions addressed the amount of information content. While the first question: "Do you miss any information in this visualization?" referred to the sufficiency, the latter question targeted to assess

"Which information do you perceive as redundant?". Finally, the participants were asked if they would like to add some additional comments. The qualitative assessment part was complemented by the individual rating of the three best visualizations given the order 1, which is the best and 3 being the least best. If

they preferred, the participants could add explanations why they had chosen this sequence for these triple of visualizations.

Inducing and Measuring Cognitive Load Since the ambient visualization should provide feedback on the user’s cognitive load, the so-called N-back task was used to elicit cognitive load in the users. Among the different realizations that have been applied in prior research and shown to induce cognitive load effectively [117, 196], for this study a sequence of numbers ranging from one to nine had been played to the participant only on audio basis using headphones.

Without knowing when this sequence ends, the task was to remember the n^th element of the sequence and speak out loud what the it was. Hereby, five N-back tasks ranging from 1-back to 5-back have been performed in difficulty increasing sequence, to avoid potential feelings of overload among the participants resulting in dropouts. For measuring physiological data, namely heart rate variability (HRV) has been recorded throughout the performance of the N-back task. This physiological measure has shown to be a reliable indicator of cognitive load [161, 164, 174, 247]; while high HRV values signify a low mental demand, low values indicate high cognitive load. To ensure that participants felt cognitively challenged, their experiences regarding mental, physical, and temporal demand, as well as performance, effort, and frustration should be rated using the NASA Task Load Index (NASA-TLX) [103].

Data Transfer The physiology recording wearable Empatica E4¹³ has been attached to the participants’ wrist. The device stored the physiological data locally and transferred them via Bluetooth Low Energy to the Android

"E4 realtime" mobile application ¹⁴ running on a Motorola Moto G3 smartphone. After each sub-task of the N-back task had been completed, the session data was uploaded from the mobile application to the cloud-based

"E4 Connect Web Dashboard" repository¹⁵being requested from a Sony Vaio laptop. Via the user interface a CSV file with the raw data could be downloaded.

Data Processing For the processing of the heart rate values being recorded with a sample frequency of 64 Hz, the R-R interval values were used to calculate the rMSSD value and threshold values to be used for visualizing the cognitive load. The calculation of the rMSSD value follows the formula

13 https://www.empatica.com/research/e4/

14 https://play.google.com/store/apps/details?id=com.empatica.e4realtime

15 https://support.empatica.com/hc/en-us/sections/200002718-E4-connect-Web-Dashboard

6.3.1 being frequently used in related work [161, 164, 165]. Hereby, the rMSSD is defined as the square root of the averaged sum of all squared neighboring values (t_i). Thus, the rMSSD value refers to the time in milliseconds between two heart beats [247].

Σ_sd= s 1

n−1

n i=1

∑

[t_i+1−t_i]² (6.1)

Participants and Procedure 18 individuals (M=22.7,SD=2.72 years) among them three females participated in the study. They were recruited via personal contact or mailing list. The study took place in a quiet room and took between 90 and 120 minutes per participant depending on the individual length of the qualitative part. Participants received 5 Euro per each 30 minutes, meaning at least 15 Euro for their participation.

Before the user study started all participants were explained the purpose and the sequence of the study addressing particularly the collection of physiological data.

After the participants had provided their consent, the Empatica E4 wristband was attached to their non-dominant arm. Then, they were asked to put on the provided headphones and to relax for two minutes, while physiological data had been recorded as a baseline measurement. Subsequently, the cognitively demanding N-back task was explained in detail; each participant started with the 1-back task followed by the incrementally increasing more difficult task up to the 5-back task. For reassuring that cognitive load had been induced, the NASA-TLX was administered after each performed N-back task using an online version¹⁶for the assessment. When the last N-back task had been accomplished, all physiological data was exported from the Empatica E4 to CSV files and processed as described in Section 6.3.2 resulting in a cognitive load value given in percent which needed to present the ambient visualizations. Accordingly, the participants could take a short break before the nine visualizations were shown in randomized order.

Each visualization was presented showing the full spectrum of cognitive load that had been perceived throughout the preceding task ranging from the individually determined minimal value to the maximum value. For each visualization the five dependent variables were assessed quantitatively using the Likert-item scales, and qualitatively asking the questions set (cf., Section 6.3.1). All interviews were audio recorded given the participants’ consent. The entire study procedure is illustrated in Figure 6.4.

16 http://jensgrubert.bplaced.net/nasa-tlx-short/TLX-English-short.html