Participatory Design

In order to integrate users into the design process, a qualitative user study was conducted at an early stage of the development of the case study.

The primary goal of this study was to evaluate the comprehension of the developed blends.

Therefore, user understanding of the navigation, orientation, and content presentation concepts in the fundamental blend ZOIL were observed in particular. The blends search and

directed to the integration of search results into the information landscape, where we expected problems of overlapping. Due to the early stage of the development, the blends virtual windows, annotations and hybrid medium were not part of the user study.

As a secondary goal of the user study, we analyzed the portability of the UI and interaction concepts on different devices. We also attempted to obtain insight into the quality of user experiences.

Participants and study design: The user study was carried out with 11 participants (7 female, 4 male), ranging in age from 21 to 27. The participants were students and teaching staff from the University of Konstanz in Law, Linguistics, Politics, and Economics. The participants were divided in two groups: One group (7 participants) performed the user study on a PC in a usability lab; the other group (4 participants) completed the study at a large, high-resolution display, simulating the public wall of the media library described in the scenario above.

The PC condition used a 21-inch TFT-display with a screen resolution of 1280x1024 controlled by a mouse and keyboard. A mouse click on a media object in the information landscape triggered an automatic zoom to the corresponding object. Alternatively, the users were able to zoom continuously using the mouse wheel and could pan with the middle mouse button. In contrast, the high-resolution display consisted of two seamless rear-projection screens with an aggregate resolution of 3840x1080 pixels (Figure 18). This setting was operated by an infrared laser-pointer (König et al. 2007), which allowed absolute pointing, in addition to a standard keyboard. To guarantee a fair comparison between the devices, the laser-pointer had buttons and a wheel similar to the mouse and therefore offered the same zooming and panning functionality as the PC condition.

Screen captures and video streaming from a camera that monitored the participants during the user study were gathered. An observer situated in the next room had live access to these streams and systematically took notes.

To reduce the influence of the experimenter on the results, the study was standardized and structured according to established principles. The user study was divided into three stages:

(1) an introductory, exploratory stage that allowed the participants to freely interact with the system without biases. In this stage, the participants were asked to “think aloud”. As soon as they felt confident in their interaction with the system, they were to describe their mental model of the system. (2) After this first stage, a video that demonstrated all sytem features

was shown. Subsequently, the participants executed 15 given tasks with the prototype. At the end of this stage, the participants were asked to complete an AttrakDiff questionnaire (Hassenzahl et al. 2003) to determine the quality of the user experience. (3) In the third and final stage of the user study, we conducted an interview in which participants were asked both closed- and open-ended questions to validate the observed impressions.

Results: The qualitative results reported below are derived from observations of the participants’ interactions as seen in the video material as well as from the participants’

statements during the explorative, task, and interview stages of the user study.

ZOIL – The navigation of the information landscape with zooming and panning was generally considered positively, but during several tasks difficulties and comprehension problems occurred. All participants preferred a direct and automatic zoom per click for the navigation. The continuous zoom triggered by the mouse wheel was regarded as a supplementary alternative. Some participants detected this feature late or did not detect it at all. In the interviews, several participants remarked that initially the navigation did not match up to their expectations. According to the participants’ statements, one factor that contributed to this perception was the zoom speed during the automated zoom triggered by a mouse click; this was regarded as being too fast.

Neither the basic layout nor the orientation of the information landscape caused any problems during the free exploration stage or during the task stage. The participants described the layout of the landscape as “clearly arranged” and “reasonable”. In contrast, the visualization of individual media objects created certain problems. Some participants stated that the information was presented in a way that was too “unstructured”. During the interviews, demand emerged for better organization of the presentation of information inside a media object, possibly via separation of different aspects of interest in visually divided regions.

Search – All participants understood the blend search immediately in the exploration stage;

however, the user study uncovered several problems. During the tasks, four participants complained about overlapping result objects and the consequent resulting lack of overview.

Two participants explicitly addressed this issue in the interview session. In addition, a textual list previewing results directly at the search input field was requested, which contradicts the idea of a seamless integration of results into the information landscape. One participant additionally asked for visualization of the search results inside a new portal in which filter

Portals – In the exploration stage, only four participants took notice of the blend portals and the connected visualizations. However, after watching the introduction video, all participants were able to solve the tasks with the help of portals without difficulties. During the interview sessions, portals were described as “comprehensible” and “reasonable” by several participants.

The concept of “magic lenses” was also understood and applied without errors after participants viewed the introduction video. In contrast, portal storing and locking remained unclear for some participants. According to the interviews, the reason for this issue was the inappropriate icon that was not recognized and interpreted in the intended way.

Different devices – Although the two groups executed the user study on different devices, they demonstrated very similar reactions. No differences could be detected in comprehension of the concepts. Handling of the system using mouse and keyboard was immediately accepted.

In contrast, handling via laser pointer was at first new and unfamiliar for the participants.

However, after a short training session during the exploration stage, participants had no difficulties working on the tasks using the laser device.

User Experience – An additional goal of the study was to gain insight into the user experience of the system. An AttrakDiff single evaluation was conducted (this evaluation can only show tendencies, due to the small number of participants). We also explicitly asked questions concerning the hedonic quality and visual design during the interview sessions. The participants assessed the visual design of the system as “very attractive” and “convincing”.

These results correspond to the results obtained from the AttrakDiff evaluation. In terms of the hedonic qualities (HQ) “attractiveness”, “identity”, and “stimulation”, the system is situated far above the average. The pragmatic qualities (PQ) achieved average values. This corresponds again to the outcome of the tasks and interviews, which identified opportunities for improvement in terms of the usability of the system.

Discussion of results: In summary, the developed blends in this case study are generally appropriate for the implementation of a Blended Library. This is true to a great extent even for the hierarchically structured blends search and portals. The blend search was comprehended very well, but as expected the overlapping of results must be resolved.

For the further development of the system, the input spaces as well as the presentation of blends should be deliberately selected. This issue emerged as an essential factor in the ease of learning for the blends. The search blend was detected and comprehended immediately during the exploration stage; in contrast, most of the participants did not encounter the concept of portals until after they had seen the introduction video. Furthermore, it was determined that

details such as the zoom speed or the organization of meta-data inside a movie object are important to understanding the system and its interaction model. This first explorative and qualitative user study showed the power of conceptual blending as a conceptual tool during the design process.