HTML Mockup

As a second step, two HTML mock-ups visualizing the design variants Level- and Gran-ularityTable were built to enhance the user’s possibility of interacting with the system.

The lessons learned as a consequence of the former evaluation were implemented in this version. By now, the users were able to get direct feedback when invoking an action.

In October 2002 the html-mockups were tested by several users (n=8) from the expected

INVISIP target user group of civil engineers from traffic planning authorities and pri-vately owned planning offices. Although there were individual differences in working and searching habits, the majority stated that they had very high expectations concerning the work with geometadata and that they expected higher efficiency with a working meta-data browser. The most popular characteristics of geometameta-data were cost-effectiveness and the potential for rationalizing daily work tasks with geodata.

The browser should give immediate access to geometadata and provide enough flexi-bility to adapt to the various tasks in a spatial planning process. The most important geodata-attributes corresponded to maps, e.g. coordinate system, scale and precision, but language, year and origin of the dataset were highly relevant as well.

After the pre-test questionnaire and a video introduction, the users were handed a script with test tasks to work on. A test monitor and a recorder conducted and documented the test, while a video camera and a screen recording took footage of the user and his ac-tions on the screen. This proved to be especially helpful (though very time-consuming) for re-evaluating critical situations in the test, where we could view and analyze the two synchronized videos.

Users approved of the two design variants. In particular, the granularity table led to some surprisingly emotional responses from those working with it. For example, when users found out that they could change the granularity of a document by a context menu in the Scatterplot, most were surprised and pleased with this feature. Comments varied from ”very clever” to ”unusual, but I like it” (see Figure 6.4 left side). This, and the possi-bility of scrolling through an extended abstract in granularity level 4 (see Figure 6.4 right side), were critical from our point of view, but approved of by users in this demonstrator’s interaction design.

(a) (b)

Figure 6.4: HTML Mock-Up: The Scatterplot allows the level of document detail to be changed (left), a scrollable abstract in the GranularityTable provides further information (right)

In the posttest, users were in favor of the GranularityTable, but our detailed analysis

showed that their actual performance did not conform with this view. Users had more problems in fully understanding the interaction concepts of the granularity design than those of the level design. We assume that users were positively influenced by the clearer and more aesthetic and appealing design of the GranularityTable, and therefore gave it better ratings than the LevelTable. We can rule out learning or last-item remembering effects. Nevertheless, this bias in the test design had to be eradicated for the next user tests. Both demonstrations lacked intuitiveness concerning the keyword highlighting and relevance attributes. To our surprise three (SuperTable) and even four (Granularity Ta-ble) users had problems in matching the designated colors with the keywords and their relevance values. Even though this aspect of the metadata browsers was explained to the users in a video before the test, the conversion of this knowledge into their mental model of working with a metadata browser seemed difficult.

Another problem was the SegmentView with its stacked columns or TileBars, which in both demonstrations gives an overview of the segmented and rated document. An exam-ple of the SegmentView with stacked columns from the LevelTable is seen in Figure 6.5, top; the TileBars can be seen in Figure 6.5, bottom.

Figure 6.5: SegmentView in StackedColumn form at the top (each bar is a text segment, relevance is shown by size and colour) and in TileBar form at the bottom (each line is a text segment, relevance is shown by saturation. Navigation in text is by the upper left arrow)

It did not occur to users that a segmented document was being visualized. None of the eight users understood this concept within the LevelTable, and only two users managed to work with this in the GranularityTable. In the latter, users were more open to trying to work with it after we explained the interaction. Then, some also rated this interaction with positive attributes like ”clever” and ”very usable for longer texts”. This was in con-trast to the LevelTable, where most users showed little interest in the feature, or wanted to work with it at all. We suggest that the vertical alignment of the TileBars of the Gran-ularityTable seems more like a ”normal” document structure (top down, left to right) and is therefore more easily accessible by users for the reception of a text.

During the test and also at the posttest, users frequently criticized a lack of connection between the Scatterplot and the corresponding SuperTable (both design variants). A dual visual response when documents were marked or the level of detail was changed seemed to be insufficient; users requested very tight coupling of both views.

In parallel with the lab tests, we started a web-based evaluation . The target user group were people involved in spatial planning, and comprised mainly of co-workers from the project or their colleagues. Questions were asked about individual search behavior, about a virtual search with the two design variants, how the users would interact with them, where they expected which interaction, and what they would like to be different. The par-ticipants were asked to download two short introductory videos, and several screenshots correlating to the tasks. The sequence of Level-/GranularityTable was randomized to ex-clude learning and last-item remembering effects. 35 users completed the questionnaire, from which 31 were put into the final evaluation.

Although screenshots are even more limited than the prototypes of the lab evalua-tion, some results from the former evaluation were confirmed. Throughout the test, the performance (measured in correct answers regarding interaction) was higher with the Lev-elTable than the GranularityTable. A lack of connection between the table visualizations and the Scatterplot was also frequently criticized.

An interesting result came from the analysis of search behavior and preferences in de-sign. With five separate questions concerning typical search tasks, we wanted to classify the users as being of the ”more analytical” or ”more browsing search-strategy”’ type [Mar95]. As might be expected, a mixture of both strategies dominated the sample. Only eight users had very clear preferences; five of them we categorized as ”only browsing strategy”, three of them as ”only analytical strategy”. Interestingly enough, four of the first category definitely preferred the GranularityTable, and all three of the second cate-gory preferred the LevelTable. We assume that, at least for the first steps of an iterative search process, the LevelTable can be efficient for analyzing the result set as a whole; it can find patterns, or reformulate/discard the query in the event of unsatisfactory results.

The primary goal is not content, but filtering and reduction of the result set. If the re-sults are then narrowed down to potentially interesting documents, the GranularityTable with its browsing comfort can be used. Now content is the primary goal; modalities can be changed frequently. In this manner, our initially-developed scenarios were partly vali-dated by empirical results, though our scenario characters begin the planning process with only analytical and very formal queries formulated in a sophistical manner, while during the planning (and possibly the iterative retrieval) process they become more informal and data driven.

Although this evidence should not be given too much weight, we took it as a sig-nal to handle both design variants equally and to further integrate them into the overall SuperTable/Scatterplot framework. A first step was to implement both versions in Java.

Thus, the VisMeB system was introduced. All visualizations and proposed interaction techniques were implemented to provide the possibility of working with them in a run-ning system. The next step should be to integrate both table ideas into a single one. The aim was to present an even more convenient way of handling the problems we were con-fronted with, such as e.g. the change of detail levels, or the complete integration of all

information-seeking process steps into a single system in a meaningful manner. Thus, the MediaGrid was developed. It combines the characteristics of LevelTable and Granu-larityTable as well as newly-introduced features to build a self-contained system. Again, different variants named MediaGrid and MovieVis were developed. This provided the opportunity to test diverse approaches.