“Interactive, direct-touch digital tables are an emerging form factor with largely immature user interface design” [SRF+06]
Such interactive, digital tables operated by direct touch input are commonly called tabletops. Tabletops provide benefits over traditional displays in various ways [SRF+06]:
1. Tabletops unify display and input surface and thus are direct input devices [Hin02].
Direct input devices allow interface elements being manipulated by directly touch-ing them. Such ability aids users in accomplishtouch-ing tasks. In detail, direct touch interaction reduces the “gulf of execution” [HHN85], “the gap between a user’s goals and the means to execute these goals” [JGH+08]. With direct touch inter-action this gap is considerably lower than e.g. with traditional mouse interinter-action.
For example, a user attempting to hit a button with a mouse would have to grasp the mouse, detect the cursor on the screen, move the cursor to the button and click the left mouse button. Thereby the user also has to map two different planes (vertical display and horizontal mouse base) and different movement speeds (mouse’s and cursor’s). With direct touch interaction the user would merely have to touch the button with a finger.
2. Besides having an interactive display surface, tabletops benefit from being tables.
In order to communicate, discuss and collaborate, people tend to gather around tables [SRF+06]. Tables are found in “homes, offices, command-and-control cen-ters, caf´es, design centers, showrooms, waiting areas, and entertainment centers”
[SRF+06] and, amongst others, have the purpose of brining people together. This can be exploited:
“... a horizontal tabletop surface provides opportunities for building and enhancing colocated collaborative environments” [SRF+06]
3. Tabletops are usually larger than desktop displays (the tabletop prototypes at the
8
Human-Computer Interaction Group of the University of Konstanz have a screen diagonal of 34" [RSF+08] and 70", Microsoft Surface is sized 30" [Mic08], the MERL DiamondTouch is sized approx. 40" [FWSB07]). This size “can positively influence working styles and group dynamics”, be employed as “external physical memory” and can “serve as an external cognitive medium for new forms of visual representation” [SRF+06].
Figure 3.1: The tabletop prototype at the Human-Computer Interaction Group of the University of Konstanz. The user is currently entering text using our Column Typing technique (see section 5.2.2.3)
Tabletops provide opportunities for fascinating novel user interfaces benefiting from direct touch interaction. However, touch interaction techniques differ from traditional mouse interaction considerably in matters of motor skill required:
• User are enabled to select interface elements directly with their fingers rather than by positioning the mouse.
• User interfaces may exploit the human’s ability to use both hands parallel. With mouse interaction, users act purely unimanual.
• Interfaces attempting to gain benefit from transferring real-world behavior to in-teraction techniques might require specific motor skills. These skills may already be existing, yet it is unclear to what extent they can be transferred into the touch interaction domain.
Hence, insight into the human’s motor skill of controlling body parts relevant for touch interaction is required. In section 4 we deal with the topic of direct touch interaction as well as we provide insight into issues related to bimanual input.
As the tabletop domain emerges, more and more user interfaces designed specifiably for touch input are likely to be developed. However, applications whose interface is designed for traditional mouse interaction techniques still take the largest part of existing user software. Supporting such existing applications is essential for making a diverse application landscape in the tabletop domain possible:
“Still, legacy applications are widely deployed and many are indispensable for real-world tasks. A digital tabletop environment must therefore address issues related to using preexisting applications on a horizontal workspace”
[SRF+06]
Most legacy applications are WIMP (i.e. windows, icons, menus and pointing device) user interfaces. Current operating systems (Microsoft Windows, Apple MacOS, Linux GUIs) follow the WIMP paradigma. Consequently, the majority of applications running on these operating systems are WIMP user interfaces, too.
Yet, little research has been done in the field of transferring WIMP user interfaces into the tabletop domain. The only existing compound framework developed for this purpose, Fluid DTMouse [ER06], suffers several drawbacks:
• Fluid DTMouse provides no technique of entering text. Yet, “text entry is a vital part of day-to-day computing familiar to most people” [HHCC07].
• Fluid DTMouse requires users to learn two modes for placing the cursor. In normal mode the cursor is set directly beneath the finger. In precision mode the cursor is set in the middle of two fingers. Hence, users have to rethink from an absolute technique to a relative technique when changing modes.
• Fluid DTMouse’s dragging technique depends on the mode for placing the cursor.
In normal mode a dragging is initiated as soon as a single finger is placed on the the display. In precision mode the user needs to tap with a third finger in between the two already placed on the display. This inconsistency is likely to irritate users.
Moreover, starting a dragging directly at the moment a finger touches the display is not conform to the standard VDI/VDE 3850 Blatt 3 - User-friendly design of
useware for machines - Design of dialogues for touchscreens [Dah06].
• Fluid DTMouse provides no visual clues about the way of performing its techniques to the user. Thus, users are likely to need extensive training time until they are able to use Fluid DTMouse “blindly”.
For these reasons we designed and implemented novel interaction techniques for con-trolling WIMP user interfaces by touch input. In detail, we designed and implemented techniques for the following tasks:
• Selections: Selections are one of the most basic tasks WIMP user interfaces re-quire the user to perform. We review existing techniques and present ZoomTap, a technique allowing user to magnify a small screen portion for enlarging a target and thus enhancing selection precision. The topic of selection tasks is dealt with in section 5.1.
• Text input: Entering text is essential for operating WIMP user interfaces:
“Text entry is one of the most frequent actions we undertake when working on desktop computers. Entering text is necessary for activi-ties that require elaborate text compositions such as coding programs, authoring articles, or writing emails, as well as in situations that demand taking notes,typing in commands, or annotating content”
[HHCC07]
We designed, implemented and evaluated two alternative text input techniques:
– Our Qwerty soft keyboard provides users a text input environment they are accustomed to.
– Our Column Typing design supports users in reducing typing errors by pro-viding explicit feedback.
A qualitative user study with six participants showed that users prefer the Qwerty layout they are accustomed to but profit from Column Typing’s feedback in matters of error recognition (see section 5.2.3).
• Dragging, Scrolling and Contextual Menus: These three aspects are further parts essential for controlling WIMP user interfaces. We designed and implemented interaction techniques allowing users to perform dragging and scrolling tasks as well as invoking contextual menus.
Summarized, our interaction techniques enable users to control all essential aspects of WIMP user interfaces. Selection precision for small targets is enhanced while
main-taining high efficiency for selecting large targets. Text can be entered aided by several feedback components compensating the lack of a physical keyboard’s tactile feedback.
Dragging and scrolling tasks can be performed in a natural way. Contextual menus are invokable with only a small and ergonomic finger motion, supported by visual clues.
Our techniques make WIMP user interfaces actuallyusable by touch input, overcoming mouse interaction’s hurdles and letting users benefit from the advantages of tabletops and direct touch input.