This thesis will contribute to the field of Longitudinal Research in HCI in a varie-ty of ways, which we will briefly outline here. Based on our own experiences and the literature, we have identified three main challenges we would like to address. First, researchers in HCI lack a basic understanding of longitudinal research. As has been apparent from several workshops, SIGs, and panel dis-cussions, there is neither a clear unifying view nor any basic literature to which people can refer. This makes it difficult to discuss issues in longitudinal re-search as well as to identify potential rere-search areas that should be addressed.
Second, especially for longitudinal field studies, we need more tools and tech-niques that could support researchers conducting such studies, thereby reduc-ing the costs and any apprehension about gettreduc-ing involved in longitudinal re-search. Third, we need specific, tailor-made methods for longitudinal data-gathering and analysis, especially in the context of qualitative data.
1.3.1 A Taxonomy for Longitudinal Research in HCI
In Chapter 2 we will address the first issue, regarding the common understand-ing of longitudinal research in HCI. To this end, we will provide a theoretical workup of the topic that will eventually lead towards a taxonomy for longitudinal research in HCI. The goals of this taxonomy are 1) to give order to the existing literature in the field, taking into account findings from other disciplines, such as the social sciences and psychology; 2) to provide guidance for researchers and practitioners new to the field, helping them with an overview of the design space of longitudinal research; and 3) to promote scientific discussion by providing a common ground to which everyone can refer. The taxonomy is intentionally not restricted to a certain type of longitudinal research in HCI. Rather, by “going broad,” we would like to encourage other researchers to challenge the taxono-my, test it, and extend it, if necessary.
1.3.2 PocketBee - A Multi-modal Diary for Longitudinal Field Research
In Chapter 3, we will address the issue of tool support for longitudinal research.
Based on the taxonomy presented in Chapter 2 we identify two areas that offer potential: interaction logging and diary/ESM approaches. As it is the more flexi-ble tool, we opted for diary/ESM approaches. The chapter presents an exhaus-tive discussion of diary and ESM approaches, their advantages and drawbacks, before eventually leading to a discussion of PocketBee, a multi-modal diary tool based on Android smart phones. We contribute towards this field by presenting a classification of research designs that unifies diary and ESM studies and by providing a direct link to an event architecture that allows free combination of these designs within the PocketBee tool. Finally, we present the user interface design of the tool for participants and researchers, seeking to provide high usa-bility and flexiusa-bility in methodology. In addition, PocketBee especially focuses on a closer connection between researcher and participant.
1.3.3 Concept Maps – A Method to Evaluate API Usability
In Chapter 4, we address the third issue by presenting a customized longitudi-nal data-gathering and alongitudi-nalysis method for evaluating application programming interfaces (APIs). We present a constructive approach that implicitly asks partic-ipants to illustrate changes over time, allowing the researcher to easily identify them – an issue that can be very difficult with qualitative data. We focused on APIs because the issues of learnability and usability over time are of particular importance here. An API is not learned once and then applied; rather, pro-grammers learn an API on the fly and to the extent needed for the task at hand.
In addition, API usability is an often-overlooked aspect of overall product quality, which we found to be well worth additional consideration within the scope of this thesis.
2 A Taxonomy for Longitudinal Research in HCI
Any kind of empirical research needs to be designed. Even though the phrase
“research method” conveys the idea of a clear step-by-step guide to solving a research question, this is hardly the case; such assumptions instead lead to uninspired and inappropriate research. Applying any kind of research paradigm requires the researcher to be aware of and acquainted with the design space the paradigm provides. Design space is a term often used in traditional design disciplines, such as graphical design or interaction design in HCI. The term re-fers to a space of possibilities for design within certain boundaries and featuring key attributes. Defining a design space basically means defining these bounda-ries and attributes. While HCI literature offers assistance in defining the design space for cross-sectional methods (including usability tests, experiments, inter-views, and surveys), the research paradigm of longitudinal research clearly lacks such guidance. For instance, the major textbooks on research methods in HCI donate very little space to this topic (Rogers, Sharp, & Preece, 2007), (Cairns & Cox, 2008), (Lazar, Feng, & Hochheiser, 2009).
There are a number of different ways to describe the design space. One way that has attracted interest in HCI and software engineering is through patterns, which have also been used in interaction design. Design patterns provide ex-amples that illustrate the basic principles of an applied design, how it was cre-ated, and whether it was successful. The patterns often try to incorporate these aspects into a single holistic and interlinked graphical representation (e.g.
(Borchers, 2001)). However, obtaining an overview is often difficult (although not always necessary). Another possibility, which we address in this thesis, is the definition of a taxonomy. A taxonomy “refers to classification according to presumed natural relationships among types and their subtype”3. The major advantage we see in the taxonomy approach is its inherent structure and clarity that allows readers to quickly comprehend the entire design space without
3 ISO/IEC 11179, 1.
ing to understand all the specifics. Details are available but are confined to low-er levels in the hilow-erarchy of the taxonomy to the point which specifically asks for this kind of information.
While a practitioner should benefit directly from access to such a taxonomy, we think its value is much more extensive. As researchers seeking to take the methodologies of longitudinal research in HCI one step further in their develop-ment, we think it is essential to share a common overview of the current state.
This allows us to identify the areas that need further research, no matter wheth-er they concwheth-ern new methods, new tools, or diffwheth-erent theoretical undwheth-erstand- understand-ings. For this thesis, the taxonomy has already served this purpose, as two ma-jor problematic areas of longitudinal research in HCI were successfully identified and subsequently addressed with the Concept Maps approach for API evalua-tion and the PocketBee diary/ESM tool (see chapter 3 & 4).
Longitudinal research in HCI is a very broad topic and it must seem to be a diffi-cult task to define and carve out a general taxonomy. However, as longitudinal studies in HCI are still rare, we feel that limiting to a specific type of research area would be too restricting and leave too many areas uncovered. Therefore, our goal here is to provide the first step for a holistic taxonomy, being aware that we are likely to miss certain research areas. Our hope is that researchers of these areas will take the chance to build upon our taxonomy and extend or modify it, accordingly. To give the reader some perspective on our background, most of our own experiences with longitudinal research come from the domain of pointing device evaluation (Gerken, Bieg, Dierdorf, & Reiterer, 2009a), infor-mation visualization (Gerken, Demarmels, Dierdorf, & Reiterer, 2008b), and API usability (Gerken J. , Jetter, Zöllner, Mader, & Reiterer, 2011).
In the remainder of this chapter, we will present the taxonomy step by step. We will start by outlining the approach that eventually led to this taxonomy – a mix-ture of experiences gathered through the design of longitudinal studies, an ex-tensive literature review of longitudinal research in other fields (including social sciences and psychology), and a review of HCI literature and in particular em-pirical studies that claim to be longitudinal.