The user-friendliness or usability of an application can be considered as a quality feature of a product and is defined as intuitive access to the operation of a product to accomplish a specific task (Nielsen 1994). Usability is thus understood as a pragmatic quality of software in terms of the achievement of objectives. Usability is defined according to ISO 9241-11 (International Organization for Standardization 2008) as the product of
1. effectiveness in the sense of usability for the fulfillment of tasks,
2. efficiency as a measure of the time and effort required to fulfill tasks, and
3. satisfaction as a measure for the positive attitude towards the use of the product in a particular context.
It has to be distinguished from user experience, which is the users’ perception of a system considering the expected utility (Hassenzahl / Tractinsky 2006). According to ISO 9241-210 (International Organization for Standardization 2019), the user experience is defined as “a person’s perceptions and responses that result from the use or anticipated use of a product, system or service.” Thus this term extends the perspective of a persons’ attitude before and after use, whereas usability is a concept considered during the use of a system.
Besides, Nielsen (1994) considers the following criteria to play an essential role in usability:
1. learnability - how easy can a user learn the operation of an application,
2. memorability - how good can a user operate an application after a certain amount of time without use, and
3. error frequency - how many errors does a user provoke, how serious are these errors, and how easily the user can find a solution to resolve the problem.
The mentioned usability attributes can be assigned to the People at the Centre of Mobile Application Development (PACMAD) model illustrated in Figure 6 (Harrison et al. 2013).
Figure 6. The PACMAD usability model
The PACMAD model focuses on the usability of a mobile application and identifies the user, the task, and the context as the primary influencing factors for usability. The context got a particular role, as the
EFFECTIVENESS EFFICIENCY SATISFACTION
ISO 924-11 NIELSEN HARRISON
LEARNABILITY MEMORABILITY ERRORS
COGNITIVE LOAD PACMAD
18 Foundations: Research Background
more critical regarding smartwatches since the devices, concerning their form factor, are used in highly dynamic contexts. Due to this high mobility, including simultaneous or interfering activities and environmental influences, a user’s full cognitive attention cannot be presumed as in traditional usability investigations of desktop applications. For this reason, PACMAD uses the cognitive load, which is necessitated by an application as a core usability attribute (Harrison et al. 2013).
The importance of usability for a system, for instance, can be derived from the Technology Acceptance Model (TAM) according to Davis et al. (1989), which describes the influencing factors of the usage of an information system. As illustrated in Figure 7, the TAM explains the usage behavior with the determinants of the perceived usefulness and the perceived ease of use.
Figure 7. The Technology Acceptance Model
According to the definition of usability, it can be assigned to the lower determinant, which significantly influences the perceived usefulness and thus indirectly and directly affects the attitude toward using a system (Venkatesh / Davis 1996). Besides, Nielsen (1994) considers usability as a determinate for practical acceptance and, consequently, for the systems’ acceptance.
During a usability evaluation, unfavorable factors provoking a negative chain of effects illustrated in Figure 8 should be identified, and the usability of a system should be increased (Watbled et al. 2018).
A usability problem can be defined as a problem that a user encounters when using the system to complete a task within an application scenario (Alshamari / Mayhew 2009). A usage problem is attributed to a usability defect arising from a violation of a usability principle and can negatively affect the user (Marcilly et al. 2015).
Figure 8. Negative chain of effects related to a violation of a usability principle
For the early detection of problems and thus avoiding and limiting the negative consequences, usability evaluation methods are used. The methods can be classified into qualitative methods producing data, which has to be interpreted (testing, observing, and questioning), and quantitative methods, which are based on defined metrics having numerical and objective data as a result (simulation and analytical
PRINCIPLE USABILITY DEFECT USAGE PROBLEM NEGATIVE OUTCOME
?
? ??
modeling) (Ivory / Hearst 2001). For qualitative methods, moderated method types with little automation are common, such as observation and recording, interviews, think-aloud protocols, or heuristic methods.
For quantitative methods in practice, unmoderated method types are frequently used, such as online questionnaires based on the usability scale system (Tullis / Albert 2013), the automated metric recording of an object of investigation, or a task model (Nielsen 1994).
The methods are used in various test environments, which is one influencing factor in the four-factor framework of contextual fidelity that describes the quality of the results of a usability evaluation (Sauer et al. 2019). Accordingly, the test environment has to resemble the real operational environment to avoid a negative impact on quality. The laboratory test is one of the most frequently used test environments (Kolbe / Ruch 2014) since it takes place in a controlled and fully definable context, almost free of accidental environmental influences. This allows collecting data through various instruments during a moderated evaluation, which is highly specified and consequently precisely reproducible. Due to the versatile use cases of a smartwatch, the simulation of the particular environment in a laboratory test is a considerable challenge (Zhang / Adipat 2005). The research on automated usability measurement of smartwatches is still in its infancy. On the one hand, recent methods split into the static analysis, evaluating the source code, and especially the design files during the development (Louridas 2006). On the other hand, the dynamic analysis considers user interactions and is for the scope of this thesis involving domain experts in studies more convenient. For the dynamic usability analysis, there are several approaches from other application domains. Lettner / Holzmann (2012) developed an automated and unsupervised system for usability evaluation by user interaction logging. Usability smells as an indication of a usability defect can be exploited (Almeida et al. 2015). Several studies investigated the analysis of data logged during user interaction like Grigera et al. (2017) who used usability smells to automatically generate a usability report for websites, Harms / Grabowski (2014) who automatically detect usage-based usability smells in web applications, Harms (2006) elaborated on automated field usability evaluation while using generated task trees, and an automated usability evaluation of virtual reality applications (Harms 2019).
Usability in this thesis is an auxiliary construct utilized during the design and development of smartwatch applications. Nevertheless, usability is a crucial factor for the acceptance of smartwatch-based IS and thus has to be considered during software development. Although there are currently no specific methods to assess the usability of smartwatch applications, these foundations can constitute a starting point for an adaption and extension of the knowledge about mobile usability.
20 Foundations: Research Background