In summary, the presented background on stress responses provides an overview on the construct of stress and highlights the disunity regarding an universal definition; besides, it has been clarified on which understanding of stressors this thesis is built upon. Moreover, the most relevant concepts in the field stress research including the explanation of the difference between eustress and distress have been addressed. Further, the key element of a stressor is described, before the four main measures assessing stress are briefly characterized.
Facing these different measures it has become obvious that stress can be sensed using various measures and means. Thus, related literature suggests to understand stress as construct being shaped by relations or so-called "transactions" embracing an individual and its environmental factors [151]. In reference to theTransactional Model of Stress and Copingby Lazarus and Folkmann [152] (see Section 2.1 for a detailed explanation) this thesis places its interventions at the layer of primary appraisal allowing to interpret stressors. While perceiving stressors as sources of stress, the presented approach of manipulating stressors leads to reflecting about those and consequently facilitates to cope or even reappraise with such sources of stress. Thus, the present work shows how intervention techniques can be used to circumvent the secondary appraisal layer which leads to stress, but rather supports the individual to either manipulate the stressful stimulus, re-consider one’s view of the stressor, or to interpret stressors differently jumping directly to the final phase.
Chapter 3
Implications of
Physiology-Aware Systems for Design
The connection between the mind and the body has been explored for a long time. In the beginning the vital functionalities have been investigated mainly in animal experiments envisioning to reveal hidden signs of affective responses.
Research in medicine, physiology, psychology, and related disciplines have come a long way in the past years since more light was shed on the relevance of physiology-based data in connection to stress responses. Particularly the work by Hans Selye [242] exploring the nature of stress and Walter Cannon’s [34] concept of the "homeostasis" have marked a central point in stress research and thus, have contributed immensely to the investigation of stress throughout the past century.
While performing his research, Selye, among others, recognized through his experiments the significant role of stress hormones activating the Hypothalamic-Pituitary-Adrenal Axis (HPA Axis) system [244]. This complex apparatus controls stress responses and triggers specific body reactions, such as digestion, the immune system, and emotional responses. Accordingly, the importance of the ANS has been revealed. Among its three subordinated systems, the enteric, the parasympathetic, and the sympathetic, the two latter ones act as
antagonists (please refer Section 2.3 for a more detailed description). Knowing how these systems work together and reflect the human stress response towards threats, or more generic stressors, has paved the way for the domain of affective computing. When Rosalind Picard in her book "Affective Computing" [197] first mentioned her vision how human-computer interaction could be revolutionized if computers would sense and react according to its user’s affective states, the challenge of integrating emotion-aware interactive systems was born. As a fundamental prerequisite for the implementation of such user-understanding technology, the recognition of affective states must be reliably [11, 292].
Consequently, this chapter focuses on the examination of physiological data in conjunction with its validation through subjectively assessed data and what implications referring to RQ1b and constraints this has for the design of stress-aware interactive systems answering RQ1c. As a foundation for the understanding of relevance of physiological in human-computer interaction research, the presented literature will illustrate first how stress responses can be measured physiologically and thereby respond toRQ1a.
This chapter is based on the following publication:
• K. Hänsel, R. Poguntke, H. Haddadi, A. Alomainy, and A. Schmidt.
What to put on the user: Sensing technologies for studies and physiology aware systems. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI ’18, pages 145:1–145:14, New York, NY, USA, 2018. ACM
3.1 Related Work
The related work embraces the topic of stress recognition with the help of physiological parameters. Subsequently, the data collection and processing of physiological signals is briefly summarized highlighting variations in signal quality which lead to differences among sensing hardware.
Physiological Parameters Signifying Stress As already described in subsection 2.3 there are physiological parameters that reflect if the human body is aroused or not. Among other studies [81, 148, 181], Quesada et al. [215] found that their participants responded to stress inducements with a an increase in their arousal valence using the SAM visual rating scale [26]. Winsky-Sommerer,
Boutrel, and de Lecea [285] proposed that the stress-triggered biochemical reaction in cortical structures affects the arousal and further the amygdala, being responsible for anxiety. Consequently, arousal is understood as a physiological reaction towards stressors. Among the mentioned signal types heart rate in conjunction with heart rate variability is regarded to deliver the most reliable indication of stress, since prior work indicated that it correlates significantly with the emotional state [57, 224]. It has been used as a stress indicator in a variety of disciplines such as medicine [95], psychology [71, 238], and HCI [175].
However, it is prone to physical activity and cumbersome to record from an user perspective [120]. More recent approaches use optical sensors, PPG for inferring heart beats from the blood flow beneath the skin [90]. This technology bears the advantage that sensors can be placed at various positions at the body, such as earlobes, fingertips or, as commonly used in fitness trackers and smartwatches, at the wrist. This has been used for example in monitoring th heart rate under physical movement [294]. Apart from it’s sufficient reliability, another advantage of the ECG signal is its richness of features, since it portraits the behaviour of the heart in detail. The signal cannot only be used to extract features such as heart rate, but also the variation of inter-beat intervalls - namely HRV. The use of HRV features for detecting stress is grounded in the effect of the sympathetic and parasympathetic nervous system on the beat beat patterns [248]. An increased variation in time between consecutive heart beats (also called inter-beat or R-R intervals) is hereby related to "cheerfullness and calmness" [80]. On the contrary, a low heart rate variability also has been associated with an incresed mortality risk, diseases [260], and decreased emotional-regulation [273] and stress [57].
Accordingly, various measures consider the data in time- or frequency domain.
The most commonly used are the Root Mean Square of Successive Differences (rMSSD), the Standard Deviation of Normal Sinus Beats (SDNN), and the High Frequency/Low Frequency Ratio (HF/LF Ratio) [259]. A further stress related feature is EDA, which is mostly referred to as the activation of sweat glands;
additional names include Galvanic Skin Response (GSR) or skin conductivity [54]. To prepare for the potential increased physical extortion due to a ’fight or flight’ reaction, a stress response is accompanied with an increase in sweat production. This increased moisture of the skin can be easily picked up by sending a small current through the skin and measuring the resistance. EDA can be found in prior work as an indicator for cognitive load [246], stress [116] and also as a
"predictor of emotional responses to stressful life events" [182]. With the increase in sweat production, there also come changes in the surface temperature of the skin due to the evaporative cooling; skin temperature showed a good prediction ability indicating stress through a drop in surface skin temperature [129, 265].
Reflecting on the accuracy of physiological signals serving as stress markers, it is still arguable in how single measures provide enough data richness to claim sufficiently reliable results [5, 120, 128, 224]. It has been broadly discussed that physiological parameters, and particularly heart activity are prone to confounding variables, such as caffeine intake or overall physical fitness. Therefore, some researchers suggested the comprehensive assessment of stress, such as Kye et al. [147] who developed a ’Multimodal Data Collection Framework for Mental Stress Monitoring’ using four different sensors (Empatica E4, LG watch style, Zephyr Bioharness, IP camera) due to the challenge of getting sufficiently reliable data. To sum up, for the successful implementation of physiology-based stress-aware systems, the data validation and related implications for its usage must be considered in the design.
Data Collection and Processing of Physiological Signals When speaking of physiological data collection, various aspects have to be considered.
For example, the richness of the sensors used is decisive when planning a research project. For researchers and consumers alike, the way how their data can be assessed also plays an important role. While many fitness trackers provide an additional smartphone application to monitor the activity i.e. in daily or weekly visualizations, experimenters are more concerned with the given preprocessing in many sensing devices, such as Fitbit2. Particularly the choice of sampling rates for assessing distinct parameters can differ according to its application scenario.
While for measuring body movements in humans the activity is contained within sampling rates below 20 Hertz (Hz) [8, 17, 124, 127, 203], previous research took frequencies between 7 Hz and 50 Hz, but also 200 Hz for recognizing activities of daily living using accelerometers [132]. Since the question of which frequency to take is often raised in empirical studies, researchers should be aware of the trade-off between high sampling rates resulting in more data points allowing better interpretation of the data, but requiring more computational effort affecting, e.g. the size or costs for hardware on the other hand. While exploring the question which frequencies influence classifiers, Khusainov et al. [132] found that sampling frequencies between 10 Hz and 20 Hz improve activities of daily living classification, which was confirmed by Bouten et al. [24].
Another relevant issue when elaborating on physiological sensing, is the data processing part. It can be understood as a process comprising different sub-processes which can differ among distinct sensing devices. Often the following exemplary methodology is being followed when dealing with physiological data. In the pre-processing phase data is filtered and cleaned for
2 fitbit.com
example with the help of Wiener filters. Also the Principle Component Analysis have been ran to remove irrelevant features from data recordings, particularly to model stress based on EEG data [55]. Another noise reduction technique that is commonly used for EEG signals alike, is the Independent Component Analysis [118]. The data segmentation aims to extract and classify features by using different techniques, e.g. the Fast Fourier Transformations. In practice, Sharma and Gedeon [249] applied this practice or the Wavelet transformations to physiological signals. Then, features need to be selected; for this the domain of the data set and the targeted discriminatory features should be known. The latter becomes even more important for the feature selection depending on the
"discriminatory qualities of the features" [132]. Finally, events can be classified.
This has been done in a vast amount of studies that have been performed to classify or model stress, features extraction techniques. For example, Han et al. [96] found that the combination of Random Forest and Support Vector Machine algorithms provides the best performance regarding the accuracy of differentiating between three different states (no stress, moderate stress and high stress). Facing the variety of analysis methods, still the question remains on how the data being used for this computations is preprocessed by the sensing technology. Likewise, the data resolution obviously has an impact on the analysis technique and therefore needs to be considered as a decision criteria when choosing a recording device, what will be discussed more detailed in Chapter 4.
A core requirement when aiming to design stress mitigating applications, is to know when users are stressed. Since from a biochemical perspective stress is triggered by arousal state [285], self-assessed arousal ratings were used as a ground truth measurement in the following study. The work presented in this chapter serves as a foundation for enabling the implementation of stress-aware interventions in interactive systems and further holds a proof-of-concept for the assumptions made that have been mentioned in prior work as for example referenced above.