Mental Load

Workload does not necessarily rise with task difficulty. Tulga and Sheridan [TS80] re-ported that an increasing demanding task does not imply a higher level of workload. They conclude that user lowered their mental effort, as the task was getting too difficult to solve appropriately.

Despite the lack of an unique definition, mental workload is been used in many studies and applications. Mental load can be measured in different ways: subjective with rating scales, by performance measures or with help of physiological signals. In the following the different measurement methods will be described further.

2.4.1. Physiological Measures

Depending on the scenario and the aspects of workload, several physiological signals come into consideration for measurement of mental workload. Kramer [Kra91] exam-ined different physiological signals as measurement for mental load under the aspects of sensitivity, diagnosticity, intrusiveness and reliability. Kramer stated, that different phys-iological signals measure different aspects of mental workload. Part of the study were event-related potentials measured from the brain, cardiovascular activity, pupil diameter and measures of respiration. EDA turned out to be only useful to identify shifts between situations of different kinds of workload. Event-related potentials achieved highest di-agnosticity to determine mental workload. On the other hand, measurement of ERP is intrusive in comparison to other measures.

Changes in pupil diameter proved to be a reliable measure for mental workload. Kramer [Kra91] recommends to use pupil diameter measurement only in controlled experimental conditions, where the head movement can be controlled, due to the difficulty of fast and exact determination of diameter changes. Since the study of Kramer, measurement devices for pupil diameter improved dramatically, allowing a fast measurement with eye-tracking systems even in situations, where the user is moving the head, e.g. during driving a car [PKSH10]. However, pupil diameter measurement requires a user to have some sort of camera in direction of the eyes.

Cardiovascular measures in the study of Kramer [Kra91] were blood pressure, blood volume and ECG. As described in chapter 2.3.5 HRV can be used for assessment of mental workload. Meshkati [Mes88] examined besides HRV several other physiological signals regarding their usage for mental load measurement and calls it the most promising mea-surement for workload.

2.4.2. Subjective Ratings

The most famous rating scale for mental workload assessment is the NASA Task Load Index (NASA-TLX) originally designed for aviation which was introduced by Hart in

2.4. Mental Load

[HS88]. The NASA-TLX is a multidimensional scale to obtain workload of a person. The NASA-TLX is divided into six different subscales, which are shown and further described in table 2.1. Each subscale is divided into 20 items to allow a rating between low and high or good and poor. Before answering the six scales (mental demand, physical demand, tem-poral demand, effort, performance and frustration level), user have to fill out a weighting for the six aspects to rate which of the scales influences them more in comparison to the other scales. Based on the results of these weighing, the results of the six scales are calcu-lated. In the end, all six scales are added together for the mental workload.

Title Endpoints Descriptions

Mental Demand Low/High How much mental and perceptual activity was required (e.g. thinking, deciding, calculating, remembering, looking, searching, etc.)? Was the task easy or demanding, simple or complex, exacting or forgiving?

Physical Demand Low/High How much physical activity was required (e.g., pushing, pulling, turning, controlling, activat-ing, etc.)? Was the task easy or demandactivat-ing, slow or brisk, slack or strenuous, restful or la-borious?

Temporal Demand Low/High How much time pressure did you feel due to the rate or pace at which the tasks or task elements occurred? Was the pace slow and leisurely or rapid and frantic?

Effort Low/High How hard did you have to work (mentally and physically) to accomplish your level of perfor-mance?

Performance Good/Poor How successful do you think you were in ac-complishing the goals of the task set by the ex-perimenter (or yourself)? How satisfied were you with your performance in accomplishing these goals?

Frustration Level Low/High How insecure, discouraged, irritated, stressed and annoyed versus secure, gratified, content, relaxed and complacent did you feel during the task?

Table 2.1.: Rating scale definitions of the NASA-TLX [Har06]

These six subscales address the challenge of a clear workload definition. The six aspects of workload were designed to meet most people’s experience of mental workload [Har06].

In [Har06] the usage of NASA-TLX has been examined over the last 20 years. In total 550 studies from different countries were reviewed by Hart [Har06] regarding different aspects like study environment, focus of the study etc. The results showed, that visual and or auditory displays were focus of most studies with a portion of 31 %. Looking on the environments of the studies, Air Traffic Control (10), civilian (12) and military cockpits (5) had the biggest portions. Computer users had a portion of 7 and user of portable technologies like smartphones had a portion of 4.

Besides the NASA-TLX several other rating scales for mental workload exist, like for example the Subjective Workload Assessment Technique questionnaire (SWAT) [RN88].

The SWAT questionnaire consists of an additive multidimensional representation of three dimensions, like shown in figure 2.10. The dimensions are: time load, psychological stress and effort load.

Figure 2.10.: Three-dimensional workload model of Reid and Nygren [RN88]

Each of these dimensions is divided into three level: low, medium and high. The single definitions of each level for each scale is shown in table 2.2. To assess workload with SWAT, two steps have to be conducted. The first step, called the scale development, is based on 27 cards, which contain all possible combinations of the three scales and its three levels. The participant is asked to sort the cards in order of increasing workload by own perception. In the next step, the participant rates its workload on the scales. Finally, each rating is converted to a value between 0 and 100, based on the scale of step 1.

Luximon et al. [LG01] developed a simplified version of the SWAT questionnaire to handle pitfalls like missing sensitivity of low workloads. They compared different types of simplification of the card sorting step. Card Sorting was compared with SWAT without card sorting and a method of pair wise comparison.

The results showed, that the approach of SWAT without card sorting was most sensitive, pairwise comparison moderate to more sensitive and the full card sorting process lead to

2.4. Mental Load

I. Time Load

1. Often have spare time. Interruptions or overlap among activities occur infrequently or not at all.

2. Occasionally have spare time. Interruptions or overlap among activi-ties occur infrequently.

3. Almost never have spare time. Interruptions or overlap among activi-ties are very frequent, or occur all the time.

II. Mental Effort Load

1. Very little conscious mental effort or concentration required. Actively is almost automatic, requiring little or no attention.

2. Moderate conscious mental effort or concentration required. Complex-ity of activComplex-ity is high due to uncertainty, unpredictabilComplex-ity, or unfamil-iarity. Considerable attention required.

3. Extensive mental effort and concentration are necessary. Very complex activity requiring total attention.

III. Psychological Stress Load

1. Little confusion, risk, frustration, or anxiety exists and can be easily accommodated.

2. Moderate stress due to confusion, frustration, or anxiety noticeably adds to workload. Significant compensation is required to maintain adequate performance.

3. High to very intense stress due to confusion, frustration, or anxiety.

High extreme determination and self-control required.

Table 2.2.: Rating scale definitions of SWAT [RN88]

least sensitivity.

Both introduced questionnaires, NASA-TLX and SWAT, have been widely used, espe-cially in determination of workload in aircraft multitask situations [RDMP04]. However, subjective ratings require that a user fills out a questionnaire or answers question. During interaction with a system, this might lead to an interruption in workflow.

2.4.3. Performance Measures

Besides subjective ratings and physiological signals, mental load can also be estimated by performance measures like error rate, click rate or time span to solve a task. Task perfor-mance can be measured by a primary-secondary-task scenario [Lin91]. User focus on a primary task, while a secondary task is offered for situations of low workload in the pri-mary task. Depending on how well users perform at the secondary task, workload can be

estimated.

This approach is more objective as the subjective ratings, but offers only limited accuracy in the determination of different workload levels. Furthermore performance measure are not reliable in every situation, a person can be distracted from the environment or occupied with multitasking switching between different other tasks. If a person does not engage enough with the secondary task, workload estimation is very inaccurate or not possible.