Analysis of gaze in the home tour

have a functional and not a communicative purpose (for example, during the guiding of the robot); (c) the body orientation structures other entities than tasks and utterances.

Conclusion

The tasks could be differentiated based on the orientation of the users. The users most often changed their orientation in the guiding task which was also due to the fact that the robot was moving. Only in this task was it at times found that the users turned their backs to the robot. But as has been mentioned above, the time they spent in this orientation was very short compared to the time they spent in more direct orientations. The object-teaching task was the only task during which the participants turned as many times to 45° and 90° (both summed up) as to a vis-à-vis orientation and also spent the same amount of time in these orientations. This finding underlines that the participants turned when teaching objects to establish a participation framework that enables the robot to optimally perceive the objects which are usually located on the side of the participants. This was not found for teaching rooms where the users most of the time faced the robot. Therefore, the actions of teaching rooms and objects seem to follow distinct rules and need to be differentiated.

Altogether, the static and dynamic body orientation have been shown to differ between the tasks and can, therefore, be used to distinguish tasks from each other and to improve the SA of the robot. However, it could not be conclusively determined how the modality might structure the interaction. This question needs to be evaluated in a scenario that is more adequate in distinguishing communicative switches of body orientation from functional switches that are related to the tasks.

Table 5-12. Relation between gaze direction and tasks gaze direction

at robot at object else overall

task count overlap

(%) count overlap

(%) count overlap

(%) count overlap (%) social tasks

greet 50 94.91 1 1.38 7 2.14 58 98.43

intro 14 90.86 0 0 9 9.13 23 100

farewell 20 94.81 0 0 4 4.48 24 99.29

functional tasks

guide 211 82.07 19 0.90 105 8.28 335 91.25

object 153 87.81 114 9.85 18 2.34 285 100

room 60 92.40 12 1.10 26 5.27 98 98.77

problem-related tasks

obstacle 42 64.19 0 0 8 1.78 50 65.96

register 58 91.19 2 0.17 27 6.41 87 97.77

reset 33 93.03 1 0.08 16 5.45 50 98.56

stop 30 86.41 4 4.87 7 5.71 41 96.99

overall 459 85.75 142 2.44 222 5.59 823 93.79

The overall looking time at the robot and the mean duration of glances (14.78 seconds) were much longer than in HHI (Argyle, 1988, see Section 3.2.4). Even though the mean duration was strongly influenced by some very long glances, the median was also very high (8.02 seconds).

The durations of glances at the objects and at places other than object and robots were much shorter (mean 1.37 and 1.98 seconds, respectively; median 1.05 and 1.35 seconds).

The question now is why the users looked at the robot for such a long time. The most probable explanation is that they were waiting for feedback which took longer in this study because the robot was acting autonomously. Actually, the longest glances were found in situations when the users were waiting for the robot to reply. This is in line with Green’s (2009) findings that gaze at the robot signals that the users need some feedback (see Section 3.2.4). Moreover, after the interaction many participants stated that they looked at the screen when the robot did not respond (see Section 5.4). Since a screen is something one is supposed to look at and not have to look away from once in a while in order to not offend the interaction partner, this is the most probable explanation. Unfortunately, from the video data that was recorded, it cannot be determined whether the participants actually looked at the screen or at some other part of the robot. An analysis of gazing behavior in the tasks was conducted to shed some more light on this question (see Table 5-12).

The first observation in the table is that the number of glances and the percentage of looking at the objects were by far highest in the object-teaching task. This is in line with the design of the study and the task. The users most often looked somewhere else other than at the robot or the objects during the intro and the guiding task. During the guiding task the participants certainly looked where they wanted to go to. During the intro the frequent glances away from the robot are more surprising because the robot talks about its usage and often refers to pictures on its

screen. When having a second look at the data, it was found that in fact only one user looked away from the robot six times. She turned to the side because the robot had called her by a wrong name (‘Igor’) and she kept laughing about this mistake. Hence, the common gazing direction during the intro really seems to be the robot. But this case also shows that the users might crucially change their behavior if some irregularities occur in the interaction. With respect to the percentage of the time that the users in general looked at the robot, no significant differences were found.

Finally, the gaze direction corresponded with certain body orientations in the tasks. In the social tasks and the problem-related tasks the body orientation was very direct and the users looked at the robot most of the time. In the guiding task the body orientation as well as the gazing direction changed most often. The users spent most of the time looking somewhere else other than the robot and the object and in indirect orientations. The user behavior differed considerably between the two teaching tasks. During object-teaching the users often slightly turned away from the robot and then back towards it as they gazed at the object and then back at the robot. In contrast, during the room-teaching task they gazed at the robot and kept a direct orientation for most of the time. Accordingly, the robot could identify that something was taught based on the utterance and, furthermore, receive information about what was shown via the gaze direction and the body orientation.

5.1.4.4 Comparison of gaze in the object-teaching studies and in the home tour

As stated before, it is assumed here that the repertoires of user behaviors are strongly bound to the situation. In the following, the gaze behavior in the object-teaching study is compared to the behavior during the object-teaching task in the home tour study in order to determine whether it differed in the two situations.

At first sight it can be seen that the users spent considerably more time looking at the robot in the apartment than in the laboratory (88% vs. 68%). Consequently, the percentage of gazes at the objects (9.85% vs. 16%) and somewhere else (2.34% vs. 16%) was much smaller. These differences can be explained on methodological as well as on situational grounds. To begin with the reasons that lie within the situation, the robot in the laboratory was not mobile and therefore it can be assumed that the users in that situation did not monitor it as closely as in the apartment.

Another crucial difference is that the screen was turned off in the laboratory while it displayed the Mindi in the apartment. The participants confirmed in the interviews that they often looked at the display for information (see Section 5.4). Moreover, the type of objects might make a difference. In the laboratory, the objects were manipulable which caused the users to hold them in their hands and do something with them. These activities certainly require more attention towards the objects than showing larger objects in a room that in only three cases (2.65% of all cases) were moved. Moreover, picking out the objects in the apartment was much easier due to their size and because the participants were told what specific objects they had to show. This explains why the duration of glances somewhere else other than at the robot and the object was much shorter in the home tour.

However, there are also some restrictions on the comparison from a methodological point of view. First of all, the number of glances that were compared differed considerably (laboratory

7828; apartment 285). Most importantly, the quality of the video recordings and, thus, the granularity of the annotations vary between the studies. While it was easy to get very good recordings in the laboratory because the participants did not move, it was much harder in the dynamic task in the apartment. Therefore, short gazes away from the robot as a result of cognitive load that might easily be recognized in the laboratory might have been overseen in the recordings from the apartment. This assumption is underlined by the finding that the mean duration of glances at the robot was much longer in the apartment (16.06 vs. 3.89 seconds). In contrast, the mean duration of glances at the objects (1.36 vs. 1.25 seconds) and somewhere else (2.10 vs. 2.18 seconds) were very similar in both studies. Therefore, the difference between the two studies mainly results from the long glances at the robot.

This section shows that the comparability between the two studies is limited. Nevertheless, it was found that in both cases most of the time was spent looking at the robot. Moreover, it can be concluded that the gaze direction was in fact influenced by the tasks.