6 Conclusion
5.3.1 Guiding the robot (Systemic Interaction Analysis)
Guiding the robot is the task that enables the users to teach it different rooms and objects.
Before being taught, the robot needs to be close to the object of interest. Therefore, the users initiate guidance tasks to place the robot and themselves in an appropriate location. Of course, the guiding task is not only a preparation for teaching but also a task of its own with difficulties, challenges, and its distinct prototypical script (see Lohse, Hanheide, Rohlfing, & Sagerer, 2009). The guiding task consists of three parts: giving the command to follow, guiding the robot, and saying stop to end the guiding sequence. As for all tasks, the subjects need to know what utterances the robot understands. Moreover, they have to estimate how far away they can be from the robot so that it still perceives them while following, but does not stop because the user is within the security distance (usually one meter for BIRON). The participants habitually expect that the spacing behavior of the robot is similar to human behavior and, thus, stand at a distance from the robot which is in accordance with an appropriate social space of European people (see, for example, Hüttenrauch et al. 2006; Walters et al., 2007). In contrast, some other concepts require training of the user. For example, saying stop at the end of a task is not something naturally done in HHI. Also the commands that the robot understands are not open ended. Therefore, the users have to learn during the interaction which utterances work well (here: “BIRON, folge mir” [“BIRON, follow me”] and “BIRON, komm mit” [“BIRON, come with me”]). On the system side, several prerequisites have to be fulfilled to conduct a follow behavior. The system needs to be in a consistent state in which all components accept the follow command and has to perceive the person stably. If this is the case, the prototypical script can proceed as follows:
• user says follow command (il) (“Biron folge mir” [“Biron follow me”], “Komm mit”
[“Come with me”])
• BIRON understands the command correctly (sl)
• PTA changes to state follow, motor commands are enabled, and the respective components are started (sl),
• Mindi changes to follow (il)
• BIRON says “Gut, ich folge dir” [“Ok I follow you.”] [first trial]; “Ok. Ich folge dir. Bitte sage stopp wenn wir am Ziel sind” [“Ok I follow you. Please say stop when we have reached our destination.”] [second trial]) (il)
• user guides the robot (il)
• user says stop command (“Biron stop” [“Biron stop”], “Anhalten” [“stop”]) (il)
• BIRON understands commands correctly (sl)
• PTA changes to person (sl), the respective motor commands are sent and the respective components are stopped
• Mindi changes to standing/userlook (il)
• BIRON replies “Gut. Ich stoppe” (“Ok. I stop”)
The off-talk analysis showed that the users were very insecure about this script. They asked how far away from the robot they had to be for it to follow and when they should start moving. One user wanted to know whether it could follow in a certain direction and another one asked whether she was in the way of the robot when it started beeping. Moreover, they wondered if BIRON would stop right away when they did not move any more, when it encountered obstacles, or when asked to do so. All utterances showed insecurity about the robot’s abilities and the prototypical interaction script. The users were concerned especially with the robot’s ability to quickly stop in order not to run someone or something over. Of course, the robot should be able to stop quickly enough due to safety reasons.
Aside from security issues, it is not assumed that a prototypical situation is “perfect” or comparable to an ideal situation in HHI. Rather, it can be observed in the data that subjects explored many different strategies to successfully guide the robot. For example, they tried different positions in front of the robot to attract its attention and to keep an adequate distance that triggers the robot to follow. They stepped closer, further away, and to the side. They did not start guiding the robot by further increasing the distance until it had announced that it would follow or when it started driving, depending on which of these two actions happened first.
Moreover, the data shows that the attention that the subjects paid to the robot varied, which is obvious in the orientation of the human body towards the robot. While some subjects always walked backwards to face the robot, others adapted their behavior according to the situation (see Section 5.1.3). When they were in open spaces such as the living room, these subjects turned their back to the robot and looked back at it once in a while. In contrast, when the path became narrow, they turned around and faced the robot the whole time. In the studies, it was also found to be prototypical that participants did not just walk around the corner assuming that the robot can interpret this like a human would do. Rather, they walked as far into the next room as possible without turning and waited until the robot came close before they turned.
In the next step, the cases in which the interaction deviates from this prototypical script will be analyzed for the 264 guiding sequences. 45 (17%) of these were categorized as prototypical interaction episodes. Consequently, 219 were deviating cases. This rate in fact is much higher than in the social tasks. With the help of the SInA method, twelve deviation patterns were identified which explain 98% (215) of the non-prototypical sequences. The deviation patterns can be categorized in four groups: speech understanding, person perception, state-related patterns, and navigation. In Table 5-15 they are analyzed according to what the users do, what happens within the system, what the robot does, the influence on the interaction, and the number of occurrences.
Four deviation patterns with respect to speech understanding were identified: speech recognition, repetition of commands, person speaks while the robot speaks, and speech fragments. Speech recognition errors are similar to the errors that occurred during the greeting
Table 5-15. Deviation patterns in the guiding task
Pattern User (il) Robot (sl) Robot (il) Influence on HRI #
Speech Understanding Errors in
speech recognition
utters follow and stop command
(a) the input cannot be interpreted at all
(a) asks the user to repeat (a) users repeat the follow command
17
(b) the input is interpreted in a wrong way
(b) starts a wrong action (b) users try to resume guiding
14
User speaks while robot speaks
utters follow command while the robot speaks
partial speech
recognition because the robot cannot listen while it talks
no reaction users wait, after some time repeat the command
11
Repetition of commands
(a) utters follow command
processing/component is too slow
(a) no reaction (a) users repeat follow command, robot answers twice, switches to follow state
7
(b) utters stop command
(b) keeps driving until reaching the security distance, no verbal reaction
(b) users repeat command, robot answers twice, stops
9
Speech fragments
guides the robot without speaking
speech recognition catches fragments (a) the robot cannot interpret
(a) asks the user to repeat (a) users repeat follow command or ignore robot utterance
10
(b) the robot interprets in a wrong way
(b) starts a wrong action (b) users try to resume guiding
4 Person Perception
User lost (a) before following
utters a follow command, stands far away enough
no user perceived according to the person model
(a) does not react (a) users wait, repeat command
22
(b) while following
(b) says that it cannot see the person anymore, requests a hello, Mindi displays weak person perception
(b) users say hello again, interaction continues
28
States Unfinished or
wrong state
utters follow command before completing some other action
does not accept utterance; utterance is interpreted within the current system state
asks whether the user wanted to do sth. else, requests a stop command
users say stop to finish the previous task
27
Follow action incomplete
guides the robot, does not say stop, starts another action
stays in follow mode asks whether the user wanted to do something else and requests a stop
command
users say stop to finish the guiding task
13
Asynchronous dialog
says follow me, stands in an appropriate distance
has lost person, thinks that no interaction partner is present; the dialog reacts because it has a different expectation
verbally announces that it will follow, but does not start driving, Mindi displays follow state
users try to attract the robot’s attention, walk around in front of the robot, try out another command after a while
8
Navigation Obstacle guides the robot an obstacle is perceived
with the laser scanner
stops, announces obstacle and asks the user for help
users pull the robot away from the obstacle
34 User standing
too close to the robot
guides the robot, stands within security distance
robot is in follow state, notices that person is standing within security distance
says that it follows, shows follow Mindi on screen, does not drive
interaction gets stuck if the users do not step back eventually
11
and the farewell tasks. Due to the errors, the speech could not be interpreted at all and the robot asked for clarification; or the speech was interpreted in a wrong way and the robot started an unexpected behavior (for example, the robot asked the users if they wanted to know where they were, instead of following). Clarification is rather natural in HHI, whereas starting wrong ac-tions is not and takes more time to be repaired. Also repetition of commands and person speaks while robot speaks is similar to the patterns describe in Sections 5.2.1 and 5.2.3. A new pattern that occurred during guiding concerned speech fragments. Even though headsets were used for speech recognition, fragments occurred when someone talked in the surrounding of the robot or when the system interpreted some noise as speech. Speech fragments led to a request for repetitions or to wrong actions (for example, the system interpreted a “stop” and stopped following).
A functionality that is particularly relevant for guiding the robot is the robust tracking of the person. With SInA, two major deviation patterns were identified with this respect: person lost before following and person lost while following. In the first case, the robot did not react to the user because it hypothesized that nobody was there. In the second case, the robot stopped and announced that it could not see the user anymore. This was usually caused by subjects who walked too fast or too far to the side. This behavior again reflects the insecurity of the subjects about how to behave in this situation.
Like in the greeting and the farewell task, robot states also posed a problem in the guiding task.
It considers unfinished or wrong states, incomplete follow actions, and asynchronous dialog. In contrast to unfinished or wrong state, which focuses on actions before the guiding sequence, the category incomplete follow action describes the incomplete follow action itself. Follow actions were not completed if the user did not say stop, but immediately tried the next action.
Asynchronous dialog occurred when the dialog component had expectations that differed from the rest of the system. While the person perception did not perceive a person, this was not communicated to the dialog component that, as a result, reacted like a user was recognized and requested a verbal input. The input, however, was not considered because the system only interpreted verbal input if a user had been recognized. This particular improvement had already been made to the other tasks (see Peltason et al., 2009).
The last group includes two deviation patterns connected to navigation. These are obstacle blockage, and person standing too close. Obstacles, in general, are objects that the robot detects with the laser range finder within a certain security distance (approximately 30 cm measured from the center of the robot). They led to an emergency stop and to the announcement by the robot that an obstacle was in the way. Moreover, the user was asked for help (“Please pull me away from it”). If the users themselves were standing too close to the robot, it stopped when it was about to enter their personal space (about one meter for BIRON). In contrast to the obstacle blockage, in this case it did not announce anything but waited for the person to continue walking. If the person did not do so, the interaction got stuck.