Scenario Description

with the first one modeling a user introduction, and the second one praising the robot.

The user’s reply is classified as a praise, due to the word ’nice’, which the robot again replies to very generically by thanking the human. Thus, thanks to the keyword spotting approach, the robot is able to react to a variety of utterances in a meaningful way, and even to utterances that had not really been foreseen.

U My name is Anneliese.

F Hello Anneliese. Nice to meet you.

U Oh, nice to meet you too!

F Thank you.

Figure 7.2: Chaining together simple statements.

Also the user’s attempt to attract the robot’s attention (“Flobi, look here”) was modeled as a simple statement which is always being replied to with “Yes, I’m looking”, without including any task communication. This pattern still does not have a pure dummy function, as the robot orients its gaze at the currently most visually salient point (or at the user, as appropriate for the task state). Thus, if the point the user tries to attract the robot’s attention to already lies in the robot’s field of view, it has a good chance to actually become the most salient point (e.g. as a result of the user’s salient nonverbal gestures or hand movements).

If the user demonstrates an object, the referred object, or rather the referred region, has to be determined. Once the region has been identified, it can be associated with the given label. Thus, an object demonstrating episode involves two consecutive tasks: the resolve reference task and thelearn object task, which can only be executed if theresolve reference task succeeds. To model this process, the Human Object Demonstration pattern was introduced. Since each of the two tasks may fail, the pattern has to consider the different error conditions. Another pattern that includes reference resolution has been introduced to model the user’s test questions to check if the robot is able to recognize a specific object:

the Human Object Test pattern. It is conceivable that, in the future, both patterns will be applied in other learning tasks, such as learning and recognizing people’s faces or rooms.

Failure of learning has to be considered also in the reverse case, when it is the robot who asks for an object on its own initiative. The existing Robot Information Request with Explicit Information pattern was not sufficient for this purpose, as it transfers the information to the responsible system component, but does not acknowledge its processing.

For this reason the Robot Information Request with Explicit Information and Task Acknowledgment was introduced. In contrast, in the original Curious Robot scenario, learning was not acknowledged, but assumed as a process that follows always. This was changed for two reasons. First, breaking down learning to a task with clearly defined states allows for more detailed feedback on the learning progress. Second, the restricted but robust object learning setup from the Curious Robot scenario was exchanged for one

7.2 Scenario Description 125

that offers more possibilities but fails occasionally, which had to be accounted for by the interaction strategy (cf. the description of the system components below).

Dialog segment Example interaction Interaction pattern

Greeting User: Hello, Flobi Human Interaction Opening

Flobi: Hi, hello.

Introducing User: My name is Anneliese. Human Simple Statement

Flobi: Hello, Anneliese. Nice to meet you.

User: What is your name? Human Simple Statement

Flobi: My name is Flobi?

User: Nice to meet you, Flobi! Human Simple Statement

Flobi: Ditto.

Exchanging pleasantries User: How are you? Human Simple Statement

Flobi: Fine, thank you!

Task description User: I am here to show you some objects. Human Simple Statement Flobi: Alright, let’s start!

Flobi: There are many objects on the table I don’t know yet. Robot Notification (only C2 & C3)

Object query Flobi: What is that? Robot Rejectable

User: This is an apple. Information Request

Flobi: Alright, an apple, is that correct? with Explicit Confirmation

User: Yes. and Task Acknowledgement

Flobi: Okay, I am going to learn the apple. (only C2 & C3) Flobi: I have learned the apple.

Yielding Initiative Flobi: You can show me something, too. Robot Notification (only C2 & C3)

Attracting attention User: Flobi, look here! Human Simple Statement

Flobi: Yes, I’m looking.

Object demonstration User: This is a yellow lemon. Human Object

Flobi: Alright, a yellow melon. Is that correct? Demonstration

User: No, it’s a lemon. with Explicit Confirmation

Flobi: Alright, a lemon. Is that correct?

User: Yes, that’s correct.

Flobi: Okay, I am going to learn the lemon.

Flobi: I have learned the lemon.

Listing learned objects User: What have you learned so far? Human Information Request Flobi: I have learned a lemon and an apple.

User: Okay, next one. Human Simple Statement

Flobi: Okay.

Checking User: Flobi, do you know this one? Human Object Test

Flobi: I will have a look at it.

Flobi: This is an apple.

Praising User: Well done, Flobi. Human Simple Statement

Flobi: Thank you.

Transitional phrases User: Would you like to learn another object? Human Simple Statement Flobi: Yes, I want to learn them all.

Closing task User: Okay, that’s it. Human Simple Statement

Flobi: Are you leaving already?

User: Yes.

Flobi: What a pity! Good bye.

Parting User: Good bye, Flobi. Take care! Human Interaction Closing

Flobi: Bye, see you later.

Clarification User: Restart. Human Interaction Reset

Flobi: Okay, let’s start over again.

Flobi: Pardon me? Robot Repeat Request

Flobi: Should we better start over again? Robot Suggest Restart User: Yes, let’s start over again.

Flobi: Okay.

Table 7.4: Interaction capabilities of the system and the employed interaction patterns.

Blocks are: i) Opening interaction, ii) transitional phase, iii) task-related phase and iv) closing interaction. Note that in addition to the listed utterances, many variations in the wording are possible.