Manipulation Action Prediction By Virtual Reality: A Comparison Between Human and
5.0.8 ESEC Matrices
As a reminder, each ESEC matrix has 30 rows. The first 10 rows indicates the T/N relations between each pair of the fundamental objects (SEC matrix) and the middle and the last 10 rows indicate the static and dynamic spatial relations between those pairs, respectively. In this section, we need to show always the full version of our 10 manipulation actions matrices. The red marker at the bottom of each matrix marks the column, where the ESEC algorithm made the decision, while the violet marker at the top shows where the most of participants (according to the histograms of fig.5.0.8) have recognized this action. For each action:
1. First we denote the columns, which had been prediction columns for any of the people by light blue color following the distributions in figure 5.14.
2. Then we show all event transitions by a green color. For example if one spatial relation changes from ”MA” to ”HT” in two subsequent columns, we color both cells green. Excep-tions:
• Sometimes one relation changes continuously between the several columns (e.g.“MA”,“HT”,“MT”,... in columns i,i+1, i+2,...). Then we show this by a sequence of green color for that relation in the consecutive columns.
• Sometimes a spatial relation changes in columns i and i+1 and also i+2 and i+3. In these case we show the first transition by light green and the second by dark green color. (e.g.
the spatial relations between two items in 4 continuous columns could be: “Ar”,“ArT”, 67
Chapter 5. Manipulation Action Prediction By Virtual Reality: A Comparison Between Human and ESEC Predictability Power
“ArT”,“To”, in this case the first transition (Ar→ArT in column i and i+1) is light green and the second transition(ArT →in column i+2 and i+3 is dark green.)
3. Also for better differentiation, we have spelt out the event of each blue column below the ESEC matrix in human terms.
Chapter 5. Manipulation Action Prediction By Virtual Reality: A Comparison Between Human and ESEC Predictability Power
1. Chop
Figure 5.19: Chop ESEC Matrix
6)When object 1 starts to penetrate into the object 2.7)When the object 1 touches the ground.
8)When object 1 exits from object 2.9)When object 1 is converted into two parts.10)When object 1 is moving apart from the new created objects.11)When the hand and object 1 are getting close to the ground.12)When the hand puts the object 1 on the ground.13)When the hand untouches object 1.14)When the hand is far away.
69
Chapter 5. Manipulation Action Prediction By Virtual Reality: A Comparison Between Human and ESEC Predictability Power
2. Cut
Figure 5.20: Cut ESEC Matrix
2)When the hand moves above the ground.5)When object 1 starts moving on the surface of object 2.6)When object 1 starts to penetrate into object 2.7)When object 1 starts to leave object 2. 8)When object 1 is divided to two parts. 9)When object 1 is moving apart from the new created objects. 10)When the hand and object 1 are getting close to the ground.
11)When the hand puts object 1 on the ground.12)When the hand untouches object 1.13) When the hand is far away.
Chapter 5. Manipulation Action Prediction By Virtual Reality: A Comparison Between Human and ESEC Predictability Power
3. Hide
Figure 5.21: Hide ESEC Matrix
4)When the hand and object 1 are moving away from the ground.5)When object 1 starts covering object 2.6)When object 1 completely covers object 2 (object 2 is absent).7)When the hand untouches object 1.8)When the hand is far away.
71
Chapter 5. Manipulation Action Prediction By Virtual Reality: A Comparison Between Human and ESEC Predictability Power
4. Uncover
Figure 5.22: Uncover ESEC Matrix
4) When object 1 starts moving away from the ground and the covered object begins to appear.5)When object 1 untouches object 2.6)When the hand and object 1 are getting close to the ground.7)When the hand puts object 1 on the ground.
Chapter 5. Manipulation Action Prediction By Virtual Reality: A Comparison Between Human and ESEC Predictability Power
5. Put
Figure 5.23: Put ESEC Matrix
5)When the hand puts object 2 on object 1.6)When the hand untouches object 2.7)When the hand is far away.8)When the hand leaves the scene.
73
Chapter 5. Manipulation Action Prediction By Virtual Reality: A Comparison Between Human and ESEC Predictability Power
6. Take
Figure 5.24: Take ESEC Matrix
1)When the hand is far away.2)When the hand moves above the ground.3)When the hand touches object 1.4)When object 1 is separated from object 2 by the hand.5)When the hand and object 1 are getting close to the ground.6)When the hand puts object 1 on the ground.7) When the hand untouches object 1.8)When the hand is far away.
Chapter 5. Manipulation Action Prediction By Virtual Reality: A Comparison Between Human and ESEC Predictability Power
7. Lay
Figure 5.25: Lay ESEC Matrix
3)When the hand touches object 1 on the ground.4)When the hand starts laying object 1 on the ground.5)When the hand stops moving object 1.6)When the hand untouches object 1.
7)When the hand is far away.
75
Chapter 5. Manipulation Action Prediction By Virtual Reality: A Comparison Between Human and ESEC Predictability Power
8. Push
Figure 5.26: Push ESEC Matrix
4)When the hand starts pushing object 1.5)When the hand untouches object 1.6)When the hand is far away.
Chapter 5. Manipulation Action Prediction By Virtual Reality: A Comparison Between Human and ESEC Predictability Power
9. Shake
Figure 5.27: Shake ESEC Matrix
4)When the hand separates object 1 from the ground and lifts it.5)When the hand starts shaking object 1.6)When the hand and object 1 are getting close to the ground.7)When the hand puts object 1 on the ground.8)When the hand untouches object 1.9)When the hand is far away.
77
Chapter 5. Manipulation Action Prediction By Virtual Reality: A Comparison Between Human and ESEC Predictability Power
10. Stir
Figure 5.28: Stir ESEC Matrix
7)When object 1 starts stirring object 2. 8)When object 1 starts leaving object 2. 9)When object 2 is exactly on top of object 1. 10)When object 1 untouches object 2. 11)When the hand and object 1 are getting close to the ground.12)When the hand puts object 1 on the ground.13)When the hand untouches object 1.