4.8 Model Extension and Ambiguity
4.8.7 Application to Planar the Snake Robot II
In order to test the extensions on a complex example we turn again to the previously considered planar Snake robot, cf. Section 2.2.7. In Section 4.6 we identified several problems with the control of this system. Large inertial effects revealed the insufficiency of the world model and the system also entered high-frequency oscillations. In the experiment considered here we use the same setup and the same parameter as in Section4.6, but apply the following three extensions:
• the continuity preference for the controller (Section 4.7.2),
• the enhanced world model (Section 4.8.5), and
• the advanced sensor setup as discussed in the previous section (4.8.6).
The robot has 16 segments resulting in m= 15 powered joints and 15 joint angle sensors.
The sensor values received by the controller consist of the 15 angular sensors, as before, and their first derivative, summing up to n = 30 sensors in total. Thus, the controller matrix C has dimensions15×30, the world model matricesA andS have dimensions30×15and 30×30respectively. To get an idea of the complexity let us calculate the dimensionality of the state space and the parameter space. We use the controller implementation in motor space (Section 4.1.2), and thus we have a 15 dimensional state space with intermediate 30dimensions and465 adapting parameters (synaptic weights) of the controller. Together with the1380 parameters of the adaptive world model this sums to1845 synaptic weights.
The conducted experiment last 200 min. Over the entire time we observe frequent changes of behaviors, where the eigen-modes of the physical body are occasionally excited, cf.
ΛCovHxL
20 40 60 80
time
@minD 0.2
0.4 0.6 0.8 1.0
Figure 4.30: Eigenvalues of the covariance matrix of theSnake’s sensor values.
These are the variances along all 15 principal components of the sensor data of 5 minute sliding windows. All principal components remain active, reflecting high-dimensional be-haviors. Parameters: C =A= 0.01,√
E, update rate 100 Hz, γc= 0.001,δ= 0.005.
[Video 9]. In Fig.4.30the eigenvalues of the covariance matrix of the sensor values of 5 min intervals are presented. The size of the eigenvalues reflects the variance in the 15 principal components. All eigenvalues remain non-zero and they almost equally span over the interval 0 to 1. This is in contrast to the experiment in Section4.6, where the eigenvalue spectrum broke down after some time (Fig. 4.19). We can conclude that the behaviors exhibited within 5 min keep a high dimensional structure. A more detailed view on the major two principal components is given in Fig.4.31, now for intervals of 2 min. The first two principal components scaled by their relative variances are plotted in Fig. 4.31(b),(c). The vectors are preprocessed to have a positive sign in the first few components, in order to increase clarity. Very different major principal components are exhibited over time. However, the main eigen-modes of the physical system are frequently excited in a similar manner as observed before. The normalized eigenvalues of the covariance matrix, Fig. 4.20(a), show that at the time when theses modes occur the first mode dominates the behaviors more strongly (brighter stripes in the rows 3-15).
Let us have a look at the parameter dynamics. The controller matrixCis no longer square, since the derivatives of the sensors values are used as well. We consider a partition ofC into two 15×15square blocks, thus C= (C1 C2). C1 connects joint angles with motor values.
The eigenvalues of this matrix and of the linearized system matrix are displayed in Fig.4.32.
All real parts of the eigenvalues of C1 stay close to 1 and the two smallest eigenvalues have non-zero complex parts. Nevertheless, the eigenvalues of the system matrix R have very small complex components. This shows again that the Snake robot cannot perform high frequency oscillations. The continuity preference extension is not able to suppress these complex parts because they do not yet lead to oscillations in the motor values, since they belong to the smallest eigenvalues. This is in contrast to the case without the continuity preference in Fig. 4.22, where the largest eigenvalues have large complex components.
Finally, let us consider the parameters of the system at the end of the experiment. For that we plot the matrices C, A, and S in Fig. 4.33. The controller matrix C essentially maps the joint angles and its derivatives to the motor values, which are nominal joint
4.8. Model Extension and Ambiguity 113
Figure 4.31: Principal components of the Snake’s sensor values when using the extended world model. The time is given in minutes. The covariance matrices have been computed on the sensor data from 2 min each. (a) Each column shows the normalized vector of eigenvalues of the covariance matrix, i. e.~λCov(x)/|~λCov(x)|;(b)Each column shows the scaled eigenvectoru1corresponding to the largest eigenvalueλ1(first row of(a)),u1Cov(x)λCov(1 x)/|~λCov(x)|. (c)The second eigenvector scaled with the corresponding normalized eigenvalue; (d)Two columns of(b) at min 8 (A) and 144 (B) enlarged.
From time to time the eigen-modes of the physical system are still excited, as marked with A and B. For more details see Fig.4.20(p.85).
(a)
Figure 4.32: Eigenvalues of the controller matrix and linearized system matrix over time for the Snake robot. The time is given in minutes and the curves show sliding averages of 3 min intervals. (a)Eigenvalues of left square 15×15block of C. The solid lines show the real part and the dashed lines show the imaginary part (only 2 are non-zero); (b) Eigenvalues of linearized system matrix in motor space R = CA and the TLE E (blue, right axes). Parameters: C =A= 0.02, update rate100Hz,γc= 0.001.
Figure 4.33: Controller and extended world model matrices at the end of the Snakerobot experiment. All matrices are taken at minute 200. (a)Controller matrix Cmapping joint angles (first 15) and their derivatives (angular velocities) (16-30) to motor values (neglecting the non-linearity). A clear separation of the joint angles (left square blockC1) and derivatives (right square blockC2) takes place. Besides the strong diagonal elements ofC1, there are long range connections (yellow and blue areas). The derivatives (C2) are actually negatively coupled to the motor values, reflecting a damping;(b)World model matrix A. It maps motor values to future sensor values. The lower square block (A2) is essentially zero because there is no correspondence between motor values (nominal joint angles) and angular velocities; (c)World model matrix S (extension of the original model). It maps actual sensor values to future sensor values. Note the different color code to account for the smaller elements due to the discount.
4.8. Model Extension and Ambiguity 115 angles (neglecting the bias term and the non-linearity in Eq. (4.18)). As above we consider a partition of C into two 15×15 blocks, i. e. C = (C1 C2). Thus, C1 maps joint angles to nominal joint angles and shows mainly a diagonal structure as expected. Connections between neighboring joints (secondary diagonals) are negative, but very weak. This cor-responds to a zig-zag shape of the robot. Moving in such a zig-zag pattern reduces the inertial effects, because the movements are locally compensated. In contrast, imagine that a joint in the center of the Snake robot is controlled to change its angle independent of the other joints. Then the entire body has to be moved which causes a large inertial torque. There are also long range connections formed, as indicated by the pale yellow and blue areas at both sides of C1 in Fig. 4.33(a). These connections lead the whole body movements indicated by the principal components in Fig.4.31(d). The second block shows that the angular velocities (sensors 16-30) are coupled inhibitory to the belonging joints (diagonal of C2). This reflects a damping of the movement speed, which is not only a consequence of the continuity preference extension of the controller, but also of the fact that high angular velocities quickly lead to the saturation region of the controller neurons at high joint angles. However, the angular velocities of the left and right neighbors have an excitatory influence. Note that behavioral changes in such highly physically coupled systems can be achieved with small changes in the controller connections.
The world model matrixA, see Fig.4.33(b), maps motor values to future sensor values and thus has the dimension 30×15. However, the lower square block (A2) is essentially zero because there is no direct correspondence between motor values and joint velocities. The new world model matrixS, Eq. (4.91), represents the learned mapping from current sensor values to future sensor values, see Fig. 4.33(c). Its elements are much smaller because of the bias towards self-induced observations realized by the discount factor δ = 0.005, Eq. (4.108). Let us consider the matrix in four15×15blocks namelyS1, . . . , S4as indicated by the dashed lines in the plot. The diagonal elements of S1 have small negative values which is due to the following relationship: The new angular position of a joint is the desired position (given by matrix A) minus some fraction of the old position (negative values in S1) due to inertia. The top-right sub-matrixS2 represents the mapping from angular joint velocities to angular positions. This is basically determined by the C2. Interestingly, not the neighbors, but the second next neighbors are coupled inhibitory. This is due to the physical interaction of the joints. If one joint increases its angle, then the second next joint is pulled, such that its angle decreases as illustrated in Fig. 4.34.
4.8.8 Summary
To summarize, the extended world model leads to a decreased prediction error, resulting in a smooth and stable parameter dynamics in the here considered high-dimensional Snake robot. The bias towards self-induced interpretation of the observed sensor values which was implemented into the extended world model, has shown to be effective. This was reflected by the large diagonal elements of the A matrix. Nevertheless, the inertial effects
l l
θ1 θ2 θ3 θ˜1 θ˜2 θ˜3
Figure 4.34: Illustration of physical joint interactions of the Snake robot. We consider 4 segments within a longer Snake robot and assume that the endpoints (blue dots) of the considered part of the robot are fixed, reflecting the inertial effects of the following segments. If the angle of the first considered joint (θ1) is decreased (˜θ1 < θ1) then the third joint angle increases (θ˜3 > θ3) whereas the second joint stays almost the same. Thus, a change in one joint angle effects the second next joints in the opposite direction. Note that the joint sensor values are actually π−θi (0 for a straight joint), see Section2.2.7.
and intrinsic joint interaction of the physical system were captured by the additional weight matrix S of the world model. The extension of the controller to prefer continuous motor values has shown the intended effect and no high-frequency oscillations have been exhibited.
We also demonstrated that due to the extended world model it is possible to use additional sensors, like the angular velocity sensors used in the previous experiment. The detailed analysis of the controller and world model structure has shown that the additional sensors have been integrated into the sensorimotor dynamics in a reasonable fashion. We also observed how the morphology of the physical body shapes the controller, such that the eigenmodes of system are excited, but also a lot of other behaviors are shown.