In this section we want to formulate the sensorimotor loop as a dynamical system and illustrate some basic properties of a controller with a single neuron. Let us assume for the moment that the world is known and is Markovian, i. e. without dependence on past values. We can write
xt+1 =W(yt, xt, t). (3.4)
3.4. Dynamical Systems Formulation of the Sensorimotor Loop 37 Based on the definitions (3.2), (3.4) we can express the sensorimotor loop in a closed form as
xt+1 =W(K(xt), xt, t). (3.5)
Since we want to restrict ourselves in this chapter to the one-dimensional case we have i. e.
m=n= 1. Let the controller K be given by the following function
yt =K(xt) = tanh (cxt+h), (3.6)
that represents a rate-based neuron with hyperbolic tangent activation function, the synap-tic connection strength and the bias h. In general, the controller is a one-layer neural network with a weight matrix. Let us use a simplified world that is defined as
xt+1 =W(yt, xt, t) = αyt+ςt, (3.7) where ςt is a zero mean noise process. In this example α is a hardware constant. The full system equation reads
xt+1 =αtanh (cxt+h) +ςt. (3.8)
This equation is very similar to the description of a single neuron with an excitatory self-connection in an open-loop setup. The dynamics of such a neuron was investigated in detail in [112,113, 131]. Nevertheless, let us now find the fixed points of the dynamics and analyze their parameter dependence. For this, we rewrite the equation (3.8) in terms of the membrane potential
zt=cxt+h. (3.9)
By neglecting the noise we obtain
zt+1 =rtanh(zt) +h with r=cα . (3.10) The fixed points are most easily determined graphically, as displayed in Fig.3.4. However, an analytical solution is also possible using the series expansion of tanh. We used the expansion up to 12th order i. e.tanh(z) =z−z33+215z5−17315z7+622835z9 −1382155925z11+O(z13). Note that this series converges slowly and the expansion to the 12th order is only accurate for z ∈[−1.5,1.5]. It is also important to stop the series at a term with negative coefficient, because otherwise the approximation erroneously produces more fixed points.
The solution of Eq. (3.10) is plotted as a bifurcation diagram for the parametersr andhin Fig.3.5 which shows a cusp bifurcation [2]. A cusp bifurcation usually appears in systems that are topologically equivalent to
˙
z =β1+β2z−z3. (3.11)
-1.5 -1 -0.5 0 0.5 1 1.5
-1.5 -1 -0.5 0 0.5 1 1.5
0.5 tanh(z)z 1.2 tanh(z) 1.2 tanh(z) + 0.2
Figure 3.4: Graphical solution for the fixed points ofz=rtanh(z)+h, Eq. (3.10) for different values of r and h. Forr = 0.5 (green line) we find only the fixed point at z= 0 (stable). For r = 1.2 and h = 0 (blue line) there are two stable fixed points at z±0.79 and the fixed point at0 becomes unstable. If additionally h = 0.2 (black line), only one fixed point remains at z≈1.2.
Writing Eq. (3.10) in terms of a differential equation
˙
z =rtanh(z) +h−z (3.12)
and using only the first two terms of the series expansion of hyperbolic tangent:
tanh(z) =z− z3
3 +O(z5), (3.13)
we obtain
˙ z =r
z− z3
3
−z+h, (3.14)
which is equivalent to Eq. (3.11) for β1 = 3h and β2 = 3(r−1). It is important to note that Eq. (3.14) has the same qualitative bifurcation behavior than Eq. (3.12).
Thus, we can draw upon the results obtained for such systems and give the equation for the locations of the saddle-nodes for the approximated system as
h=±2
3(−1 +r)3/2, r >1. (3.15)
The saddle-nodes are also seen in Fig.3.5at the line where the red and green surface meet.
For better illustration Fig. 3.6 shows different sections of the fixed point structure. The positions of the saddle notes, Eq. (3.15), show the typical cusp wedge, see Fig.3.6(d). For r <1the system has only one stable fixed point. We call the system subcritical, since the fixed point is at smallz and thus little activity occurs in the sensorimotor loop. Forh= 0 the system shows a pitchfork bifurcation point at r = 1, see Fig. 3.6(a). For h 6= 0 two
3.4. Dynamical Systems Formulation of the Sensorimotor Loop 39
Figure 3.5: Bifurcation diagram for z = rtanh(z) +h, Eq. (3.10), as a surface.
Colors: blue, green are stable fixed points andredis unstable.
(a)
0.5 1.0 1.5 2.0 r
-1.5 -1.0 -0.5 0.5 1.0 1.5 z
(b)
0.5 1.0 1.5 2.0 r
-1.5 -1.0 -0.5 0.5 1.0 z
(c)
-0.2 -0.1 0.1 0.2 h -1.0
-0.5 0.5 1.0 z
(d)
1.1 1.2 1.3 1.4 1.5 r -0.2
-0.1 0.1 0.2 h
Figure 3.6: Bifurcation diagrams for z =rtanh(z) +h, Eq. (3.10). (a)Section at h = 0; (b) Catastrophic bifurcation at h =−0.1; (c) The hysteresis in dependence of h in the supercritical regime at r = 1.2, also indicated in Fig. 3.5. (d) Typical cusp wedge showing the saddle nodes in dependence of r andh, cf. Eq. (3.15). The colored points are correspondingly marked in (b,c). Colors: blue, green are stable fixed points and red marks unstable fixed points.
branches emerge, see Fig. 3.6(b), and the bifurcation becomes catastrophic. Catastrophic bifurcation means that the system state can undergo a drastic transition for small changes in parameter values. To illustrate the consequences, Fig. 3.6(c) displays the bifurcation diagram for the supercritical parameter r = 1.2 in dependence of h and shows a clear hysteresis effect. This means that the system resides in its fixed point when the parameter h is slowly decreased or increased until the fixed point disappears and the state jumps to the fixed point with the opposite sign (dashed lines). When the parameter his changed in the other direction the same behavior is observed. Hence, for one parameter configuration the system can be in two possible states and h can be used to force a transition.
To summarize, given a simple closed-loop system with a single nonlinear neuron we find non-trivial fixed points and hysteresis depending on the control parameters. This type of dynamics is found in many systems, e. g. in statistical mechanical models of magnets, see [157]. For robot control it seems suitable to have r slightly above the bifurcation point in order to have two stable fixed points which can be changed either by changing h or by external influence. It is also important that the basins of attraction are large enough to avoid noise induced switches. In the case of multiple sensors and motors we get, of course, a much larger number of attractors and limit-cycles, which will be demonstrated by application to robotic hardware later in this work.