Stability Analysis

The control law (4.2) leads to the closed-loop collective dynamics

˙ We analyze the stability in two steps:

Part A: Define a dissipative Hamiltonian system, assuming that the agents can measure the gradient of the scalar field locally without any communication with neighbors, and analyze this system.

Part B: Show that the proposed protocol leads to a system that can be seen as a per-turbed evolution of the dissipative Hamiltonian system defined in Part A and prove that under some assumptions, the states remain bounded.

[Part A]: We consider a scenario where each agent can measure the gradient ∇ψ(qi) locally and apply the following control law:

u^A_i (t) =X where Ψ : R^{N m} −→ R is the scalar field whose argument comprises the positions of all agents and is defined as Ψ(q) = P

iψ(qi), and ∇Ψ(q) ∈ R^{N m} is defined as ∇Ψ(q) =

4.1. GRADIENT-FREE SOURCE SEEKING USING FLOCKING BEHAVIOR 49 [∇ψ(q1),∇ψ(q2), . . . ,∇ψ(qN)]^T and for the scalar case, this is reduced to

∇Ψ(q) = [^dψ(q_dq¹⁾,^dψ(q_dq²⁾, . . . ,^dψ(q_dq^N)]^T.

Remark 1: For general scalar fields ψ(qi), it is not possible to decompose ∇Ψ(q) as in [Olfati-Saber, 2006]. Hence, the translation and structural dynamics cannot be de-coupled and analyzed independently.

Remark 2: For quadratic scalar fields, e.g.,ψ(qi) =−q^THq, whereH is symmetric positive definite, we obtain linear forcing terms ∇ψ(qi) =−2Hq, which allow the decomposition as suggested in [Olfati-Saber, 2006] and facilitates the analysis.

Decoupling the translational and structural dynamics completely, for general case, is dif-ficult. Therefore, to characterize the system’s equilibria, we produce the agents center-of-mass dynamics. Let qc, pc∈R^mbe two center-of-mass variables, where

qc= 1

N(1^T⊗Im)q, (4.6)

pc= 1

N(1^T⊗Im)p. (4.7)

The translational dynamics of the center of mass are then

˙

We can observe that, for the translation dynamics of the center of mass to be in equilibrium, the positions q must satisfy (1^T ⊗Im)· ∇Ψ(q) = 0. The following Lemma characterizes the equilibrium for a special case when the scalar fieldψ(qi) is radially symmetric about the stationary pointqs:= argmaxqψ(qi).

Let, for a givenq= [q^T₁, q^T₂, . . . , q^T_N]^T, convexhull(q) :={q∈R^m|∃α1, α2, . . . , αN ∈[0,1], s.t. P

iαi= 1 andq=P

iαiqi}.

Lemma 4.1. Assume that the scalar field satisfies assumptions A1,A2. Additionally, if it is radially symmetric, i.e., ψ(x) =ψ(y) ∀x, y, s.t||x−qs||=||y−qs||,

then all possible equilibria of the translational dynamics of the center-of-mass correspond toq^∗ such thatqs∈convexhull(q^∗).

Proof. Without loss of generality, we assume that the source qs is at the origin. A ra-dially symmetric scalar field ψ(qi) can be represented in polar coordinates by a function ψr:R+→Rsuch thatψr(r) =ψ(qi)∀ ||q||=r. For||q|| 6= 0,∇ψ(qi) = ^dψ_dr^r(r)·_||^qq|| and from concavity of ψ(qi), we can show that ψr(r) is concave with the maximum located at r=rs =||qs||= 0 from our earlier assumption. Similarly, ^dψ_dr^r(r) <0 ∀r6= 0. Hence,

∇ψ(qi) is always in the direction of−q, ∀q6= 0.

Let us assume that there exists an equilibrium of the translational dynamics of the center of mass corresponding toq^∗, such thatqs∈/ convexhull(q^∗). Hence, there exists a separating hyperplane such thatqsis located on the opposite half-space to that ofqi ∀i∈1,2, . . . , N.

50 CHAPTER 4. FLOCKING-BASED SOURCE SEEKING Define a unit vectorhperpendicular to the separating hyperplane and pointing towards qs; then, because all agents belong to the opposite side of the hyperplane, note that q_i^Th <0 ∀i. The forcefiacting on agentialonghis given by

fi= (∇ψ(qi))^Th=dψr(ri) dr · q_i^Th

||qi|| =ki·q_i^Th,

where ki:= ^dψ_dr^r(ri)·_||q¹_i|| <0 ∀||q|| 6= 0 andq^T_i h <0. Therefore, the force on each agent alonghis strictly positive. Hence, the net forcefhon the center of mass alonghis

fh=f1+f2+· · ·+fN >0.

Therefore, this cannot be an equilibrium of the translational dynamics of the center of mass. We have thus shown by contradiction that for all equilibria, there exists no hyper-plane separating the source qs and the positions of the agents qi. Then, by considering hyperplanes sequentially, the source must lie in the convexhull(q^∗).

A similar result can be reached by replacing the assumption of radial symmetry with strong concavity of the scalar fieldψ, which is weaker. Now, we have the following theorem.

Theorem 4.1. Under protocol (4.4), almost all trajectories asymptotically converge to the equilibrium (q^∗,0)characterized by ∇V(q)− ∇Ψ(q) = 0. For the special case when ψ(qi) is radially symmetric, the trajectory converges to the equilibrium characterized by Lemma 4.1.

Proof. Straightforward application of La Salle’s invariance theorem.

Organize equation (4.5) as

˙ q=p

˙

p=−∇U(q)−( ˆL(q) +c)p, (4.9)

where U(q) =V(q)−Ψ(q) with the corresponding Hamiltonian H(q, p) =U(q) +K(p).

Notice that the Hamiltonian is constructed from the potentialU(q) andK(p) =¹₂PN i=1||pi||² is a dissipative kinetic energy. In this case, the isolated local minima ofU(q) correspond to the stable equilibrium point. Derivation ofH yields

H˙ = ˙q^T∇U+p^Tp˙ solution asymptotically converges to an equilibriumq^∗a minima of the potential function U(q).

[Part B]:Now, we relax the assumption that each agent can independently measure the gradient locally. We want to analyze the gradient-free protocol proposed earlier, and show that this protocol leads to dynamics that can be seen as a modification of the

4.1. GRADIENT-FREE SOURCE SEEKING USING FLOCKING BEHAVIOR 51 evolution of the collective dynamics shown in Part A.

The dynamics in (4.3) can be written as

˙

Then, we have the following theorem.

Theorem 4.2. Assume that is possible to tuners,d,a, b, andk such that there exists a constantC >0where||e||∞≤C ∀q. Then the proposed protocol leads to stable dynamics.

Moreover, it behaves qualitatively similar to the protocol proposed in Part A.

Proof. Consider the total energy of the system.

T(q, p) =V(q)−Ψ(q) +1

2p² (4.12)

By differentiating with respect to time, we obtain T˙(q, p) =(∇V(q)− ∇Ψ(q))^Tp+p^Tp˙

=(∇V(q)− ∇Ψ(q))^Tp

+p^T(−∇V(q)−L(q)pˆ −c·p+∇Ψ(q) +e(q))

=−p^TL(q)pˆ −p^Tcp+p^Te(q).

(4.13)

We will now use dissipative arguments. From the fundamental theorem of calculus, we have

52 CHAPTER 4. FLOCKING-BASED SOURCE SEEKING Note that−Ψ(q) is a convex function with a maximum slope ofC. Therefore,||e||∞≤C and ψattains slopeC for someq. It can be shown that the set of pointsq satisfying the above inequality is compact.

Remark 3: This set can be geometrically visualized as the set of points where the hyper-planeK1+||e||∞1^Tqis above−Ψ(q). Hence, there exists a boundqmax (supremum over allqalong the intersection of the two surfaces mentioned in the above remark) such that ||q|| ≤qmax for all time.

Remark 4: It is possible to use the information about the scalar field to develop algorithms that reduce the bound on ||e||∞ using, for example, approximation theory for functions.