In this section, we explain how the finite horizon OCPPN(x0)can be used in order to construct an approximately optimal feedback law for the infinite horizon problemP∞(x0).
The ’usual’ or ’standard’ MPC algorithm proceeds iteratively as follows.
Algorithm 1.3.1. (Standard MPC)
(1) Measure the state x(k)∈Xof the system at time instantk
(2) Setx0:=x(k)and solve the finite horizon problemPN(x0). Let u∗(·)∈ UN(x0)denote the optimal control sequence and define the MPC feedback law
µN(x(k), k) :=u∗(0)
1.3. MPC algorithms
(3) Apply the control valueµN(x(k), k)to the system, setk:=k+ 1and go to (1)
This iteration, also known as a receding horizon strategy, gives rise to a non-time-dependent feedbackµN which — under appropriate conditions, see Section 2.1 — approximately solves the infinite horizon problem. It generates a nominal closed-loop trajectoryxµN(k)according to the rule
xµN(k+ 1) =f(xµN(k), µN(xµN(k), k)) (1.6) In this work, we consider two other variants of MPC controllers. First, we considermultisteporm-step feedback MPC(see [32]),m∈ {2, . . . , N−1} in which the optimization in Step (2) is performed less often, by applying the firstm elements of the optimal control sequence obtained after optimization.
Algorithm 1.3.2. (Multisteporm-step MPC)
(1) Measure the state x(k)∈Xof the system at time instantk
(2) Setx0:=x(k)and solve the finite horizon problemPN(x0). Let u∗(·)∈ UN(x0)denote the optimal control sequence and define the time-dependent m-step MPC feedback
µN,m(x(k), k+j) :=u∗(j), j= 0, . . . , m−1 (1.7) (3) Apply the control valuesµN,m(x(k), k+j),j= 0, . . . , m−1, to the system,
setk:=k+mand go to (1)
Remark 1.3.3. Observe that through the scheme, the loop is only closed every m-steps, i.e., the system runs in open-loop withinm-steps before optimization is performed again to compute a new set of controls.
Here, the valuemis called the control horizon. The resulting nominal closed-loop system is given by
xµN,m(k+ 1) =f(xµN,m(k), µN,m(xµN,m(bkcm), k)) (1.8) withk(k) =˜ bkcm for the notation introduced in (1.2) wherebkcm denotes the largest integer multiple ofm less than or equal tok. The motivation behind consideringm-step MPC is that the number of optimizations is reduced by the factor1/m, thus the computational effort decreases accordingly.
Second, we also consider the updated multistep feedback MPC which, similar to the usual MPC, entails performing optimization every time step, but unlike the standard MPC, wherein we perform optimization over full horizonN, we re-optimize over shrinking horizons.
Algorithm 1.3.4. (Updatedm-step MPC)
(1) Measure the state x(k)∈Xof the system at time instantk
(2) Setj:=k−bkcm,xj:=x(k)and solve the finite horizon problemPN−j(xj).
Letu∗(·)∈UN−j(x0)denote the optimal control sequence and define the updated MPC feedback
ˆ
µN,m(x(k), k) :=u∗(0) (1.9)
Chapter 1. MPC setting and preliminaries
(3) Apply the control value µˆN,m(x(k), k)to the system, setk:=k+ 1and go to (1)
The nominal updated multistep MPC closed loop is then described by
xµˆN,m(k+ 1) =f(xµˆN,m(k),µˆN,m(xµˆN,m(k), k)) (1.10) We note that due to the dynamic programming principle in Theorem 1.2.1, in the nominal setting the closed loop generated by the multistep feedback (1.8) and by the updated multistep feedback MPC closed-loop system (1.10) coincide. For this reason, the use of Algorithm 1.3.4 only becomes meaningful in the presence of perturbations. These will be formalized in Section 2.2.
In presence of perturbations, we expect the updated multistep feedback to provide more robustness, in the sense that stability is maintained for larger perturbations and performance degradation is less pronounced as for the non-updated case.
This will be rigorously analyzed in Chapter 4. Compared to standard MPC, the optimal control problems on shrinking horizon needed for the updates are faster to solve than the optimal control problems on full horizon. Moreover, for small perturbations the updates may also be replaced by approximative updates in which re-optimizations are approximated by a sensitivity approach, leading to another significant reduction of the computation time. This variant is analyzed in Chapter 6.
2 MPC stability and performance
This chapter provides the fundamental theorems that will serve as the basis of the analysis that we will conduct on various MPC schemes. In Section 2.1, we present some established results in the analysis of nominal MPC (see e.g., [36, 32, 37]) consisting of statements on stability guarantees and performance in terms of suboptimality with respect to the infinite horizon problemP∞(x0). We aim to apply the MPC variants on real systems and for this reason, we introduce in Section 2.2 perturbed systems, as opposed to nominal systems. After having summarized the main steps of the analysis of the nominal MPC without terminal constraints, we adapt the statements to the analysis of feedback laws under perturbations.
2.1 Nominal stability and performance
Supposex∗ is an equilibrium of (1.1). MPC determines µ: X×N→U that approximately solves the infinite horizon OCP such thatx∗ is asymptotically stable for the feedback-controlled system (1.6) in the following sense.
Definition 2.1.1. An equilibrium x∗ ∈Xis asymptotically stable for the closed-loop system (1.2) if there existsβ∈ KLsuch that
kxµ(k, x0)kx∗≤β(kx0kx∗, k)
holds for allx0∈Xand allk∈N0 where kxkx∗:=kx−x∗k. In this case, we say that the feedback lawµasymptotically stabilizesx∗.
Conditions ensuring that the MPC feedback law asymptotically stabilizes the system have been well-developed in the literature. On one hand, refer, e.g., to [57], [36, Chapter 5] and references therein, we see that employing stabilizing terminal constraints or adding Lyapunov function terminal costs to the objective function ensure asymptotic stability of the MPC closed loop. On the other hand, see, e.g., [32], [37] and [36, Chapter 6] and references therein, we observe that imposing such terminal constraints and costs are not necessary conditions for achieving stability. In addition, due to the simplicity in design and implementation, MPC without terminal constraints and costs is often preferred in practice and with this motivation, we will be interested in analyzing the properties of MPC without terminal conditions in this thesis.
To achieve asymptotic stability, an appropriate choice of the stage cost`is needed
Chapter 2. MPC stability and performance
and is typically obtained by penalizing the distance of the state to the desired equilibrium and the control effort. This is enforced by making the following assumption.
Assumption 2.1.2. There existK∞-functionsα1, α2 such that the inequality α1(kxkx∗)≤`∗(x)≤α2(kxkx∗) (2.1) holds for allx∈X, where`∗(x) := infu∈U`(x, u).
The following gives the key statement for the analysis of MPC without terminal constraints or costs.
Proposition 2.1.3. (i) Consider a time-dependent feedback lawµ:X×N→U, the corresponding solution xµ(k, x0) of (1.2), and a function V : X → R+0
satisfying therelaxed dynamic programming inequality V(x0)≥V(xµ(m, x0)) +α
m−1
X
k=0
`(xµ(k, x0), µ(xµ(bkcm, x0), k)) (2.2) for someα∈(0,1], somem≥1and allx0∈X. Then for allx∈Xthe estimate V∞(x)≤J∞cl(x, µ)≤V(x)/α (2.3) holds.
(ii) If, moreover, Assumption 2.1.2 holds and there exist α3, α4∈ K∞ such that α3(kxkx∗)≤V(x)≤α4(kxkx∗)
for allx∈X, then the equilibriumx∗ is asymptotically stable for the closed-loop system.
Proof. (i) The proof follows [32, Proof of Proposition 2.4]. Considerx0∈Xand the closed-loop trajectoryxµ(k, x0). Then from (2.2) we obtain for alln∈N0
α
m−1
X
k=0
`(xµ(nm+k, x0), µ(xµ(bnm+kcm, x0), nm+k))
≤ V(xµ(nm, x0))−V(xµ((n+ 1)m, x0)) Performing a summation overngives
α
Km−1
X
k=0
`(xµ(k, x0), µ(xµ(bkcm, x0), k))
= α
K−1
X
n=0 m−1
X
k=0
`(xµ(nm+k, x0), µ(xµ(bnm+kcm, x0), nm+k))
≤ V(x0)−V(x(Km, x0)) ≤ V(x0)
The leftmost sum is bounded from above for everyK∈Nand is monotonically increasing which implies convergence asK→ ∞, therefore
V∞(x)≤J∞cl(x, µ)≤V(x)/α
2.1. Nominal stability and performance
(ii) Following [32, Proof of Theorem 5.2], by standard construction (see [43, Section 4.4]) we obtain a functionρ∈ KLsuch thatV(xµ(km, x0))≤ρ(V(x0), k) holds for allx0∈X. Now considerk∈Nwhich is not an integer multiple of m.
By (2.2) withxµ(bkcm, x0)in place ofx0 and the nonnegativity of`, we have
`(xµ(k, x0), µ(xµ(bkcm, x0), k))≤V(xµ(bkcm, x0))/α SinceV(x)≤α4◦α−11(`(x, u))holds for allu, we obtain
V(xµ(k, x0)) ≤ α4◦α−11(V(xµ(bkcm, x0))/α)
≤ α4◦α−11 (ρ(V(x0),bkcm)/α) which yields
kxµ(k, x0)kx∗ ≤α−31◦α4◦α−11(ρ(α4(kx0kx∗),bkcm)/α)
Therefore,kxµ(k, x0)kx∗≤β(kx0kx∗, k)for allk∈N, i.e., the desired asymptotic stabilty withKL-function
β(r, k) :=α−13 ◦α4◦α−11 (ρ(α4(r),bkcm)/α) +e−k
which is easily extended to aKL-function by linear interpolation in its second argument.
In Proposition 2.1.3, to show asymptotic stability of a closed-loop system driven by µN,m, we need to show existence of a function V and a value α ∈ (0,1]
satisfying the relaxed dynamic programming inequality (2.2). The use of the relaxed dynamic programming inequality in the form (2.2) was first introduced for the analysis of MPC schemes in [38]. Other forms, however, were earlier used in [60].
Proposition 2.1.3 implies that aside from providing the estimate (2.3) (on which a so-called suboptimality estimate, discussed towards the end of the section, will be based), showing the existence of a positiveαalso ensures asymptotic stability for the closed-loop system. In the sequel, we examine the feedback lawµN,m
and considerV :=VN. We present in the following an approach of computingα.
One way to obtainαis by requiring the following assumption.
Assumption 2.1.4. There existsBk∈ K∞such that the optimal value functions ofPk(x0)satisfy
Vk(x)≤Bk(`∗(x)) for all x∈Xand all k= 2, . . . , N
Remark 2.1.5. The existence of the functionsBkcan be concluded, for instance, by assuming certain controllability assumptions. See, e.g., [36, Assumption 6.4]
or [66, Assumption 3.2 and Lemma 3.5] wherein the system is assumed to be asymptotically controllable with respect to `, i.e. if there existsβ ∈ KL0 such that for everyx∈Xand everyN ∈N, there exists an admissible control sequenceux∈UN(x)satisfying
`(xux(k, x), ux(k))≤β(`∗(x), k) for allk∈ {0, . . . , N−1}.
Example 2.1.6. Suppose there exist constantsC >0andσ∈(0,1) such that
Chapter 2. MPC stability and performance
for everyx∈Xand everyN ∈N, there isux∈UN(x)such that
`(xux(k, x), ux(k))≤Cσk`∗(x)
for allk∈ {0, . . . , N−1}. Then we takeβ(r, k) =Cσkr∈ KL0 givingBN(r) = PN−1
k=0 β(r, k) =CPN−1
k=0 σkr that fulfills Assumption 2.1.4. In this case, the system is said to be exponentially controllable with respect to`.
The following proposition considers arbitrary valuesλn, n= 0, . . . , N−1, andν and gives necessary conditions which hold if these values coincide with optimal stage costs`(xu∗(n, x0), u∗(n)) and optimal valuesVN(xu∗(m, x0)), respectively.
Proposition 2.1.7. Let Assumption 2.1.4 hold and consider N ≥ 1, m ∈ {1, . . . , N−1}, a sequenceλn≥0, n= 0, . . . , N−1, a valueν ≥0. Consider x0∈X and assume that there exists an optimal control function u∗(·)∈Ufor the finite horizon problemPN(x0) with horizon lengthN, such that
λn=`(xu∗(n, x0), u∗(n)), n= 0, . . . , N−1 holds. Then
N−1
X
n=k
λn≤BN−k(λk), k= 0, . . . , N −2 (2.4) holds. If, furthermore,
ν=VN(xu∗(m, x0)) holds, then
ν≤
j−1
X
n=0
λn+m+BN−j(λj+m), j= 0, . . . , N −m−1 (2.5) holds.
Proof. Observe that fork= 0, . . . , N−2,
VN(x0) = Jk(x0, u∗(·)) +JN−k(xu∗(k, x0), u∗(k+·)) (2.6)
= Jk(x0, u∗(·)) +VN−k(xu∗(k, x0)) (2.7) by (2.7) and Assumption 2.1.4, we have
VN(x0)≤Jk(x0, u∗(·)) +BN−k(`(xu∗(k, x0))) (2.8) Subtracting (2.6) from (2.8) gives
JN−k(xu∗(k, x0), u∗(k+·))≤BN−k(`∗(xu∗(k, x0))) yielding (2.4). Next we define the control function
˜ u(n) =
u∗(m+n), n≤j−1 u∗∗(n), n≤j
2.1. Nominal stability and performance
whereu∗∗(·)is the optimal control forPN−j(xu∗(m+j)). Then we obtain VN(xu∗(m, x0)) = JN(xu∗(m),u(˜ ·))
= Jj(xu∗(m, x0), u∗(m+·)) +JN−j(xu∗(m+j, x0), u∗∗(·))
= Jj(xu∗(m, x0), u∗(m+·)) +VN−j(xu∗(m+j, x0))
≤ Jj(xu∗(m, x0), u∗(m+·)) +BN−j(`∗(xu∗(m+j, x0))) yielding (2.5).
By using the proposition, we arrive at the following theorem giving sufficient conditions for suboptimality and stability of them-step MPC feedback lawµN,m
and an approach to compute the suboptimality indexα.
Theorem 2.1.8. Let Assumption 2.1.4 hold and assume that the optimization problem
α:= inf
λ0,...,λN−1,ν
PN−1 n=0 λn−ν Pm−1
n=0 λn
subject to the constraints (2.4) and(2.5) and Pm−1
n=0 λn>0, λ0, . . . , λN−1, ν≥0
Pα
has an optimal valueα∈(0,1]. Then, the optimal value function VN of PN(x) and the m-step MPC feedback law µN,m satisfy the assumptions of Proposi-tion 2.1.3(i) and, in particular, the inequality
V∞(x)≤J∞cl(x, µN,m)≤VN(x)/α≤V∞(x)/α (2.9) holds for allx∈X. If, moreover, Assumption 2.1.2 holds then the closed loop is asymptotically stable.
Proof. From the solutionu∗(·)ofPN(x0)forx0∈X, we construct them-step feedbackµN,mgiving the equalities
µN,m(x0, k) =u∗(k), k= 0, . . . , m−1 xµN,m(k, x0) =xu∗(k, x0), k= 0, . . . , m
`(xµN,m(k, x0), µN,m(x0, k)) =`(xu∗(k, x0), u∗(k)), k= 0, . . . , m−1 which implies
VN(xµN,m(m, x0)) +α
m−1
X
k=0
`(xµN,m(k, x0), µN,m(xµN,m(k, x0), k))
=VN(xu∗(m, x0)) +α
m−1
X
k=0
`(xu∗(k, x0), u∗(k)) (2.10) for anyα∈R. SincePαhas a solution, the valuesλk =`(xu∗(k, x0), u∗(k)) and ν=VN(xu∗(m, x0)satisfy (2.4), (2.5) and
N−1
X
k=0
λk−ν ≥α
m−1
X
k=0
λk
Chapter 2. MPC stability and performance
Together with (2.10), this yields (2.2) and thus the assertion. The second assertion follows from Proposition 2.1.3(ii) settingα4:=BN.
Because of (2.9), we refer toαas anindex of suboptimalitywhich provides a performance bound indicating how well the feedback lawµN,m approximates the solution of the infinite horizon problemP∞(x0). Ifα= 1, then the feedback law is infinite horizon optimal. This implies that the closer to 1 the positive indexα is, the closer the feedback law approximates the solution ofP∞(x0)while the smallerαis, the larger the suboptimality gap becomes.
Remark 2.1.9. The proof of Theorem 2.1.8 particularly shows the relaxed dynamic programming inequality (2.2) forV =VN andµ=µN,m, i.e., for allx0∈X. This inequality can be seen as a Lyapunov inequality and shows thatVN is anm-step Lyapunov function indicating the descent property of the value function along the closed-loop trajectory at everymtime instants. Refer, e.g., to [36, Section 2.3], [57, Appendix B] or [43, Chapter 4] for discussions on Lyapunov stability theory.
The optimization problem Pα may be nonlinear depending on the nature of Bk(r) from Assumption 2.1.2. However,Pα becomes a linear program inrif Bk(r)is linear. An explicit formula forαcan be derived in this case.
Theorem 2.1.10. LetBK,K= 2, . . . , N, be linear functions and defineγK :=
BK(r)/r. Then the optimal valueαof problemPαfor given optimization horizon N, control horizonm satisfies satisfiesα= 1if and only if γm+1≤1 and then equality holds in (2.12).
Proof. See Theorem 5.4 and Remark 5.5 of [37].