G(K, L) =− N +
d
X
i=1
(Di)0KDi
!−1
KB+
d
X
i=1
(Ci)0KDi+
d
X
i=1
LiDi
!0 ,
where, of course, we have to make sure that
N +Pd
i=1(Di)0KDi−1
exists.
Our aim is to carry over this method to the non-quadratic case. To this end, we should know a bit more about how, for example ¯y, depends on h (looking at the FBSDE (3.11)-(3.14), a linear dependence such as in the quadratic case can not be expected). In the next section we collect some properties of the solution of the FBSDE that can be derived quite immediately.
3.3 Basic properties of the solution
Consider a problemP(τ, h) and assume, that ¯xis an optimal state for this problem. Assume that ¯x happens to attain zero. Intuitively, it is clear that ¯u vanishes after this time and that ¯x stays at 0. We want to make this precise and therefore introduce the following:
Definition 3.7 (stopping time τ0) Suppose that we are given a problem P(τ, h) and an optimal state x¯ for this problem. We define τ0 as the first time that x¯ attains 0, i.e. τ0 is the stopping time
τ0 := inf{t∈[0, T] : ¯x(t∨τ) = 0} ∧T.
Then, given one of our standard assumptions, we have the following Lemma.
Lemma 3.8 Consider problem P(τ, h) and let one of the Assumptions A1-A3 hold. Let (¯x,u,¯ y,¯ z)¯ be the corresponding solution of the FBSDE (3.11) - (3.14). Then (¯x,u,¯ y,¯ z)¯ vanishes after τ0. More precisely
(¯x,u,¯ y,¯ z) =¯ 1[τ,τ0](¯x,u,¯ y,¯ z).¯
Proof: Set (¯x0,u¯0,y¯0,z¯0) =1[τ,τ0](¯x,u,¯ y,¯ z). From the definition of¯ τ0 we get ¯x(· ∧τ0) = 1[τ,τ0]x. This yields that ¯¯ x0 is the solution of the state equation (1.14) foru= ¯u0. We have
|¯u0| ≤ |¯u|, |¯x0| ≤ |¯x|, and since our assumptions demandQ, N and M to be non-negative, we get J(¯u0)≤J(¯u). As the optimal control is unique, it follows that
¯
u0 = ¯u, x¯0 = ¯x.
We will now show that (¯x0,u¯0,y¯0,z¯0) solves the FBSDE (3.11) - (3.14). The initial and terminal values (3.13), as well as the auxiliary condition (3.14) are easily checked. Let us consider the adjoint equation (3.12). Let Γ be the solution of the SDE dΓ = AΓdt+
ΓPd
i=1Cidwi, Γ(τ) = 1. For a stopping time γ with τ ≤ γ ≤ T we then have the representation (compare [EM:BSDE] Prop 1.2.4.)
¯ the unique solution of the BSDE (3.12) corresponding to the coefficients of the problem P(τ, h). Let γ ≥τ again be a stopping time. Itˆo’s formula, applied to Γ¯y0, then yields
3.3. BASIC PROPERTIES OF THE SOLUTION 31
Yet, this lemma is redundant to the extent that ¯xnever reaches 0, except on the set{h= 0}
of the zeros of the initial value. This is due to the fact that the optimal state will turn out to be a stochastic exponential. However, the previous lemma will be useful in proving this exponential representation for the optimal state.
The next lemma relates the initial value of ¯y to the optimal cost J(¯u).
Lemma 3.9 If u¯ is optimal for problem P(τ, h), then J(¯u) = 1
qE[¯y(τ)h],
where y¯is the adjoint process from the corresponding solution of (3.11) - (3.14).
Proof: Let ¯x be an optimal state for problem P(τ, h). Note that Qϕ(¯x)¯x = Q|¯x|q, M ϕ(¯x(T))¯x(T) =M|¯x(T)|q. Applying Itˆo’s formula to ¯y¯x and integrating yields
E[¯y(T)¯x(T)−y(τ¯ )¯x(τ)] = E[M|¯x(T)|q−y(τ¯ )h]
= E[
Z T τ
−Q|¯x|q+ ¯u0 B0y+
d
X
i=1
(Di)0zi
! ds]
= E[
Z T τ
−Q|¯x|q−u¯0N ϕ(¯u)ds]
= E[
Z T τ
−Q|¯x|q−N|u|¯q],
where in the third line we used the auxiliary condition (3.14). Rearranging the last equation (as an equality of the right hand side of the first and the last line) yields the assertion.
The next proposition may be viewed as one of the central observations of this work. Almost all the statements of this section are similar in the quadratic case q = 2. Concerning the dependence of ¯y on h, there’s a difference. In the quadratic case, h 7→ y¯ is linear and bounded. It turns out that in the “general” case, h 7→ f(¯y) still is linear and bounded.
It may not be seen why this is of particular importance. This will become clear in the next chapter. For the moment, we will give a very rough outline of the opportunities this linearity offers: the mapping h7→f(¯y(τ)) will be continuous, too, and this will imply the existence of a random variableK(τ) such thatf(¯y(τ)) =K(τ)h=K(τ)¯x(τ). Varying over the initial time will yield a family of random variables (indeed, a process) (K(t∨τ))t≤T, that allows us to represent the optimal state in terms of the adjoint process and vice-versa by
f(¯y(t∨τ)) = K(t∨τ)¯x(t∨τ), t≤T.
From this representation it is clear thatK possesses a differential at least on the stochastic interval [τ, τ0) (where τ0 is the first vanishing time for ¯x). Further,K is part of the solution of a BSDE, namely the Riccati-equation. This equation will only depend on the coefficients
of the problem P(τ, h), i.e. it does not involve ¯x,u,¯ y¯ or ¯z - but its solution (K, L) will enable us to explicitly construct ¯x and ¯u.
The differential of K will be found simply by differentiating f(¯x¯y) (of course we must take care about the zeros of ¯x- this is a crucial point). The differential of this quotient may be considered independently and without any knowledge that the quotient represents some particular processK. But it will actually be of great importance for us that we can derive some properties of K (essentially concerning boundedness and positivity) that could not be proved easily for f(¯x¯y).
For convenience let us introduce the following notation.
Notation 3.10 (solution of the FBSDE depending onτ andh) For a stopping timeτ < T and an h∈LqF
τ(R) we denote by
(¯xτ,h,u¯τ,h,y¯τ,h,z¯τ,h)
the solution of (3.11)-(3.14) corresponding to problem P(τ, h), given that this FBSDE has a unique solution.
With this notation, we have
Proposition 3.11 Let one of the Assumptions A1-A3 hold. Then, for a given stopping time τ < T and a given h0 ∈LqFτ(R) we have for all b ∈L∞Fτ(R)
(¯xτ,bh0,u¯τ,bh0, f(¯yτ,bh0)) =b(¯xτ,h0,u¯τ,h0, f(¯yτ,h0)).
In particular, the mapping
LqFτ(R) −→ LqF(Ω, C([τ, T];R))× U ×LqF(Ω, C([τ, T];R)), h 7→ (¯xτ,h,u¯τ,h, f(¯yτ,h)),
is linear.
Proof: We start with the first assertion. Set (¯x,u,¯ y,¯ z) := (¯¯ xτ,h0,u¯τ,h0,y¯τ,h0,z¯τ,h0) and for b ∈ L∞F
τ(R) define (x0, u0, f(y0)) := b(¯x,u, f(¯¯ y)), in particular y0 = ϕ(b)¯y. Additionally, set z0 = ϕ(b)¯z. We will check that (x0, u0, y0, z0) satisfies (3.11)-(3.14). Note that these processes inherit their integrability from ¯x, etc., due to the essential boundedness of b.
We use the fact that (stochastic and deterministic) integration, starting withτ, commutes with multiplication with anFτ-measurable random variable. Thus, from (3.11) we get for all stopping times γ ≥τ
bx(γ) =¯ bx(τ¯ ) +b Z γ
τ
A¯x+Buds¯ +
d
X
i=1
b Z γ
τ
Cix¯+Diu¯ dwi
= bx(τ¯ ) + Z γ
τ
A(bx) +¯ B(bu)ds¯ +
d
X
i=1
Z γ τ
Ci(b¯x) +Di(bu)¯ dwi,
3.3. BASIC PROPERTIES OF THE SOLUTION 33 and
ϕ(b)¯y(γ) = ϕ(b)¯y(τ) +ϕ(b) Z γ
τ
A¯y−
d
X
i=1
Ciz¯−Qϕ(¯x)ds+
d
X
i=1
ϕ(b) Z γ
τ
¯ zidwi
= ϕ(b)¯y(τ) + Z γ
τ
A(ϕ(b)¯y)−
d
X
i=1
Ci(ϕ(b)¯z)−Qϕ(b)ϕ(¯x)ds
+
d
X
i=1
Z γ τ
(ϕ(b)¯zi)dwi.
Since ϕ(b)ϕ(¯x) =ϕ(bx), this implies that (x¯ 0, u0, y0, z0) satisfies the differential equations (3.11) and (3.12). We have y0(T) = ϕ(b)¯y(T) = ϕ(b)M ϕ(¯x) = M ϕ(bx) =¯ M ϕ(x0), hence the terminal condition in (3.13) holds, and the initial condition is obvious. Using the compatibility of ϕwith multiplication again shows that the auxiliary condition (3.14) also holds for u0, y0 and z0. Hence, the first assertion of the proposition is proved.
For the assertion on linearity choose h1, h2 ∈LqFτ(R) and set H = 1 +|h1|+|h2| ∈LqFτ(R).
Define bi := hHi, i = 1,2. The bi are essentially bounded, Fτ-measurable, and we have h1+h2 = (b1+b2)H =b1H+b2H. Thus,
(¯xτ,h1+h2,u¯τ,h1+h2, f(¯yτ,h1+h2))
= (¯xτ,(b1+b2)H,u¯τ,(b1+b2)H, f(¯yτ,(b1+b2)H))
= (b1+b2)(¯xτ,H,u¯τ,H, f(¯yτ,H))
= b1(¯xτ,H,u¯τ,H, f(¯yτ,H)) +b2(¯xτ,H,u¯τ,H, f(¯yτ,H))
= (¯xτ,b1H,u¯τ,b1H, f(¯yτ,b1H)) + (¯xτ,b2H,u¯τ,b2H, f(¯yτ,b2H))
= (¯xτ,h1,u¯τ,h1, f(¯yτ,h1)) + (¯xτ,h2,u¯τ,h2, f(¯yτ,h2)).
Finally, forα∈R,α(¯xτ,h,u¯τ,h, f(¯yτ,h)) = (¯xτ,αh,u¯τ,αh, f(¯yτ,αh)) follows from the first
asser-tion, and linearity is shown.
In the above proposition we did not explicitly mention what happens to z if h varies.
However, from the proof it is clear, that ¯zτ,bh0 =ϕ(b)¯zτ,h0 (where we used the notation of the proof). Yet, for q < 2 the process f(¯zτ,h) does not in general belong to Hq(τ, T;Rm), and a statement about ¯zτ,h may be misleading in this respect. Nevertheless, it is correct thath7→f(¯zτ,h) is linear as a mapping fromLqF
τ(R) to the linear space of alld-dimensional adapted processes.
In the next lemma, the linearity eases the proof of continuous dependence of (¯xτ,h,u¯τ,h) onh.
Lemma 3.12 Under each of the the Assumptions A1-A3, the mapping LqF
τ(R) −→ LqF(Ω, C([τ, T];R))× U ×LqF0(Ω, C([τ, T];R))×Hq0(τ, T;Rd), h 7→ (¯xτ,h,u¯τ,h,y¯τ,h,z¯τ,h),
is continuous. The optimal cost depends continuously on the initial value, i.e. LqF
τ(R)−→
R, h7→J(¯uτ,h), is continuous.
Proof: First, let us show that
h7→(¯xτ,h,u¯τ,h)
is continuous. By linearity, it suffices to show continuity in h= 0. Let (hn)nbe a sequence in LqF
τ(R) with hn →0, n → ∞. Set (¯xn,u¯n) := (¯xτ,hn,u¯τ,hn). Plugging the control u= 0 (with initial value hn) in J yields J(¯uτ,hn) ≤ J(0) → 0, n → ∞, by Corollary 2.4. The next step depends on the assumption that is in force.
If Assumption A2 holds, the process N is uniformly positive, hence J(¯un)→0 implies k¯unkLq
F →0, n→ ∞. From Corollary 2.4 it follows that ¯xn→0, n→ ∞.
If Assumption A1 or A3 holds, the r.v. M is uniformly positive, hence fromJ(¯un)→0 we get E[|¯xn(T)|q]→0, n→ ∞. We write the SDE (3.11) again as in (3.2) as
dx={Ax+θ0σ0u}ds+{xC+u0σ}dw and consider it as a BSDE as in (3.3), (3.4),
dRn = {(A−θC)Rn+θ0Sn0}ds+Sndw, Rn(T) = x¯n(T),
with Rn = ¯xn and Sn = ¯xnC + ¯unσ. From [EPQ:BSDE] it follows that (Rn, Sn) → 0, n → ∞, in LqF(Ω, C([τ, T];R))×Hq(τ, T;Rd). The uniform positivity of σσ0 then gives, that ¯un→0 in Hq(τ, T;Rm),n → ∞.
So, if one of the Assumptions A1-A3 holds, the mapping h7→(¯xτ,h,u¯τ,h) is continuous.
We will now have to show that h 7→ (¯yτ,h,z¯τ,h) is also continuous. Let (hn)n be a se-quence that converges inLqF
τ(R) tohand set (¯xn,u¯n,y¯n,z¯n) = (¯xτ,hn,u¯τ,hn,y¯τ,hn,z¯τ,hn) and (¯x,z,¯ y,¯ z) =: (¯¯ xτ,h,u¯τ,h,y¯τ,h,z¯τ,h). We already know that ¯xn → x¯ in LqF(Ω, C([τ, T];R)), hence Qϕ(¯xn)→Qϕ(¯x) inLqF0(Ω, C([τ, T];R)), and M ϕ(¯xn(T))→M ϕ(¯x(T)) in LqF0
T(R), n → ∞. It follows from [EPQ:BSDE], Prop. 5.1 (or the Closed Graph Theorem), that (¯yn,z¯n) → (¯y,z) in¯ LqF0(Ω, C([τ, T];R))×Hq0(τ, T;Rd), n → ∞. The continuity of h7→J(¯uτ,h) is clear from the continuity of h7→(¯xτ,h,u¯τ,h), and the lemma is proved.
Given a fixed initial time τ, it turns out that we need not be overly concerned about the initial value h if we are only interested in ¯xτ,h and ¯uτ,h. To some extent, it suffices to considerh= 1.
Lemma 3.13 Let one of the Assumptions A1-A3 hold. Then, for h ∈LqFτ(R) we have (¯xτ,h,u¯τ,h) = h(¯xτ,1,u¯τ,1).
Further, yτ,h =ϕ(h)yτ,1.
3.3. BASIC PROPERTIES OF THE SOLUTION 35 Proof: Forh∈LqF
τ(R), choose a sequence (hn) withhn ∈L∞Fτ(R), such that|h−hn|Lq → 0, n→ ∞. From Lemma 3.12 we have that
(¯xτ,hn,u¯τ,hn)→(¯xτ,h,u¯τ,h),
n → ∞, in LqF(Ω, C([τ, T];R)) × U; from Proposition 3.11 we get (¯xτ,hn,u¯τ,hn) = hn(¯xτ,1,u¯τ,1). There is a subsequence (nk)k such thatP −a.s. hnk →h, k → ∞, hence we haveP −a.s.
hnkx¯τ,1(t∨τ) = ¯xτ,hnk(t∨τ)→h¯xτ,1(t∨τ), for all t ∈[0, T] and
hnku¯τ,1 = ¯uτ,hnk →hu¯τ,1,
Leb⊗P−a.s.,k → ∞. Besides, the norm-convergence (¯xτ,hnk,τ,hnk)→(¯xτ,h,u¯τ,h),k → ∞, in LqF(Ω, C([τ, T];R)) × U implies the existence of a sub-sub-sequence (nkj)j such that P −a.s.
¯
xτ,hnkj(t∨τ)→x¯τ,h(t∨τ),
for all t ∈[0, T] and ¯uτ,hnkj →u¯τ,h, Leb⊗P −a.s., j → ∞. As the limits are unique, this now implies that P −a.s.
¯
xτ,h(t∨τ) =h¯xτ,1(t∨τ), for all t ∈[0, T] and
¯
uτ,h =hu¯τ,1
Leb⊗P −a.s., which proves the first assertion of the lemma.
Now recall the representation (3.15) for the adjoint process from Lemma 3.8. Usingxτ,h = hxτ,1, this representation yields for every stopping time γ with τ ≤γ ≤T
¯ yτ,h(γ)
= Γ(γ)−1E[Γ(T)M ϕ(¯xτ,h(T)) + Z T
γ
Γ(s)Q(s)ϕ(¯xτ,h(s))ds|Fγ]
= Γ(γ)−1E[Γ(T)M ϕ(h)ϕ(¯xτ,1(T)) + Z T
γ
Γ(s)Q(s)ϕ(h)ϕ(¯xτ,1(s))ds|Fγ]
= ϕ(h)Γ(γ)−1E[Γ(T)M ϕ(¯xτ,1(T)) + Z T
γ
Γ(s)Q(s)ϕ(¯xτ,1(s))ds|Fγ].
In the fourth line we used the fact that h is Fγ−measurable. Reading the representation (3.15) “from the right to the left” now shows that
¯
yτ,1(γ) = Γ(γ)−1E[Γ(T)M ϕ(¯xτ,1(T)) + Z T
γ
ΓQϕ(¯xτ,1)ds|Fγ],
hence yτ,h =ϕ(h)¯yτ,1, as required.
Remarks on Chapter 3 This chapter was very much in the spirit of [B:LQOC]. There,
in the quadratic case q = 2, the solution of problem P(τ, h) is also characterized in terms of a FBSDE. Bismut himself states that his method is taken from the theory of controlled partial differential equations, especially from [L:OCSG], Chapter 4. When the above cited article of Bismut was published, there was not much known about BSDEs, not even linear BSDEs; hence Bismut had to develop the necessary tools on his own - with the goal of deriving another BSDE that is very much harder to treat than linear BSDEs.
Nowadays, there’s a vast literature on BSDEs, and the situation of “Lipschitz-drivers”
is covered very thoroughly. In this chapter, one of the benefits we extracted from this
“ready-to-use” theory is the assertion that Assumptions A1, A3 are sufficient to guarantee the existence of an optimal control. This type of condition (i.e. uniform positivity of σσ0 andM) is well known in quadratic theory, see for example [CLZ: SLQ]. In this article, one also finds examples where the quadratic problem is well posed even if the control weighting process is indefinite. In [KT:GAS], the global solvability of the Riccati equation (1.7), (1.8) is shown to be solvable under Assumption A1 for q= 2.
The statement that the solution of a problem P(τ, h) can be characterized in terms of the FBSDE (3.11) -(3.14) is not surprising. It may be more interesting to note that the isoelastic cost functions allow for an explicit representation of the optimal cost in terms of the adjoint process, as stated in Lemma 3.9. Though this observation is immediate, it seems to be new. Also, the assertion on the linear and continuous dependence h 7→
(xτ,h, uτ,h, f(yτ,h)) seems to be new. The proof of this fact relies heavily on the special form of the cost functions. As the reader may notice, the last two results mentioned were not really hard to produce, although, they will turn out to be crucial (especially the linearity of h 7→ f(yτ,h)) in our attempt to find and solve a Riccati-equation for problem P(τ, h).
Chapter 4
The Riccati equation
In the introduction, we announced that we wanted to solve problems of type P(τ, h) with the help of Backward Stochastic Riccati Differential Equations. But so far, we have not yet defined this equation (except in the Introduction). The reason is that we need some implicitly defined functionGfor establishing the BSRDE. We will investigate this function G in the third section of this chapter.
In the first section we will make an initial attempt to decouple the FBSDE (3.11)-(3.14).
This is achieved by introducing a process (in the first instance rather a family of random variables)K that allows us to represent the adjoint process ¯y in terms of the optimal state
¯
x. Actually, we will have f(¯y) = Kx. This¯ K will be the first part of the solution (K, L) of the Riccati-equation. In order to derive the BSRDE, we will consider the differential of
f(¯y)
¯
x in Section 4.2.
It will turn out that we can get quite strong a-priori knowledge about the boundedness of K. In Section 4.4 we will investigate what we can deduce about the second componentLof a solution (K, L) of the Riccati-equation - given that first component K actually satisfies the mentioned boundedness condition.
4.1 A feedback representation for the adjoint process
One of our principal aims is to “decouple” the FBSDE (3.11)- (3.14). Basically, this means that we want to specify a stochastic process that follows some (backward) stochastic differential equation and that allows us to construct the optimal control ¯u. The specification of this decoupling process should be “exogenous” in the sense that it does not make use of the optimal state or the optimal control - the quantities we are actually looking for. Thus, the specification, i.e. the stochastic differential equation, is expected to involve only the coefficients of the problem P(τ, h), but not ¯x or ¯u. The next proposition is a first step in this direction, although there is no concern about the optimal control. The proposition introduces the family of r.v. K that allows the construction of ¯y(the adjoint process) from
¯
x (the optimal state) by f(¯y) = Kx. For non-vanishing ¯¯ x this of course yields K = f(¯x¯y), 37
and differentiating this fraction will make us suspect (in Section 4.2) thatK is a candidate for an “exogenously definable object” that is strongly related to the solution of problem P(τ, h).
The techniques of the following proposition are those from [B:LQOC], Proposition 4.2.
There, in the quadratic case (q = 2) we have ¯y = Kx. The key insight that allows us to¯ make use of these techniques in the isoelastic case is the linear and continuous dependence of f(¯yτ,h) on h.
Proposition 4.1 Let one of the Assumptions A1-A3 hold and let τ be a stopping time with 0≤τ < T. Then, for every t ∈[0, T] there is a P −a.s.-unique K(t∨τ)∈L∞Ft∨τ(R) such that
f(¯yτ,h(t∨τ)) =K(t∨τ) ¯xτ,h(t∨τ) (4.1) for every h ∈LqFτ(R).
Proof: We start by considering t = 0, i.e. we are looking for a K(τ) such that f(¯yτ,h(τ)) = K(τ)¯xτ,h(τ) = K(τ)h. From Proposition 3.11, we know that the appli-cation LqFτ(R) −→ LqFτ(R), h 7→ f(¯yτ,h(τ)) is linear; from Lemma 3.12 we see that it is also continuous. Let A, B be two disjoint sets in Fτ. As 1A∪B = 1A +1B, linear-ity entails that f(¯yτ,1A∪B)(τ) = f(¯yτ,1A)(τ) +f(¯yτ,1B)(τ), hence the mapping Fτ −→ R, A7→E[f(¯yτ,1A(τ))] is a signed measure. It is clear that sets withP(A) = 0 are mapped to 0. By the Radon-Nikodym-Theorem, there is aP−a.s.-unique, integrable,Fτ-measurable r.v. K(τ) such that E[f(¯yτ,1A(τ))] =E[1AK(τ)], compare [H:MT], Sect. 31, Thm B. As f(¯yτ,1A) (τ) = 1Af(¯yτ,1) (τ), see Proposition 3.11, we get E[1A(f(¯yτ,1(τ))−K(τ))] = 0 for all A ∈ Fτ. This entails K(τ) = f(¯yτ,1(τ)), P −a.s.. By Proposition 3.11 again, this gives hf(¯yτ,1(τ)) =f y¯τ,h(τ)
=K(τ)h if h=1S, S ∈ Fτ. By linearity, this immediately extends to initial values h that are finitely valued, simple r.v., i.e. to the initial values h=Pk
j=1αj1Sj, where theSj are disjoint,Fτ-measurable sets, and αj ∈R. As h7→f(¯yτ,h) is linear and continuous, there is a k > 0 such that
f(¯yτ,h(τ))
q
Lq ≤k|h|qLq
for all h ∈ LqFτ(R). Consequently, for all simple, finitely valued and Fτ-measurable h we have E[|f(¯yτ,h(τ))|q] = E[|K(τ)|q|h|q] ≤ kE[|h|q]; this entails that K(τ) is essentially bounded. Hence, the mapping LqF
τ(R)−→LqF
τ(R), h7→K(τ)h is well defined, linear and continuous. It coincides withh7→f(¯yτ,h(τ)) for simple r.vs. h, and as the simple r.vs. are dense inLqFτ(R), it follows that f(¯yτ,h(τ)) =K(τ)h for all h∈LqFτ(R).
Now recall Definition 1.3 and consider the family of problemsP(t∨τ, ht∨τ),t∈[0, T), whose coefficients are those of problem P(τ, h), restricted to the subinterval [t∨τ, T], i.e the co-efficients ofP(t∨τ, ht∨τ) areA|[t∨τ,T], B|[t∨τ,T], . . . , N|[t∨τ,T]and M. If Assumption A1, A2 or A3 holds for problem P(τ, h), this is also true for problemP(t∨τ, ht∨τ). With the same procedure as above, we can now construct a P −a.s.-unique, essentially bounded, Ft∨τ -measurable r.v. K(t∨τ) such thatf(¯yt∨τ,ht∨τ(t∨τ)) =K(t∨τ)ht∨τ =K(t∨τ)¯xt∨τ,ht∨τ(t∨τ) for all ht∨τ ∈LqFt∨τ(R).
In this way we have constructed a family of r.v. K(t∨τ), t ∈ [0, T), to which we add K(T) :=f(M); let us show that this family indeed satisfies (4.1). For t = T this is clear from the terminal condition on ¯yτ,h. Now fix some arbitrary h ∈LqF
τ(R) and consider, as
4.1. A FEEDBACK REPRESENTATION FOR THE ADJOINT PROCESS 39 above, for t ∈[0, T) the problem P(t∨τ,x¯τ,h(t∨τ)), where we take the optimal state of problem P(τ, h) at time t∨τ as the initial value of a new problem with initial time t∨τ. Set ht∨τ := ¯xτ,h(t ∨τ). From the uniqueness of the optimal control it is clear that the solution of these problems, arising from a “premature halt” of ¯xτ,h, are the restriction of
¯
xτ,h, ¯uτ,h to [t∨τ, T], ¯xt∨τ,ht∨τ = ¯xτ,h|[t∨τ,T], ¯ut∨τ,ht∨τ = ¯uτ,h|[t∨τ,T]. Clearly, ¯yτ,h|[t∨τ,T] then satisfies (3.12)-(3.14) of the FBSDE for problem P(t∨τ, ht∨τ), thus ¯yt∨τ,ht∨τ = ¯yτ,h|[t∨τ,T]. Hence, from the construction of K(· ∨τ) we get for all t∈[0, T], that
f(¯yτ,h(t∨τ)) = f(¯yt∨τ,ht∨τ(t∨τ)) =K(t∨τ)ht∨τ,
which is the desired representation, and the proposition is shown.
The independence of K on h reflects what we have seen in Lemma 3.13, i.e. the fact that ¯xτ,h = hx¯τ,1 and f(¯yτ,h) = hf(¯yτ,1). So far, we do not know if the “family of r.vs.”
K(t∨τ), t ∈ [0, T], is actually a stochastic process, i.e. we do not know if there is any measurability with respect to t. However, if τ0 := inf{t ∈ [τ, T] : ¯xτ,1(t) = 0} ∧T, then we have K(t∨τ) = f(¯x¯yτ,1τ,1(t∨τ)(t∨τ)) on the stochastic interval [τ, τ0), and due to the continuity of paths of ¯xτ,1,y¯τ,1, at least the restriction K : [τ, τ0) −→ R is a continuous stochastic process (note that τ < τ0).
Next to this t-measurability/continuity of the restricted family K, there are two further properties ofK that will help us (and we are going to show next): its uniform boundedness and its positivity. To get the latter in the form we will need, unfortunately we will have to strengthen Assumption 2. The boundedness property is readily available:
Lemma 4.2 Let one of the Assumptions A1, A2 or A3 hold. Then the familyK(t∨τ), t ∈ [0, T], of Proposition 4.1 is uniformly essentially bounded, i.e. there is some k > 0 such that
sup{|K(t∨τ)|L∞ : t∈[0, T]} ≤k.
Proof: Recall the representation of the optimal cost from Lemma 3.9, J(¯uτ,h) =
1
qE[¯yτ,h(τ)h]. Applying this to the problems P(t ∨τ, ht∨τ), using Proposition 4.1 and the optimality of ¯ut∨τ,ht∨τ, yields for all t∈[0, T]
Jt∨τ(¯ut∨τ,ht∨τ) = 1
qE[¯yt∨τ,ht∨τ(t∨τ)ht∨τ]
= 1
qE[ϕ(K(t∨τ)ht∨τ)ht∨τ]
= 1
qE[ϕ(K(t∨τ))|ht∨τ|q]
≤ Jt∨τ(0),
whereJt∨τ denotes the cost functional for initial timet∨τ. Denote by ˜x0,ht∨τt∨τ the solution of the state equation for problem P(t∨τ, ht∨τ) (i.e. equation (1.14), (1.15) with initial time t∨τ instead of τ) with u = 0 . By Corollary 2.4, there is a k1 > 0, independent of
t and ht∨τ, such that x˜0,ht∨τt∨τ
q Lqc
≤ k1|ht∨τ|qLq. From the essential boundedness of Q and M there is a k2 independent of t and h such that Jt∨τ(0) ≤ k2
x˜0,ht∨τt∨τ
q Lqc
. Putting this together, we get for allt ∈[0, T] and all ht∨τ ∈LqF
t∨τ(R) 1
qE[ϕ(K(t∨τ))|ht∨τ|q] ≤ Jt∨τ(0)
≤ k2 x˜0,ht∨τt∨τ
q Lqc
≤ k2k1E[|ht∨τ|q],
withk1, k2 independent oftandht∨τ. This entails the uniform boundedness ofϕ(K(· ∨τ)),
hence that of K(· ∨τ), and the lemma is proved.
In the previous lemma, our main argument relied on the relationship between the optimal cost and the adjoint process (stated in Lemma 3.9) for the problems P(τ, h). We want to exploit this connection again in order to prove that K is strictly positive, respectively uniformly positive (depending on the assumption in force). This positivity, respectively uniform positivity, is a technical requirement for the following. Unfortunately, under As-sumption A2 the aforementioned link between the optimal cost and K shows us thatK(τ) cannot always be strictly positive. Consider problemP(τ, h) with, say, N ≡1,Q= 0 and M = 0. This matches the requirements of Assumption A2. It is clear that the optimal control is identically zero, ¯uτ,h = 0, since the only contribution to the cost functional is the immediate control cost. For all h∈LqF
τ(R), the optimal cost is zero, thus, by Lemma 3.9 0 = J(¯uτ,h) = 1
qE[¯yτ,h(τ)h] = 1
qE[ϕ(K(τ))|h|q], for all h ∈ LqF
τ(R), hence K(τ) = 0. With the same argument applied to the problems P(t∨τ, ht∨τ), it follows that K ≡0.
This example is, of course, pathological, but it is not excluded by Assumption A2. However, the reasoning shows that strict positivity ofK could be expected if the following condition holds: For all t∈[0, T] and all ht∨τ ∈LqFt∨τ(R) we have
Jt∨τ(¯ut∨τ,ht∨τ)>0 ifht∨τ 6= 0, (4.2)
where Jt∨τ is the cost functional and ¯ut∨τ,ht∨τ is the optimal control for problem P(t ∨ τ, ht∨τ). The following Assumption is stronger than A2 and will imply (4.2).
Assumption A4 In addition to Assumption A2 we have M >0, P −a.s..
This yields what we desire.
Remark 4.3 Under Assumption A4, (4.2) holds.
4.1. A FEEDBACK REPRESENTATION FOR THE ADJOINT PROCESS 41 Proof: Suppose that there is a ht∨τ ∈ LqFt∨τ(R) such that J(¯ut∨τ,ht∨τ) = 0. We have to show that ht∨τ = 0. From the uniform positivity of N it follows that ¯ut∨τ,ht∨τ = 0, thus ¯xt∨τ,ht∨τ satisfies the SDE dx = Axds+Pd
i=1Cixdwi, x(t ∨τ) = ht∨τ. Therefore
¯
xt∨τ,ht∨τ(T) = ht∨τexp{RT
t∨τA−12Pd
i=1|Ci|2ds+Pd i=1
RT
t∨τCidwi}. SinceJ(¯ut∨τ,ht∨τ) = 0, the expectation E[M|¯xt∨τ,ht∨τ(T)|q] equals zero. The strict positivity of M and of the exponential part of ¯xt∨τ,ht∨τ now implies ht∨τ = 0, P −a.s., and since J(u) ≥ 0 for all
u∈ U, the remark is proved.
In the case of Assumption A1 and A3, the cost functional is even uniformly positive.
Remark 4.4 If Assumptions A1 or A3 hold, there is a δ > 0 such that for all t ∈ [0, t]
and all ht∨τ ∈LqFt∨τ(R)
J(¯ut∨τ,ht∨τ)≥δ|ht∨τ|qLq.
Proof: From the representation of (1.14) as a BSDE as in (3.3) and (3.5), it fol-lows with Prop. 5.1. in [EPQ:BSDE] that there is a k > 0 such that
¯xt∨τ,ht∨τ LqF + Cx¯t∨τ,ht∨τ + (¯ut∨τ,ht∨τ)0σ
Hq ≤ k
x¯t∨τ,ht∨τ(T)
Lq. The constant k can be chosen indepen-dently of t and ht∨τ; it depends on the the Lipschitz-constant of the “driver” of the BSDE and ofT. Since ¯xt∨τ,ht∨τ(t∨τ) = ht∨τ, this yields, usingM for some >0 in the third line,
E[|ht∨τ|q] ≤
¯xt∨τ,ht∨τ
q Lqc
≤ kq
¯xt∨τ,ht∨τ
q Lq
≤ kq1
E[M|¯xτ,h(T)|q]
≤ kq1
J(¯ut∨τ,ht∨τ)
for all t ∈[0, T] and all ht∨τ ∈LqFt∨τ(R). This proves the remark.
These positivity properties of the the cost functional carry over to K.
Lemma 4.5 Let one of the Assumptions A1, A3 or A4 hold. Consider the family of r.v.
K(t∨τ), t∈[0, T], from Proposition 4.1.
a) If Assumption A1 or A3 holds, then there is a δ0 >0 such that K(t∨τ)≥δ0, P −a.s. for all t ∈[0, T].
b) If Assumption A4 holds, then K(t∨τ)>0, P −a.s. for all t ∈[0, T].
Proof: Let t ∈ [0, T) be arbitrary. Applying Lemma 3.9 to problem P(t∨τ, ht∨τ) for some ht∨τ ∈LqFt∨τ(R) yields
J(¯ut∨τ,ht∨τ) = 1
qE[¯yt∨τ,ht∨τ(t∨τ)ht∨τ] = 1
qE[ϕ(K(t∨τ))|ht∨τ|q].
Hence, by the previous remarks, we have for all ht∨τ ∈LqFt∨τ(R) that 1
qE[ϕ(K(t∨τ))|ht∨τ|q]>0 if Assumption A4 holds, and that
1
qE[ϕ(K(t∨τ))|ht∨τ|q]> δE[|ht∨τ|q]
(δ independent of t and ht∨τ) if Assumption A1 or A3 holds. This implies the assertions of the lemma for t < T. Noting K(T) =f(M) completes the proof of the lemma.
So far in this section, we have often considered problems P(t∨τ, ht∨τ). In the sequel, we will focus again on the initial timeτ. In view of Lemma 3.13 we will also restrict ourselves to the initial value h= 1. Recall Definition 3.7 and let us fix the following notation.
Notation 4.6 Let τ be a stopping time with τ < T and let one of the Assumptions A1, A3 or A4 hold. Until further notice, we will use the notation
x:= ¯xτ,1, u:= ¯uτ,1, y := ¯yτ,1, z := ¯zτ,1.
The definition of the stopping time τ0 introduced in Definition 3.7 then applies to the optimal state process x,
τ0 := inf{s∈[τ, T] :x(s) = 0} ∧T.
K will denote the family K(t∨τ), t∈[0, T], from Proposition 4.1.
Note that, due to the continuity of x, we have τ < τ0 P −a.s..
The properties of K we encountered in this section rely on the connection between K and the optimal cost of problem P(τ, h) (via the adjoint process y). The next step is to investigate the differential of K. This will ultimately lead to to the Riccati-equation for the linear isoelastic problem.