Discretisation of Stochastic Control Problems for Continuous Time Dynamics with Delay

SFB 649 Discussion Paper 2005-038

Discretisation of Stochastic Control

Problems for Continuous Time Dynamics with

Delay

Markus Fischer*

Markus Reiss*

* Weierstraß-Institut für Angewandte Analysis und Stochastik (WIAS), Germany

This research was supported by the Deutsche

Forschungsgemeinschaft through the SFB 649 "Economic Risk".

http://sfb649.wiwi.hu-berlin.de ISSN 1860-5664

SFB 649, Humboldt-Universität zu Berlin

S FB 6 4 9 E C O N O M I C R I S K B E R L I N

time dynamis with delay

∗

Markus Fisher

Weierstraÿ-Institutfür Angewandte

Analysis und Stohastik (WIAS)

Mohrenstr. 39

10117 Berlin

Germany

Markus Reiÿ

Weierstraÿ-Institutfür Angewandte

Analysisund Stohastik (WIAS)

Mohrenstr. 39

10117Berlin

Germany

August 2,2005

Abstrat

Asamain stepin thenumerialsolutionofontrol problemsin ontinuoustime,

theontrolledproessisapproximatedbysequenesofontrolledMarkovhains,thus

disretizingtimeandspae. Anewfeaturein thisontextistoallowfordelayin the

dynamis. The existene of an optimalstrategy with respet to theost funtional

an beguaranteedin thelassofrelaxedontrols. Weakonvergeneof theapproxi-

matingextended Markovhains to theoriginalproess togetherwithonvergeneof

theassoiatedoptimalstrategiesisestablished.

1 Introdution

Ageneralstrategyforrenderingontrolproblemsinontinuoustimeaessibletonumerial

omputation isthe following: Taking asa startingpoint the original dynamis, onstrut

afamilyofontrol problemsin disretetime with disretestate spaeanddisretizedost

funtional. Standard numerial shemes an be applied to ndan optimal ontrol and to

alulatetheminimalostsfor eahofthedisreteontrolproblems. Theimportantpoint

to establish is then whether the disrete optimal ontrols and minimal osts onverge to

theontinuous-time limitasthemeshsizeofthe disretisationtendstozero. Ifthatisthe

ase,thenthe disreteontrolproblemsareavalidapproximationtothe originalproblem.

Thedynamisofthe ontrolproblemweareinterestedinaredesribedbyastohasti

delaydierentialequation(SDDE).Thus,thefutureevolutionofthedynamismaydepend

not only on the present state, but also on the past evolution. For an exposition of the

general theory of SDDEs see Mohammed (1984) or Mao (1997). The development of

∗

FinanialsupportbytheDFG-Sonderforshungsbereih649EonomiRiskisgratefullyaknowledged.

Hu et al. (2004) and the referenes therein. In Calzolari et al. (2003), segmentwise Euler

shemes are used in a non-linear ltering problem for approximating the state proess,

whih isgiven byan SDDE. Numerialproedures for deterministi ontrol with delayed

dynamis have already been used in appliations, see Bouekkine et al. (2005) for the

analysis of an eonomi growth model. The algorithm proposed there is based on the

disretisationmethodstudied here, but no formalproofof onvergene isgiven.

Also the mathemati al analysis of stohasti ontrol problems with time delay in the

state equation has been the objet of reent works, see e.g. Elsanosi et al. (2000) for

ertain expliitly available solutions, Øksendal and Sulem (2001) for the derivation of a

maximum prinipleandLarssen(2002)forthedynamiprogramming approah. Although

one an invoke the dynami programmin g priniple to derive a Hamilton-Jaobi-Bellman

equationforthevaluefuntion,suhanequationwillingeneralbeanon-lineardierential

equation with innite-dim ensional state spae. A dierent approah to treat stohasti

ontrol problems with delay is based on representing the state equation as an evolution

equationin Hilbertspae, seeBensoussan et al. (1992).

The lass of ontrol problems is speied in Setion 2. In Setion 3 we prove the

existeneofoptimalstrategiesforthoseproblemsinthe lassofrelaxedontrols. Setion4

introduestheapproximatingproessesandprovidesatightnessresult. Finally,inSetion5

the disrete ontrol problems are dened and the onvergene of the minimal osts and

optimalstrategies isshown.

2 The ontrol problem

We onsider the ontrol of a dynamial system given by a one-dimensional stohasti

delay dierential equation (SDDE) driven by a Wiener proess. Both drift and diusion

oeient may depend on the solution's history a ertain amount of time into the past.

Let

r > 0

^{denote the delay length , i.e. the maximal length of dependene on the past.}

For simpliity, we restritattention to the ase, where onlythe driftterm an be diretly

ontrolled.

Typially, the solutionproessof anSDDEdoesnotenjoythe Markovproperty, while

the segment proess assoiated with thatsolution does. For a real-valuedàdlàg funtion

(i.e.,right-ontinuous funtionwith left-handlimits)

ψ

^{livingonthe time interval}

[ − r, ∞ )

the segment attime

t ∈ [0, ∞ )

^{isdened to be the funtion}

ψ _t : [ − r, 0] → R , ψ _t (s) := ψ(t+s).

Thus,thesegmentproess

(X t ) t≥0

^{assoiatedwitha}real-valuedàdlàgproess

(X(t)) t≥−r

takes its values in

D ₀ := D([ − r, 0])

^{, the spae of all} real-valued àdlàg funtions on the interval

[ − r, 0]

^{. There are two natural topologies on}

D ₀

^{. The rst is the one indued by}

the supremum norm, whih we denote by

k . k ∞

^{. The seond is the Skorohod topology of}

and the uniform topology lies in the dierent evaluationof onvergene of funtions with

jumps,whihappearnaturallyasinitialsegmentsanddisretizedproesses. Forontinuous

funtions both topologies oinide. Similar statements hold for

D _∞ := D([ − r, ∞ ))

^and

D ˜ _∞ := D([0, ∞ ))

^{, the spaes of all} real-valued àdlàg funtions on the intervals

[ − r, ∞ )

and

[0, ∞ )

^, respetively. The spaes

D _∞

^and

D ˜ _∞

^{will always be supposed to arry the}

Skorohodtopology, while

D 0

^{will anoniallybeequipped with the uniform topology.}

Let

(Γ, d _Γ )

^{be a ompat metri spae, the spae of ontrol ations. Denote by}

b

the drift oeient of the ontrolled dynamis, and by

σ

^{the diusion oeient. Let}

(W (t)) t≥0

^{be a} one-dimensional standard Wiener proess on a ltered probability spae

(Ω, F , ( F t ) _t≥0 , P)

^{satisfyingtheusualonditions,andlet}

(u(t)) _t≥0

^{beaontrolproess,i.e.}

an

( F t )

^{-adapted measurable proess with values in}

Γ

^{. Consider the ontrolledSDDE}

(1)

dX(t) = b X t , u(t)

dt + σ(X t ) dW (t), t ≥ 0.

Theontrol proess

u(.)

^{together with its stohastibasisinludingthe Wiener proessis}

alled an admissible ontrol if, for every deterministi initial ondition

ϕ ∈ D ₀

^{, equation}

(1)hasa uniquesolution whih isalso weakly unique. Write

U ad

^{for the setof admissible}

ontrolsofequation(1). Thestohastibasisoming withan admissibleontrolwill often

be omittedin the notation.

Asolutioninthesenseusedhereisanadaptedàdlàgproessdenedonthe stohasti

basisoftheontrolproesssuhthattheintegral versionofequation(1)issatised. Given

a ontrol proess together with a standard Wiener proess, a solution to equation (1) is

unique ifit isindistinguishable from anyother solutionalmost surely satisfying the same

initial ondition. Asolution is weakly unique ifit hasthe same lawasanyother solution

with the same initial distribution and satisfying equation (1) for a ontrol proess on

a possibly dierent stohasti basis so that the joint distribution of ontrol and driving

Wienerproessisthe sameforbothsolutions. Letus speifyregularityassumptionsto be

imposedon the oeients

b

^and

σ

^:

(A1) Càdlàg funtionals: the mappings

(ψ, γ) 7→

t 7→ b(ψ _t , γ), t ≥ 0

, ψ 7→

t 7→ σ(ψ _t ), t ≥ 0

dene measurable funtionals

D _∞ × Γ → D ˜ _∞

^and

D _∞ → D ˜ _∞

^, respetively, where

D ∞

^,

D ˜ ∞

^{areequipped with the Borel}

σ

^-algebras.

(A2) Continuityof the drift oeient: there isa ountable subset of

[ − r, 0]

^{, denoted by}

I ev

^{, suh thatfor every}

t ≥ 0

^{the funtion dened by}

D ∞ × Γ ∋ (ψ, γ) 7→ b(ψ t , γ)

is ontinuouson

D ev (t) × Γ

^uniformlyin

γ ∈ Γ

^{, where}

D ev (t) := { ψ ∈ D ∞ | ψ

^{isontinuous at}

t + s

^{for all}

s ∈ I ev } .

(A3) Global boundedness:

| b |

^,

| σ |

^{are bounded byaonstant}

K > 0

^.

(A4) UniformLipshitzondition: Thereisaonstant

K _L > 0

^{suhthatforall}

ϕ, ψ ∈ D ₀

^,

all

γ ∈ Γ

| b(ϕ, γ) − b(ψ, γ) | + | σ(ϕ) − σ(ψ) | ≤ K _L k ϕ − ψ k ∞ .

(A5) Elliptiity of the diusion oeient:

σ(ϕ) ≥ σ ₀

^{for all}

ϕ ∈ D ₀

^{, where}

σ ₀ > 0

^{is a}

positive onstant.

Assumptions(A1)and (A4)onthe oeientsallowus toinvokeTheorem V.7in Protter

(2003:p.253), whihguarantees the existeneofa uniquesolution tothe ontrolledSDDE

(1)for every pieewise onstant ontrol attaining only nitely manydierent values. The

boundednessAssumption(A3)posesnolimitationexeptfortheinitialonditions,beause

the state evolution will be stopped when the state proess leaves a bounded interval.

Assumption(A2) allows us to use segmentwise approximations of the solution proess,

see the proof of Proposition 1. The assumptions imposed on the drift oeient

b

^are

satised,for example,by

b(ϕ, γ) := f ϕ(r 1 ), . . . , ϕ(r n ), Z 0

−r

ϕ(s)w(s) ds

· g(γ),

where

r ₁ , . . . , r n ∈ [ − r, 0]

^arexed,

f

^,

g

^{arebounded ontinuous funtionsand}

f

^{is Lips-}

hitz, and the weight funtion

w

^liesin

L ¹ ([ − r, 0])

^.

Weonsiderontrolproblemsin theweakformulation (f.YongandZhou,1999:p.64).

Given an admissible ontrol

u(.)

^{and a} deterministi initial segment

ϕ ∈ D ₀

^{, denote by}

X ^ϕ,u

^{the unique solution to equation (1). Let}

I

^{be a ompat interval with non-empty}

interior. Denethestopping time

τ _ϕ,u ^{T ¯}

^{ofrstexitfromtheinteriorof}

I

^beforetime

T > ¯ 0

by

(2)

τ _ϕ,u ^{T ¯} := inf { t ≥ 0 | X ^ϕ,u (t) ∈ / int(I) } ∧ T . ¯

In order to dene the osts, we presribe a ost rate

k : R × Γ → [0, ∞ )

^{and a boundary}

ost

g : R → [0, ∞ )

^{, whih are (jointly) ontinuous bounded funtions. Let}

β ≥ 0

^denote

the exponential disount rate. Thendene the ost funtional on

D ₀ × U ad

^by

(3)

J (ϕ, u) := E Z τ

0

exp( − βs) · k X ^ϕ,u (s), u(s)

ds + g X ^ϕ,u (τ )

,

where

τ = τ _ϕ,u ^{T ¯}

^{. Our aimis to minimize}

J (ϕ, .)

^{. We introdue the value funtion}

(4)

V (ϕ) := inf { J (ϕ, u) | u ∈ U ad } , ϕ ∈ D ₀ .

The ontrol problem now onsists in alulating the funtion

V

^{and nding admissible}

ontrols that minimize

J

^{. Suh ontrol proesses are alled optimal ontrols or optimal}

strategies.

In the lass

U ad

^{of admissible ontrols it may happen that there is no optimal ontrol}

(Kushnerand Dupuis, 2001:p.86). A way out isto enlarge the lassof ontrols, allowing

forso-alled relaxedontrols,sothatthe existeneofanoptimal(relaxed)ontrolisguar-

anteed, whiletheinmumofthe ostsoverthe newlassoinides withthe valuefuntion

V

^{asgiven by(4).}

A deterministi relaxed ontrol is a positive measure

ρ

^{on the Borel}

σ

^-algebra

B (Γ × [0, ∞ ))

^{suh that}

(5)

ρ(Γ × [0, t]) = t

^{for all}

t ≥ 0.

For eah

G ∈ B (Γ)

^{, the funtion}

t 7→ ρ(G × [0, t])

^{is absolutely ontinuous with respet}

toLebesgue measureon

[0, ∞ )

^{byvirtueof property(5). Denote by}

ρ(., G) ˙

^anyLebesgue

densityof

ρ(G × [0, .])

^{. Thefamilyofdensities}

ρ(., G) ˙

^,

G ∈ B (Γ)

^{, anbehosenina Borel}

measurable waysuhthat

ρ(t, .) ˙

^isa probabilitymeasureon

B (Γ)

^foreah

t ≥ 0

^{, and}

ρ(B ) =

Z _∞

0

Z

Γ

1 _{(γ,t)∈B} ρ(t, dγ) ˙ dt

^{for all}

B ∈ B (Γ × [0, ∞ )).

Denoteby

R

^{the spaeof}deterministi relaxedontrolswhih isequipped withthe weak- ompat topology indued by the following notion of onvergene: a sequene

(ρ n ) _n∈

N

of

relaxedontrolsonverges to

ρ ∈ R

^if

Z

Γ×[0,∞)

g(γ, t) dρ _n (γ, t) ^n→∞ −→

Z

Γ×[0,∞)

g(γ, t) dρ(γ, t)

^{for all}

g ∈ C _c (Γ × [0, ∞ )),

where

C _c (Γ × [0, ∞ ))

^{isthespaeofall}real-valuedontinuousfuntionson

Γ × [0, ∞ )

^having

ompatsupport. Underthe weak-ompattopology,

R

^isa(sequentially)ompat spae.

Suppose

(ρ _n ) _n∈

N

is a onvergent sequene in

R

^{with limit}

ρ

^{. Given}

T > 0

^{, let}

ρ _n|T

denote the restrition of

ρ n

^{to the Borel}

σ

^{-algebra on}

Γ × [0, T ]

^{, and denote by}

ρ _|T

^the

restrition of

ρ

^to

B (Γ × [0, T ])

^{. Then}

ρ _n|T

^,

n ∈ N

,

ρ _|T

^{areall nite measures,and}

(ρ _n|T )

onvergesweaklyto

ρ _|T

^.

A relaxed ontrol proess is an

R

^{-valued random variable}

R

^{suh that the mapping}

ω 7→ R(G × [0, t])(ω)

^is

F ^t

-measurable for all

t ≥ 0

^,

G ∈ B (Γ)

^{. For a relaxed ontrol}

proess

R

^{equation (1)takes onthe form}

(6)

dX (t) = Z

Γ

b(X _t , γ) ˙ R(t, dγ )

dt + σ(X _t ) dW (t), t ≥ 0,

where

( ˙ R(t, .)) t≥0

^{is the family of derivative measures assoiated with}

R

^{. The family}

( ˙ R(t, .))

^{an be onstruted in a measurable way (f. Kushner, 1990:p.52). A relaxed}

ontrol proess together with its stohasti basis inluding the Wiener proess is alled

admissible relaxed ontrol if, for every deterministi initial ondition, equation (6) has

a unique solution whih is also weakly unique. Any ordinary ontrol proess

u

^{an be}

represented asa relaxedontrol proessbysetting

R(B ) :=

Z _∞

0

Z

Γ

1 _{(γ,t)∈B} δ _u(t) (dγ) dt, B ∈ B (Γ × [0, ∞ )),

where

δ γ

^{is the Dirameasureat}

γ ∈ Γ

^.

Denoteby

U ˆ ^ad

^{the setofalladmissible relaxedontrols. Insteadof (3)wedeneaost}

funtionalon

D ₀ × U ˆ ad

^by

(7)

J(ϕ, R) := ˆ E Z τ

0

Z

Γ

exp( − βs) · k X ^ϕ,R (s), γ R(s, dγ) ˙ ds + g X ^ϕ,R (τ )

,

where

X ^ϕ,R

^{is the solution to equation (6) with initial segment}

ϕ

^and

τ

^{is dened in}

analogyto (2). Instead of (4) asvalue funtionwehave

(8)

V ˆ (ϕ) := inf { J ˆ (ϕ, R) | R ∈ U ˆ ^ad } , ϕ ∈ D ₀ .

Theostfuntional

J ˆ

^{dependsonly onthe joint} distributionof the solution

X ^ϕ,R

^{and the}

underlying ontrol proess

R

^{, sine}

τ

^{, the time horizon,is a} deterministi funtion of the solution. The distribution of

X ^ϕ,R

^{, in turn, is determined bythe initial ondition}

ϕ

^and

thejointdistribution oftheontrolproessandits aompanying Wienerproess. Letting

thetime horizonvary,we mayregard

J ˆ

^{asafuntionofthe lawof}

(X, R, W, τ )

^{, thatis, to}

be dened on a subset of the set of probability measures on

B (D _∞ × R × D ˜ _∞ × [0, ∞ ])

^.

Thedomainof denitionof

J ˆ

^{isdetermined bythe lassofadmissible relaxedontrolsfor}

equation(6),thedenitionofthetimehorizonandthedistributionsoftheinitialsegments

X 0

^.

The idea in proving existene of an optimal strategy is to hekthat

J ˆ (ϕ, .)

^{is a (se-}

quentially)lowersemi-ontinuou sfuntiondenedona(sequentially)ompatset. Itthen

follows fromatheorem byWeierstraÿ (f. Yong andZhou, 1999:p.65) that

J ˆ (ϕ, .)

^attains

its minimum at some point of its ompat domain. The following proposition gives the

analogueofTheorem10.1.1inKushnerandDupuis(2001:pp.271-275)foroursetting. We

present the proof in detail,beause the identiation ofthe limit proess isdierent from

the lassialase.

Proposition1. Assume(A1)(A4). Let

((R ⁿ , W ⁿ )) _n∈

N

beanysequeneofadmissiblere-

laxedontrolsforequation (6),dened onalteredprobability spae

(Ω n , F ⁿ , ( F t ⁿ ) t≥0 , P n )

^.

Let

X ⁿ

^{bea solutiontoequation (6)underontrol}

(R ⁿ , W ⁿ )

^withdeterministi initialon- dition

ϕ ⁿ ∈ D ₀

^{, and assume that}

(ϕ ⁿ )

^{tends to}

ϕ

^{uniformly for some}

ϕ ∈ D ₀

^{. For eah}

n ∈ N

, let

τ ⁿ

^{be an}

( F t ⁿ )

^{-stopping time. Then}

((X ⁿ , R ⁿ , W ⁿ , τ ⁿ )) _n∈

N

is tight.

Denote by

(X, R, W, τ )

^{a limit point of the sequene}

((X ⁿ , R ⁿ , W ⁿ , τ )) _n∈

N

. Dene

a ltration by

F ^t := σ(X(s), R(s), W (s), τ 1 _{τ≤t} , s ≤ t)

^,

t ≥ 0

^{. Then}

W (.)

^{is an}

( F ^t )

^-

adaptedWienerproess,

τ

^isan

( F t )

^{-stoppingtime,}

(R, W )

^{isanadmissiblerelaxedontrol,}

and

X

^{isa solution to (6) under}

(R, W )

^{withinitial ondition}

ϕ

^.

Proof. Tightness of

(X ⁿ )

^{follows from the Aldous riterion (f.} Billingsley, 1999:pp.176 - 179): given

n ∈ N

, anybounded

( F t ⁿ )

^{-stoppingtime}

ν

^and

δ > 0

^{we have}

E _n

X ⁿ (ν + δ) − X ⁿ (ν)

2 F ^ν

≤ 2K ² δ(δ + 1)

asaonsequene ofAssumption(A3)andthe Itisometry. Notiethatwehave

X ⁿ (0) → X(0)

^as

n → ∞

^byhypothesis. The sequenes

(R ⁿ )

^and

(τ ⁿ )

^{are tight,beause the value}

spaes

R

^and

[0, ∞ ]

^, respetively, are ompat. The sequene

(W ⁿ )

^{is tight, sine all}

W ⁿ

^{indue the same measure. Finally,} omponentw ise tightness implies tightness of the produt (f.Billingsley, 1999:p.65).

Byabuseofnotation,wedonotdistinguishbetweentheonvergentsubsequeneandthe

originalsequeneandweassumethat

((X ⁿ , R ⁿ , W ⁿ , τ ⁿ ))

^{onvergesweaklyto}

(X, R, W, τ )

^.

The random time

τ

^{is an}

( F t )

^{-stopping time by onstrution of the ltration. Likewise,}

R

^is

( F ^t )

^{-adapted by} onstrution, and it is indeed a relaxed ontrol proess, beause

R(t, Γ) = t

^,

t ≥ 0

^,

P

^{-almost surely by weak onvergene of the relaxed ontrol proesses}

(R ⁿ )

^to

R

^{. Theproess}

W

^hasWiener distributionandontinuouspathswith probability one, being the limit of standard Wiener proesses. To hek that

W

^{is an}

( F ^t )

^-Wiener

proess,weusethe martingaleproblemharaterization ofBrownian motion. Tothis end,

for

g ∈ C _c (Γ × [0, ∞ ))

^,

ρ ∈ R

^{dene the pairing}

(g, ρ)(t) :=

Z

Γ×[0,t]

g(γ, s) dρ(γ, s), t ≥ 0.

Notie that real-valued ontinuous funtions on

R

^{an be} approximated by funtions of the form

R ∋ ρ 7→ H ˜ (g _j , ρ)(t _i ), (i, j) ∈ N _p × N _q

∈ R ,

where

p

^,

q

^{are natural numbers,}

{ t _i | i ∈ N _p } ⊂ [0, ∞ )

^{, and}

H ˜

^,

g _j

^,

j ∈ N _q

, are suitable ontinuous funtions with ompat support and

N _N := { 1, . . . , N }

^{for any}

N ∈ N

. Let

t ≥ 0

^,

t ₁ , . . . , t p ∈ [0, t]

^,

h ≥ 0

^,

g ₁ , . . . , g q

^{be funtions in}

C _c (Γ × [0, ∞ ))

^{, and}

H

^{be a}

ontinuous funtion of

2p + p · q + 1

^{arguments with ompat support. Sine}

W ⁿ

^{is an}

( F t ⁿ )

^{-Wiener proess for eah}

n ∈ N

, we havefor all

f ∈ C ² _c ( R )

E _n

H X ⁿ (t _i ), (g _j , R ⁿ )(t _i ), W ⁿ (t _i ), τ ⁿ 1 _{τ n

≤t} , (i, j) ∈ N _p × N _q

·

f W ⁿ (t + h)

− f W ⁿ (t)

− 1 2

Z t+h

t

∂ ² f

∂x ² W ⁿ (s) ds

= 0.

Bythe weak onvergene of

((X ⁿ , R ⁿ , W ⁿ , τ ⁿ )) _n∈

N

to

(X, W, R, τ )

^{we seethat}

E

H X(t i ), (g j , R)(t i ), W (t i ), τ 1 _{τ≤t} , (i, j) ∈ N _p × N _q

·

f W (t + h)

− f W (t)

− 1 2

Z t+h

t

∂ ² f

∂x ² W (s) ds

= 0

for all

f ∈ C ² _c ( R )

^{. As}

H

^,

p

^,

q

^,

t _i

^,

g _j

^{vary over all} possibilities, the orresponding random variables

H(X(t i ), (g j , R)(t i ), W (t i ), τ 1 _{τ≤t} , (i, j) ∈ N _p × N _q )

^{indue the}

σ

^-algebra

F ^t

^.

Sine

t ≥ 0

^,

h ≥ 0

^{werearbitrary, itfollowsthat}

f W (t)

− f W (0)

− 1 2

Z t

0

∂ ² f

∂x ² W (s)

ds, t ≥ 0,

isan

( F ^t )

-martingale for every

f ∈ C ²

c ( R )

^. Consequently,

W

^isan

( F ^t )

^{-Wiener proess.}

Itremains toshowthat

X

^{solvesequation(6)under ontrol}

(R, W )

^{withinitial ondi-}

tion

ϕ

^{. Notiethat}

X

^{hasontinuouspathson}

[0, ∞ ) P

^{-almostsurely,beausetheproess}

(X(t)) t≥0

^{is the weak limit in}

D ˜ ∞

^{of ontinuous proesses. Fix}

T > 0

^{. We have to hek}

that

P

^{-almost surely}

X(t) = ϕ(0) +

Z t 0

Z

Γ

b(X s , γ) ˙ R(s, dγ) ds + Z t

0

σ(X s ) dW (s)

^{for all}

t ∈ [0, T ].

By virtue of the Skorohod representation theorem (f. Billingsley, 1999:p.70) we may

assume that the proesses

(X ⁿ , R ⁿ , W ⁿ )

^,

n ∈ N

, are all dened on the same probability spae

(Ω, F , P)

^as

(X, R, W )

^{and that onvergene of}

((X ⁿ , R ⁿ , W ⁿ ))

^to

(X, R, W )

^is

P

^-

almostsure. Sine

X

^,

W

^{have ontinuous paths on}

[0, T ]

^and

(ϕ ⁿ )

^{onverges to}

ϕ

^{in the}

uniform topology, onends

Ω ˜ ∈ F

^with

P( ˜ Ω) = 1

^{suh thatfor all}

ω ∈ Ω ˜ sup

t∈[−r,T ]

X ⁿ (t)(ω) − X(t)(ω)

n→∞ −→ 0, sup

t∈[−r,T ]

W ⁿ (t)(ω) − W (t)(ω)

n→∞ −→ 0,

andalso

R ⁿ (ω) → R(ω)

ⁱⁿ

R

^{. Let}

ω ∈ Ω ˜

^{. Werst showthat}

Z t

0

Z

Γ

b X _s ⁿ (ω), γ R ˙ ⁿ (s, dγ)(ω) ds ^n→∞ → Z t

0

Z

Γ

b X s (ω), γ R(s, dγ)(ω) ˙ ds

uniformlyin

t ∈ [0, T ]

^{. Asaonsequene ofAssumption(A4), the uniform onvergeneof}

the trajetories on

[ − r, T ]

^{and property(5) ofthe relaxed ontrols, we have}

Z

Γ×[0,T]

b X _s ⁿ (ω), γ

− b X s (ω), γ

dR ⁿ (γ, s)(ω) ^n→∞ → 0.

ByAssumption(A2), we nd aountable set

A _ω ⊂ [0, T ]

^{suhthatthe mapping}

(γ, s) 7→

b(X s (ω), γ)

^{is ontinuous in all}

(γ, s) ∈ Γ × ([0, T ] \ A ω )

^{. Sine}

A ω

^{is ountable we have}

R(ω)(Γ × A ω ) = 0

^{. Hene,bythe}generalizedmappingtheorem(f.Billingsley,1999:p.21), we obtain

Z

Γ×[0,t]

b X s (ω), γ

dR ⁿ (γ, s)(ω) ^n→∞ → Z

Γ×[0,t]

b X s (ω), γ

dR(γ, s)(ω).

Theonvergene is again uniform in

t ∈ [0, T ]

^{, as}

b

^{is bounded and}

R ⁿ

^,

n ∈ N

,

R

^areall

positive measures with mass

T

^on

Γ × [0, T ]

^.

Denote by

( ˆ X(t)) _t≥−r

^{the unique solution to equation (6) under ontrol}

(R, W )

^with

initial ondition

ϕ

^{. If we an showthat}

(9)

sup

t∈[0,T ]

X ⁿ (t) − X(t) ˆ

n→∞ −→ 0

ⁱⁿ probability

P,

then

X

^willbeindistinguishablefrom

X ˆ

^on

[ − r, T ]

^{andwillsolve(6)aswell. Letusdene}

àdlàgproesses

C ⁿ

^,

n ∈ N

, on

[0, ∞ )

^by

C ⁿ (t) := ϕ ⁿ (0) +

Z

Γ×[0,t]

b(X _s ⁿ , γ) dR ⁿ (γ, s), t ≥ 0,

anddene

C

^{in analogyto}

C ⁿ

^{. Wealreadyknowthat}

C ⁿ (t) → C(t)

^{holdsuniformlyover}

t ∈ [0, T ]

^forany

T > 0

^withprobabilityone. Deneoperators

F ⁿ

^,

n ∈ N

, mappingàdlàg proesses to àdlàgproesses by

F ⁿ (Y )(t)(ω) := σ



[ − r, 0] ∋ s 7→







Y (t+s)(ω)

^if

t ≥ − s, ϕ ⁿ (t+s)

^else



 , t ≥ 0, ω ∈ Ω,

and dene

F

^{in the same way as}

F ⁿ

^{. Assumption (A4) and the uniform onvergene of}

(ϕ ⁿ )

^to

ϕ

^{imply that}

F ⁿ ( ˆ X)

^{onverges to}

F( ˆ X)

^{uniformly on ompats in} probability (onvergene in up). Observing that

X ⁿ

^solves

X ⁿ (t) = C ⁿ (t) + Z t

0

F ⁿ (X ⁿ )(s − ) dW ⁿ (s), t ≥ 0,

andanalogouslyfor

X ˆ

^{, TheoremV.15inProtter(2003:p.265 )assertsthat}

(X ⁿ )

^onverges

to

X ˆ

^{in upand (9)follows.}

If the time horizon were deterministi, then the existene of optimal strategies in the

lassofrelaxedontrolswouldbelear. Givenaninitialondition

ϕ ∈ D ₀

^{,onewouldselet}

asequene

((R ⁿ , W ⁿ )) _n∈

N

suhthat

(J (ϕ, R ⁿ ))

^{onvergestoitsinmum. By}Proposition1, asuitablesubsequeneof

((R ⁿ , W ⁿ ))

^{andtheassoiatedsolutionproesseswouldonverge}

weaklyto

(R, W )

^{andtheassoiated solutionto equation(6). Takingintoaount (7), the}

denitionof the osts,this in turn wouldimply that

J (ϕ, .)

^{attainsits minimum value at}

R

^{or,more preisely,}

(X, R, W )

^.

Asimilarargumentisstillvalid,ifthetimehorizon dependsontinuouslyonthepaths

with probability one under every possible solution. Thatis to say, the mapping

ˆ

τ : D _∞ → [0, ∞ ], τ ˆ (ψ) := inf { t ≥ 0 | ψ(t) ∈ / int(I) } ∧ T ¯

(10)

is Skorohod ontinuous with probability one under the measure indued by any solution

X ^ϕ,R

^,

R

^{anyrelaxedontrol. Thisisindeedtheaseifthediusionoeient}

σ

^isbounded

awayfrom zeroasrequired byAssumption(A5).

Byintroduingrelaxedontrols, we haveenlarged the lassofpossible strategies. The

inmumoftheosts,however,remainsthesameforthenewlass. Thisisaonsequeneof

onstant ordinarystohastiontrolswhihattainonly anitenumberofdierentontrol

values. A proof of this assertion is given in Kushner (1990:pp.59-60) in ase the time

horizon is nite, and extended to the ase of ontrol up to an exit time in Kushner and

Dupuis(2001:pp.28 2-286). Notiethatnothinghingesonthepreseneor abseneofdelay

in the ontrolled dynamis. Letus summarize our ndings.

Theorem 1. Assume (A1)(A5). Given any deterministi initial ondition

ϕ ∈ D ₀

^{, the}

relaxed ontrol problem determined by (6) and (7) possesses an optimal strategy, and the

minimalosts are the same as for the original ontrol problem as dened by (1) and (3).

4 Approximating hains

In order to onstrut nite-dimensional approximations to our ontrol problem, we dis-

retize time and state spae. Denote by

h > 0

^{the mesh size of an} equidistant time disretization starting at zero. Let

S _h := √

h Z

be the orresponding state spae, and set

I _h := I ∩ S _h

^{. Notie that}

S _h

^{is ountable and}

I _h

^{is nite. Let}

Λ _h : R → S _h

^{be a}

round-o funtion. We will simplify things even further by onsidering only mesh sizes

h = _M ^r

^{for some}

M ∈ N

, where

r

^{is the delay length. The number}

M

^{will be referred to}

asdisretizationdegree .

Theadmissibleontrolsforthenite-dimen sio nalontrolproblemsorrespondtopiee-

wise onstant proesses in ontinuous time. A time-disrete proess

u = (u(n)) _n∈

^N

0

^on

(Ω, F , P)

^{with values in}

Γ

^{is a disrete admissible ontrol of degree}

M

^if

u

^{takes on only}

nitely manydierent values in

Γ

^and

u(n)

^is

F nh

-measurable for all

n ∈ N ₀

. Denote by

(˜ u(t)) _t≥0

^{the pieewise onstant àdlàg}interpolatio n to

u

^.

We allatime-disrete proess

(ξ(n)) n∈{−M,...,0}∪

^{N on}

(Ω, F , P)

^{adisrete hain of de-}

gree

M

^if

(ξ(n))

^{takesitsvaluesin}

S h

^and

ξ(n)

^is

F ^nh

-measurableforall

n ∈ N ₀

. Inanalogy to

u ˜

^,write

( ˜ ξ(t)) _t≥−r

^{fortheàdlàg}interpolatio ntothedisretehain

(ξ(n)) n∈{−M,...,0}∪

^N.

We denoteby

ξ ˜ t

^the

D 0

^{-valued segment of}

ξ(.) ˜

^{at time}

t ≥ 0

^.

Let

ϕ ∈ D ₀

^beadeterministiinitialondition, andsupposewearegivenasequene of disrete admissibleontrols

(u ^M ) M ∈

^N,thatis

u ^M

^{isadisrete admissibleontrolofdegree}

M

^{on a stohasti basis}

(Ω M , F ^M , ( F t ^M ), P M )

^{for eah}

M ∈ N

. In addition, suppose thatthe sequene

(˜ u ^M )

^ofinterpolated disreteontrolsonverges weaklyto somerelaxed ontrol

R

^{. We are then looking for a sequene} approximatingthe solution

X

^{of equation}

(1)under ontrol

(R, W )

^{with initial ondition}

ϕ

^{, where the Wienerproess}

W

^{has to be}

onstrutedfrom the approximatingsequene.

Given

M

^{-step or extended Markov transition funtions}

p ^M : S ^M _h ⁺¹ × Γ × S _h → [0, 1]

^,

M ∈ N

, we dene a sequene of approximating hains assoiated with

ϕ

^and

(u ^M )

^{as a}

family

(ξ ^M ) _{M ∈}

N

of proessessuh that

ξ ^M

^{is adisrete hainof degree}

M

^{dened on the}

samestohastibasisas

u ^M

^{,providedthefollowingonditionsarefullledfor}

h = h _M := _M ^r

tendingto zero:

(i) Initial ondition:

ξ ^M (n) = Λ _h (ϕ(nh))

^{for all}

n ∈ {− M, . . . , 0 }

^.

(ii) Extended Markovproperty: for all

n ∈ N ₀

, all

x ∈ S _h P _M ξ ^M (n+1) = x

F nh ^M

= p ^M ξ ^M (n − M ), . . . , ξ ^M (n), u ^M (n), x .

(iii) Loal onsistenywith the drift oeient:

µ _ξ ^M (n) := E _M ξ ^M (n+1) − ξ ^M (n) F nh ^M

= h · b ξ ˜ _nh ^M , u ^M (n)

+ o(h) =: h · b _h ξ ˜ _nh ^M , u ^M (n) .

(iv) Loal onsistenywith the diusion oeient:

E _M ξ ^M (n+1) − ξ ^M (n) − µ _ξ ^M (n) 2 F nh ^M

= h · σ ² ( ˜ ξ ^M _nh ) + o(h) =: h · σ _h ² ( ˜ ξ _nh ^M ).

(v) Jump heights: thereis apositive number

N ¯

^, independent of

M

^{, suh that}

sup

n | ξ ^M (n + 1) − ξ ^M (n) | ≤ N ¯ p h M .

It is straightforward, under Assumptions (A3) and (A5), to onstrut a sequene of ex-

tendedtransitionfuntionssuhthatthe jumpheight andtheloalonsistenyonditions

arefullled.

We will represent the interpolatio n

ξ ˜ ^M

^{as a solution to an equation} orresponding to equation(1) with ontrol proess

u ˜ ^M

^{and initial ondition}

ϕ

^{. Dene the disrete proess}

(L ^M (n)) _n∈

N

0

^by

L ^M (0) := 0

^and

ξ ^M (n) = ϕ(0) +

n−1

X

i=0

h · b h ξ ˜ _ih ^M , u ^M (i)

+ L ^M (n), n ∈ N .

Observe that

L ^M

^{is a martingale in disrete time with respet to the ltration}

( F nh ^M )

^.

Setting

ε ^M ₁ (t) :=

⌊ h ^t ⌋−1

X

i=0

h · b _h ξ ˜ _ih ^M , u ˜ ^M (ih)

− Z t

0

b ξ ˜ ^M _s , u ˜ ^M (s)

ds, t ≥ 0,

the interpolated proess

ξ ˜ ^M

^{an be} represented assolutionto

ξ ˜ ^M (t) = ϕ(0) + Z t

0

b ξ ˜ _s ^M , u ˜ ^M (s)

ds + L ^M ( ⌊ _h ^t ⌋ ) + ε ^M ₁ (t), t ≥ 0.

For the error termwehave

E _M | ε ^M ₁ (t) |

≤

⌊ h ^t ⌋−1

X

i=0

h E _M

b h ξ ˜ _ih ^M , u ^M (i)

− b ξ ˜ _ih ^M , u ^M (i)

+ K · h

+

Z _{h⌊ h} ^t _⌋

0

E _M

b ξ ˜ _h⌊ ^{M s}

h ⌋ , u ˜ ^M (s)

− b ξ ˜ _s ^M , u ˜ ^M (s)

ds,

whih tends to zero as

M → ∞

^{uniformly in}

t ∈ [0, T ]

^by Assumptions (A2), (A3), dominatedonvergene andthe deningproperties of

(ξ ^M )

^{. The} disrete-timemartingale

L ^M

^{an be rewritten asdisrete stohastiintegral. Dene}

(W ^M (n)) _n∈

N

0

^by

W ^M (0) := 0

and

W ^M (n) :=

n−1

X

i=0

1

σ( ˜ ξ _ih ^M ) L ^M (i+1) − L ^M (i)

, n ∈ N .

Usingthe pieewiseonstant interpolation

W ˜ ^M

^of

W ^M

^{, the proess}

ξ ˜ ^M

^{an be expressed}

assolutionto

(11)

ξ ˜ ^M (t) = ϕ(0) + Z t

0

b ξ ˜ ^M _s , u ˜ ^M (s) ds +

Z t 0

σ( ˜ ξ _h⌊ ^{M s} −

h ⌋ ) d W ˜ ^M (s) + ε ^M ₂ (t)

for

t ≥ 0

^{, where the error terms}

(ε ^M ₂ )

^{onverge to zeroas}

(ε ^M ₁ )

^before.

Wearenowpreparedfortheonvergeneresult,whihshouldbeomparedtoTheorem

10.4.1in KushnerandDupuis(2001:p.290). Theproofissimilarto thatof Proposition 1.

We merelypoint out the main dierenes.

Proposition 2. Assume (A1)(A5). For eah

M ∈ N

, let

τ ^M

^{be a stopping time with}

respet to the

σ

^{-algebra generated by}

( ˜ ξ ^M (s), u ˜ ^M (s), W ˜ ^M (s))

^,

s ≤ t

^{. If}

( ˜ ξ ₀ ^M )

^onverges

to

ϕ

^{in the uniform topology, then}

(( ˜ ξ ^M , R ^M , W ˜ ^M , τ ^M )) M ∈

^{N istight. For any limit point}

(X, R, W, τ )

^dene

F ^t := σ X(s), R(s), W (s), τ 1 _{τ≤t} , s ≤ t

, t ≥ 0.

Then

W

^{is an}

( F t )

^{-adapted Wiener proess,}

τ

^{is an}

( F t )

^{-stopping time,}

(R, W )

^{is an}

admissible relaxed ontrol, and

X

^{is a solution to (6) under}

(R, W )

^{withinitial ondition}

ϕ

^.

Proof. For the rst part, the only dierene is the proof of tightness for

( ˜ W ^M )

^{and the}

identiationofthelimitpoints. Wealulatetheorderofonvergeneforthedisrete-time

previsiblequadrati variations of

(W ^M )

^:

h W ^M i ⁿ =

n−1

X

i=0

E (W ^M (i+1) − W ^M (i)) ² F ih ^M

= nh + o(h)

n−1

X

i=0

1 σ ² ( ˜ ξ _ih ^M )

for all

M ∈ N

,

n ∈ N ₀

. Taking into aount Assumption (A5) and the denition of the time-ontinuous proesses

W ˜ ^M

^{, we see that}

h W ˜ ^M i

^{tends to}

Id _[0,∞)

ⁱⁿ probability for

M → ∞

^{. By Theorem VIII.3.11 of Jaod and Shiryaev (1987:p.432) we onlude}

that

( ˜ W ^M )

^{onverges weakly to a standard Wiener proess}

W

^{. That}

W

^has independent inrements with respet to the ltration

( F t )

^{an be seen by onsidering the rst and}

seond onditional moments of the inrements of

W ^M

^{for eah}

M ∈ N

and applying the onditionson loalonsisteny and the jumpheightsof

(ξ ^M )

^.

ByvirtueofSkorohod'stheorem,wemayagainworkunder

P

^{-almostsureonvergene.}

Theremaining slightly dierent partis the identiation of

X

^{assolution to equation (6)}

under

(R, W )

^{withinitialondition}

ϕ

^{. Notiethat}

X

^{isontinuouson}

[0, ∞ )

^beauseofthe

onditionon the jumps of the

ξ ^M

^{, f.Theorem 3.10.2 in Ethier and Kurtz (1986:p.148).}

Letus dene àdlàgproesses

C ^M , C

^on

[0, ∞ )

^by

C ^M (t) := ϕ ^M (0) +

Z t 0

b ξ ˜ _s ^M , u ˜ ^M (s)

ds + ε ^M ₂ (t), t ≥ 0, C(t) := ϕ(0) +

Z

Γ×[0,t]

b(X _s , γ) dR(γ, s), t ≥ 0.

We then infer that

C ^M → C

^{in up as before. Dene operators}

F ^M

^{, mapping àdlàg}

proesses to àdlàgproesses,by

F ^M (Y )(t) := σ



[ − r, 0] ∋ s 7→







Y h ⌊ _h ^t ⌋ +s

if

t ≥ − s, ξ ˜ ^M h ⌊ h ^t ⌋ +s

else



 , t ≥ 0,

anddene

F

^{asintheproofof}Proposition1. Denoteby

( ˆ X(t)) _t≥−r

^{theuniquesolutionto}

equation(6)under ontrol

(R, W )

^{with initialondition}

ϕ

^{. Assumption(A4),the uniform}

onvergene of

( ˜ ξ ^M ₀ )

^to

ϕ

^{and the} right-ontinui ty of

ϕ

^{imply that}

F ^M ( ˆ X)

^{onverges to}

F( ˆ X)

^{in up. Notiethat}

X ˆ

^{isontinuouson}

[0, ∞ )

^{assolutionto (6).}

ξ ˜ ^M

^solves

ξ ˜ ^M (t) = C ^M (t) + Z t

0

F ^M ( ˜ ξ ^M )(s − ) d W ¯ ^M (s), t ≥ 0,

wherewehavetakenaontinuousmartingaleinterpolatio n

W ¯ ^M

^of

W ^M

^insteadof

W ˜ ^M

^as

integratorin the stohastiintegral, whihyields anidentialresultsine the integrandis

a pure jump proess with jump times at

kh

^,

k ∈ N ₀

.

( ¯ W ^M )

^{also onverges to}

W

^{in up,}

suh that by Exerise IV.4.14 in Revuz and Yor (1999) onvergene in the semimartin-

galetopology follows. We onlude again byTheorem V.15 in Protter (2003) that

(X ^M )

onvergesto

X ˆ

^{in upand that}

X

^and

X ˆ

^areindistinguishable.

5 Convergene of the minimal osts

Theobjetivebehindtheintrodutionofsequenes ofapproximatinghainswastoobtain

adevieforapproximatingthevaluefuntion

V

^{oftheoriginalproblem. Theideanowisto}

dene,foreahdisretizationdegree

M ∈ N

,adisreteontrolproblemwithostfuntional

J ^M

^sothat

J ^M

^isanapproximationofthe ostfuntional

J

^{oftheoriginalproblemin the}

following sense: Given an initial segment

ϕ ∈ D ₀

^{and a sequene of disrete admissible}

ontrols

(u ^M )

^{suh that}

(˜ u ^M )

^{weakly onverges to}

u ˜

^{, we have}

J ^M (ϕ, u ^M ) → J (ϕ, u) ˜

^as

M → ∞

^{. Under the} assumptions introdued above, it will follow that also the value funtions assoiated with the disrete ost funtionals onverge to the value funtion of

the originalproblem.

Fix

M ∈ N

, and let

h := _M ^r

^{. Denote by}

U ad ^M

^{the set of disrete admissible ontrols of}

degree

M

^{. Denethe ost funtionalof degree}

M

^by

(12)

J ^M ϕ, u := E

N h −1

X

n=0

exp( − βnh) · k ξ(n), u(n)

· h + g ξ(N h )

!

,

where

ϕ ∈ D ₀

^,

u ∈ U ad ^M

^{is dened on the stohasti basis}

(Ω, F , ( F t ), P)

^and

(ξ(n))

^{is a}

disrete hain of degree

M

^{dened aording to}

p ^M

^and

u

^{with initial ondition}

ϕ

^{. The}

disreteexit time step

N _h

^{is given by}

(13)

N h := min { n ∈ N ₀ | ξ(n) ∈ / I h } ∧ ⌊ ^T h ^¯ ⌋ .

Denote by

˜ τ ^M := h · N h

^{the exit time for the} orresponding interpolated proesses. The valuefuntion of degree

M

^{isdened as}

(14)

V ^M (ϕ) := inf

J ϕ, u u ∈ U ad ^M , ϕ ∈ D ₀ .

We are now in a position to state the result about onvergene of the minimal osts.

Proposition 3 and Theorem 2 are omparable to Theorems 10.5.1 and 10.5.2 in Kushner

andDupuis (2001:pp.292-295).

Proposition 3. Assume (A1)(A5). If the sequene

( ˜ ξ ^M , u ˜ ^M , W ˜ ^M , τ ˜ ^M )

^of interpolated proesses onverges weakly to a limit point

(X, R, W, τ )

^{, then}

X

^{is a solution to equation}

(6)under relaxed ontrol

(R, W )

^{withinitialondition}

ϕ

^,

τ

^{isthe exit timefor}

X

^{as given}

by (2), andwe have

J ^M (ϕ, u ^M ) ^M −→ ^→∞ J ˆ (ϕ, R).

Proof. The onvergene assertion for the ostsisa onsequene of Proposition 2,the fat

that,byvirtueofAssumption(A5),the exittime

τ ˆ

^{denedin(10)is}Skorohod-onti nuous, andthe denition of

J ^M

^and

J

^(or

J ˆ

^).

Theorem2. Assume(A1)(A5). Thenwehave

lim _M→∞ V ^M (ϕ) = V (ϕ)

^{for all}

ϕ ∈ D ₀

^.

Proof. First notie that

lim inf _M→∞ V ^M (ϕ) ≥ V (ϕ)

^{as a onsequene of} Proposition 2.

In order to show

lim sup _M→∞ V ^M (ϕ) ≤ V (ϕ)

^{hoose a relaxed ontrol}

(R, W )

^{so that}

J(ϕ, R) = ˆ V (ϕ)

^by Proposition 1. Given

ε > 0

^{, one an onstrut a sequene of disrete}

admissibleontrols

(u ^M )

^suhthat

(( ˜ ξ ^M , u ˜ ^M , W ˜ ^M , τ ˜ ^M ))

^{isweaklyonvergent,where}

( ˜ ξ ^M )

^,

( ˜ W ^M )

^,

(˜ τ ^M )

^{areonstruted asabove,and}

lim sup _{M →∞} | J ^M (ϕ, u ^M ) − J ˆ (ϕ, R) | ≤ ε

^{. The}

existeneofsuhasequeneofdisreteadmissibleontrolsisguaranteed,f.the disussion

at the end of Setion 3. By denition,

V ^M (ϕ) ≤ J ^M (ϕ, u ^M )

^{for eah}

M ∈ N

. Using Proposition 3 we ndthat

lim sup

M→∞

V ^M (ϕ) ≤ lim sup

M→∞

J ^M (ϕ, u ^M ) ≤ V (ϕ) + ε,

andsine

ε

^{wasarbitrary, the assertion follows.}

AllainBensoussan,GiuseppeDaPrato,MihelC.Delfour,andSanjoyK.Mitter.Represen-

tation and Control of Innite-Dimensional Systems , volume 1 of Systems& Control.

Birkhäuser, Boston,1992.

Patrik Billingsley. Convergene of probability measures. John Wiley & Sons, NewYork,

2nd edition, 1999.

RaoufBouekkine, Omar Liandro, LuisA. Puh, and Fernandodel Rio. Vintageapital

and the dynamis of the AKmodel. Journalof Eonomi Theory ,120:3972,2005.

Evelyn Bukwar. Introdution to the numerial analysis of stohasti delay dierential

equations. J. Comput.Appl. Math., 125(1-2):297307, 2000.

Antonella Calzolari, Patrik Florhinger, and Giovanna Nappo. Convergene in nonlin-

ear ltering for stohasti delay systems. Tehnial report, Università di Roma La

Sapienza, 2003.

IsmailElsanosi,BerntØksendal,andAgnèsSulem. Somesolvablestohastiontrolprob-

lems with delay. Stohastis and Stohasti Reports,71(1-2):6989, 2000.

Stewart N. Ethier and Thomas G. Kurtz. Markov Proesses. Charaterization and Con-

vergene. John Wiley&Sons, NewYork,1986.

YaozhongHu,Salah-EldinA.Mohammed,andFengYan.Disrete-timeapproximationsof

stohastidelayequations: theMilstein sheme. Ann. Probab.,32(1A):265314, 2004.

JeanJaodand AlbertN.Shiryaev. Limit Theorems for Stohasti Proesses, volume 288

of Grundlehren der mathematishen Wissenshaften. Springer, Berlin,1987.

HaroldJ.Kushner.WeakConvergeneMethodsandSingularlyPerturbedStohastiControl

and Filtering Problems ,volume 3 ofSystems& Control. Birkhäuser,Boston, 1990.

HaroldJ. Kushnerand Paul Dupuis. Numerial Methods for Stohasti Control Problems

in Continuous time . Springer, NewYork,2nd edition, 2001.

Bjørnar Larssen. Dynami programmin g in stohasti ontrol of systems with delay.

Stohastis and Stohastis Reports,74(3-4):651673, 2002.

XuerongMao. Stohasti Dierential Equations andtheir Appliations. Horwood, Chih-

ester, 1997.

Salah-Eldi n A.Mohammed. Stohasti funtional dierential equations. Pitman Publish-

ing, London, 1984.

Bernt ØksendalandAgnès Sulem. A maximum priniplefor optimal ontrol of stohasti

systems with delay, with appliations to nane. In J.L. Menaldi, E. Rofman, and

A. Sulem, editors, Optimal Control and PDE., pages 6479. IOS Press, Amsterdam,

2001.

PhilipE. Protter. Stohasti Integrationand Dierential Equations. Springer,Berlin,2nd

edition, 2003.

Grundlehrender MathematishenWissenshaften. Springer,Berlin,3rdedition,1999.

Jiongmin Yong and Xun Yu Zhou. Stohasti Controls. Hamiltonian Systems and HJB

Equations. Springer, NewYork,1999.

SFB 649 Discussion Paper Series

For a complete list of Discussion Papers published by the SFB 649, please visit http://sfb649.wiwi.hu-berlin.de.

001 "Nonparametric Risk Management with Generalized Hyperbolic Distributions" by Ying Chen, Wolfgang Härdle and Seok-Oh Jeong, January 2005.

002 "Selecting Comparables for the Valuation of the European Firms" by Ingolf Dittmann and Christian Weiner, February 2005.

003 "Competitive Risk Sharing Contracts with One-sided Commitment" by Dirk Krueger and Harald Uhlig, February 2005.

004 "Value-at-Risk Calculations with Time Varying Copulae" by Enzo Giacomini and Wolfgang Härdle, February 2005.

005 "An Optimal Stopping Problem in a Diffusion-type Model with Delay" by Pavel V. Gapeev and Markus Reiß, February 2005.

006 "Conditional and Dynamic Convex Risk Measures" by Kai Detlefsen and Giacomo Scandolo, February 2005.

007 "Implied Trinomial Trees" by Pavel Čížek and Karel Komorád, February 2005.

008 "Stable Distributions" by Szymon Borak, Wolfgang Härdle and Rafal Weron, February 2005.

009 "Predicting Bankruptcy with Support Vector Machines" by Wolfgang Härdle, Rouslan A. Moro and Dorothea Schäfer, February 2005.

010 "Working with the XQC" by Wolfgang Härdle and Heiko Lehmann, February 2005.

011 "FFT Based Option Pricing" by Szymon Borak, Kai Detlefsen and Wolfgang Härdle, February 2005.

012 "Common Functional Implied Volatility Analysis" by Michal Benko and Wolfgang Härdle, February 2005.

013 "Nonparametric Productivity Analysis" by Wolfgang Härdle and Seok-Oh Jeong, March 2005.

014 "Are Eastern European Countries Catching Up? Time Series Evidence for Czech Republic, Hungary, and Poland" by Ralf Brüggemann and Carsten Trenkler, March 2005.

015 "Robust Estimation of Dimension Reduction Space" by Pavel Čížek and Wolfgang Härdle, March 2005.

016 "Common Functional Component Modelling" by Alois Kneip and Michal Benko, March 2005.

017 "A Two State Model for Noise-induced Resonance in Bistable Systems with Delay" by Markus Fischer and Peter Imkeller, March 2005.

SFB 649, Spandauer Straße 1, D-10178 Berlin

http://sfb649.wiwi.hu-berlin.de

018 "Yxilon – a Modular Open-source Statistical Programming Language" by Sigbert Klinke, Uwe Ziegenhagen and Yuval Guri, March 2005.

019 "Arbitrage-free Smoothing of the Implied Volatility Surface"

by Matthias R. Fengler, March 2005.

020 "A Dynamic Semiparametric Factor Model for Implied Volatility String Dynamics" by Matthias R. Fengler, Wolfgang Härdle and Enno Mammen, March 2005.

021 "Dynamics of State Price Densities" by Wolfgang Härdle and Zdeněk Hlávka, March 2005.

022 "DSFM fitting of Implied Volatility Surfaces" by Szymon Borak, Matthias R. Fengler and Wolfgang Härdle, March 2005.

023 "Towards a Monthly Business Cycle Chronology for the Euro Area" by Emanuel Mönch and Harald Uhlig, April 2005.

024 "Modeling the FIBOR/EURIBOR Swap Term Structure: An Empirical Approach" by Oliver Blaskowitz, Helmut Herwartz and Gonzalo de Cadenas Santiago, April 2005.

025 "Duality Theory for Optimal Investments under Model Uncertainty" by Alexander Schied and Ching-Tang Wu, April 2005.

026 "Projection Pursuit For Exploratory Supervised Classification"

by Eun-Kyung Lee, Dianne Cook, Sigbert Klinke and Thomas Lumley, May 2005.

027 "Money Demand and Macroeconomic Stability Revisited" by Andreas Schabert and Christian Stoltenberg, May 2005.

028 "A Market Basket Analysis Conducted with a Multivariate Logit Model" by Yasemin Boztuğ and Lutz Hildebrandt, May 2005.

029 "Utility Duality under Additional Information: Conditional Measures versus Filtration Enlargements" by Stefan Ankirchner, May 2005.

030 "The Shannon Information of Filtrations and the Additional Logarithmic Utility of Insiders" by Stefan Ankirchner, Steffen Dereich and Peter Imkeller, May 2005.

031 "Does Temporary Agency Work Provide a Stepping Stone to Regular Employment?" by Michael Kvasnicka, May 2005.

032 "Working Time as an Investment? – The Effects of Unpaid Overtime on Wages, Promotions and Layoffs" by Silke Anger, June 2005.

033 "Notes on an Endogenous Growth Model with two Capital Stocks II: The Stochastic Case" by Dirk Bethmann, June 2005.

034 "Skill Mismatch in Equilibrium Unemployment" by Ronald Bachmann, June 2005.

SFB 649, Spandauer Straße 1, D-10178 Berlin

http://sfb649.wiwi.hu-berlin.de

035 "Uncovered Interest Rate Parity and the Expectations Hypothesis of the Term Structure: Empirical Results for the U.S. and Europe" by Ralf Brüggemann and Helmut Lütkepohl, April 2005.

036 "Getting Used to Risks: Reference Dependence and Risk Inclusion" by Astrid Matthey, May 2005.

037 "New Evidence on the Puzzles. Results from Agnostic Identification on Monetary Policy and Exchange Rates." by Almuth Scholl and Harald Uhlig, July 2005.

038 "Discretisation of Stochastic Control Problems for Continuous Time Dynamics with Delay" by Markus Fischer and Markus Reiss, August 2005.

SFB 649, Spandauer Straße 1, D-10178 Berlin

http://sfb649.wiwi.hu-berlin.de