Dynamic Aspects: Multistage Recourse Problems

It should be emphasized that the "stages" of a two-stage recourse problem do not necessarily refer to time units. They correspond to steps in the decision process, x may be a here-and·now decision whereas they correspond toallfuture actions to be taken in different time period in response to the environment created by the chosen x and the observed w in that specific time period. In another instance, the x,'IIsolutions may represent sequences of control actions over a given time horizon,

x

= (x(O),x(I),_ .. ,x(T)),

'11=

(y(O),y(I), ... ,y(T)),

the y.decisions being used to correct for the basic trend set by the x·control variables. As a special case we have

x

= (x(O),x(I), ,x(s)),

'11=

(y(s+I), ,y(T)),

that corresponds to a mid-course maneuver at time s when some observations have become available to the controller. We speak oftwo'stage dynamic mod·

els. In what follows, we discuss in more detail the possible statements of such problems.

In the case of dynamical systems, in addition to the x,'II solutions of prob.

lems (1.26)-(1.25), there may also be an additional group of variables

z = (z(O),z(I), ... , z(T))

that record the state of the system at times 0,1, ... ,T. Usually, the variables x,'11,z, ware connected through a (differential) system of equations of the type:

..:lz(t) = h(t,z(t),x(t),y(t),w), t =

0, ...

,T -1,

where

..:lz(t) = z(t +

^{1) -}

^z(t),z(O) =

zo, or they are related by an implicit function of the type:

h(t,z(t + 1),z(t),x(t),y(t),w) =

0,

t =

0, . ..,T-1.

(1.28)

(1.29) The latter one of these is the typical form one finds in operations research mod·

els, economics and system analysis, the first one (1.28) is the conventional one inthe theory of optimal control and its applications in engineering, inventory control, etc. In the formulation (1.28) an additional computational problem arises from the fact that it is necessary to solve a large system of linear or nonlinear equations, in order to obtain a description of the evolution of the system.

16 Stochastic Optimization Problems The objective and constraints functions of stochastic dynamic problems are generally expressed in terms of mathematical expectations of functions that we take to be:

gj(z(O),x(O),y(O), ... ,z(T),x(T),y(T)),

i = 0, 1, .••

,m.

(1.30) Ifno observations are allowed, then equations (1.28), or (1.29), and (1.30) do not depend on y, and we have the following one-stage problem

find x=

(x(O),x(I), ... ,x(T))

such that

x(t)

^E

X(t) c R", t

= 0, ...

,T,

.az(t)=h(t,z(t),x(t),w), t=O, ... ,T-l,

E(g;(z(O),x(O), ... ,z(T),x(T),w)

~0, i = 1, ... ,m and

v

=

E{go (z(O),x(O), ... , z(T), x(T), w)}

is minimized

(1.31 )

or with the dynamics given by (1.29). Since in (1.28) or (1.29), the variables

z(t)

are functions of

(x,w),

the functionsgjare also implicit functions of

(x,w),

i.e. we can rewrite problem (1.31) in terms of functions

/;(x,w) =gj(z(x,w),x,w),

the stochastic dynamic problem (1.31) is then reduced to a stochastic opti-mization problem of type (1.10). The implicit form of the objective and the constraints of this problem requires a special calculus for evaluating these func-tions and their derivatives, but it does not alter the general solution strategies for stochastic programming problems.

The two-stage recourse model allows for a recourse decision ythat is based on (the first stage decision x and) the result of observations. The following simple example should be useful in the development of a dynamical version of that model. Suppose we are interested in the design of an optimal trajectory to be followed, in the future, by a number of systems that have a variety of (dynamical) characteristics. For instance, we are interested in building a road between two fixed points (see Figure 1.4) at minimum total cost taking into account, however, certain safety requirements. To compute the total cost we take into account not just the construction costs, but also the cost of running the vehicles on this road.

For a fixed feasible trajectory

z

= (z(O), z(I), ... , z(T)),

and a (dynamical) system whose characteristics are identified by a parameter w E 0, the dynamics are given by the equations, for t = 0, ... ,T - 1, and

.az(t)

=

z(t +

^{1) -}

^z(t),

.az(t)

=

h(t,z(t),y(t),w),

(1.32)

Stochastic Programming, An Introduction

z(O)

Ground level

17

IzIT) I I I I

o t=1 T

Figure 1.40 Road design problem.

and

z(O)

= zo,z(T) =

^ZT.

Here the variablet records position (between 0 andT). The variables

y = (y(O),y(I), ... ,y(T))

are the control variables at t

=

0,1, ... ,T that determine the way a dynamical system of type w will be controlled when following the trajectory z from 0 to T. The choice of the z·trajectory is subject to certain restrictions, that include safety considerations, such as

laz(t)l:5 d1,laz(t) - az(t -1)1:5 d

2 , (1.33) i.e. the first two derivatives cannot exceed certain prescribed levels.

For a specific systemwE 0, and a fixed trajectory z,the optimal control actions (recourse)

y(z,w) = (y(O,z,w),y(I,z,w), ... ,y(T,z,w))

is determined by minimizing the loss function

go(z(O), y(O), ... , z(T - 1), y(T - 1), z(T), w)

subject to the system's equations (1.32) and possibly some constraints on y. If P is the a prioridistribution of the systems parameters, the problem is to find a trajectory (road design) z that minimizes in the average the loss function, i.e.

Fo(z) = E{go(z(O),y(O,z,w), ... ,z(T -I),y(T -I,z,w),z(T),wn

(1.34)

18 Stocha,tic Optimization Problem, subject to constraints oftype (1.33).

In this problem the observation takes place in one step only. We have amalgamated allfuture observations that will actually occur at different time periods in a single collection of possible environments (events). There are situ-ations when whas the structure

w

=

(w(O),w(l), ... ,wIT))

and the observations take place in T steps. As an important example of such a class, let us consider the following problem: the long term decision z =

(z(O),z(l), ... ,z(T))

and the corrective recourse actions

y

=

(y(O),y(l), ... ,

y(T)) must satisfy the linear system of equations:

Aooz(O)

where the matrices

Au" Bt

and the vectors

hIt)

are random, i.e. depend onw.

The sequence

z

=

(z(O), ... , zIT))

must be chosen beforeanyinformation about the values of the random coefficients can be collected. At timet = 0, ... ,T, the actual values of the matrices, and vectors,

Au-, k

= 0, ...

,tjBf, h(t),d(tr

are revealed, and we adapt to the existing situation by choosing a corrective action

y(t,z,w)

such that

t

y(t,z,w) E

argmin[d(t)yIBty~

hIt) - 'LAtkZ(k),y

_~

0].

k=O

The problem is to find

z

= (z(O), ... ,z(T))that minimizes

T

Fo(z)

=

'Llc(t)z(t) +E{d(t)y(t,z,w)}]

t=o

(1.35) subject to z(O) _~O, ... ,z(T) _~0.

In the functional (1.35), or (1.34), the dependence of

y(t,z,w)

on

z

is nonlinear, thus these functions do not possess the separability properties nee·

essary to allow direct use of the conventional recursive equations of dynamic programming. For problem (1.31), these equations can be derived, provided the functions gi,i = 0, ... , m, have certain specific properties. There are, how-ever, two major obstacles to the use of such recursive equations in the stochastic

Stochastic Programming, An Introduction 19 case: the tremendous increase of the dimensionality, and again, the more serious problem created by the need of computing mathematical expectations.

For example, consider the dynamic system described by the system of equations (1.28). Let us ignore all constraints except

x(t)

^E

X(t),

for

t

=

0,1, ... ,T. Suppose also that

w

=

(w(O),w(I), ... ,w(T))

where

w(t)

only depends on the past, i.e. is independent of

w(t +

^{1), ...}

^,w(T).

Since the minimization of

Fo(x)

⁼

E{go(z(O),x(O), ... , z(T),x(T),w)}

with respect to x can then be written as:

min min ... minE{go}

z(O) z(l)

z(T)

and ifgo is separable, i.e. can be expressed as

T-l

go :=

L

god~z(t),x(t),w(t))

+ gOT(z(t),w(T))

t=o

then

minFo (x) = minE{goo(~z(O),x(O),w(O))}

+

minE{god~z(I),

x(l) ,w(I))}

z z(O) z(l)

+ ... + min

E{go,T

-1(~z(T

-1),x(T -1),w(T - 1))}+

z{T-I)

+ E{gOT(Z(T),w(T))}

Recall that here, notwithstanding its sequential structure, the vectorwis to be revealed in one global observation. Rewriting this in backward recursive form yields the Bellman equations:

vdzt}

=

min[E{got (h(t, Zt, x,w(t)), x,w (t))

+ Vt+dZt + h(t, Zt, X,w (t)))}lx

E

X(t)]

for t = 0, ...,T - 1, and

Vr(ZT)

=

E{gOr(ZT,W(T))},

(1.36)

(1.37) where Vt is the value function (optimalloss.to.go) from time t on, given state Zt at time

t,

that in turn depends on x(O),

x(I), ... , x(t -

1).

To be able to utilize this recursion, reducing ultimately the problem to:

find xE X(O) eRn such thatVo is minimized, where

Vo =

E{goo(h(O,zo, x,w(O)),x,w(O)) +

^VI(zo

+ h(O,zo, x,w(O)))},

20 Stochastic Optimization Problems we must be able to compute the mathematical expectations

E{got{.D.z(t) ,

x,

w(t))}

as a function of the intermediate solutions x(O), ... ,x(t -1), that detennine

.D.z(t) ,and this is only possible in special cases. The main goal in the de·

velopment of solution procedures for stochastic programming problems is the development of appropriate computational tools that precisely overcome such difficulties.

A much more difficult situation may occur in the (full) multistage version of the recourse model where observation of some of the environment takes place at each stage of the decision process, at which time (taking into account the new information collected) a new recourse action is taken. The whole process looks like a sequence of alternating: decision-observation· ...-observation-decision.

Let x be the decision at stage k = 0, which may itself be split into a sequence x(O), .. _, x(N),each

x(k)

corresponding to that component ofxthat enters into play at stage k, similar to the dynamical version of the two-stage model introduced earlier. Consider now a sequence

y=

(y(O),y(I), ... ,y(N))

of recourse decisions (adaptive actions, corrections),

y(k)

being associated specif·

ically to stage k. Let

Bir := information set at stage k,

consisting of past measurements and observations, thusB ir ^C B_Ir+I '

The multistage recourse problemis find xE X C Rⁿ

suchthat !OJ(x)~O, i=I, ... ,mo,

E{hj(x,y(I),w) IBtl

~0, i = 1, ...

,ml'

(1.38)

E{/N;{X, y(I), ... , y(N),w)IBN}

~ 0, i = 1, ...

,mN, y(k)

E

Y(k), k

= 1, ...

,N,

and F

o(x)

is minimized where

Fo(x)

⁼ EBo{min

EBI {...

min

EBN-I {/(x,y(I), ... , y(N),w)).}}

y(l) y(N-l)

Ifthe decision x affects only the initial stagek= 0, we can obtain recursive equa-tions similar to (1.36) - (1.37) except that expectationEmust be replaced by the conditional expectationsEBt ,which in no way simplifies the numerical problem of finding a solution. In the more general case whenx= (x(O), x(I), ... , x(N)),

one can still write down recursion formulas but of such (numerical) complexity that all hope of solving this class of problems by means of these formulas must quickly be abandoned.

Stochastic Programming, An Introduction

1.7 Solving the Deterministic: Equivalent Problem

Im Dokument Stochastic quasigradient methods. Numerical techniques for stochastic optimization (Seite 31-37)