Lipschitzian Stability in Optimization: The Role of Nonsmooth Analysis

Working Paper

IJPSCHITZIAN STABILITY IN OPTJMIZATION:

THE ROLE OF NONSMOOTH ANALYSIS

R. T. RockqfeLLar

September 1986 WP-86-46

International Institute for Applied Systems Analysis

A-2361 Laxenburg, Austria

NOT FOR QUOTATION

WITHOUT THE PERMISSION OF THE AUTHOR

LZPSCHITZIAN STABILITY IN OPTIMIZATION:

THE ROLE OF NONSMOOTH ANALYSIS

September 1986 WP-86-46

Working Papers are interim r e p o r t s on work of t h e International Institute f o r Applied Systems Analysis and have r e c e i v e d only limited review. Views or opinions e x p r e s s e d h e r e i n d o not necessarily r e p r e s e n t those of t h e Institute or of i t s National Member Organizations.

INTERNATIONAL INSTITUTE FOR APPLIED SYSTEMS ANALYSIS A-2361 Laxenburg, Austria

FOREWORD

This p a p e r i s t h e s u r v e y of r e c e n t developments in nonsmooth analysis and i t s applications t o optimization problems. A t f i r s t t h e motivations of nonsmooth analysis are discussed and concepts of derivative f o r Lipschitzian and lower sem- icontinuous functions are presented. Then t h e concepts of nonsmooth analysis are used t o g e t sensitivity r e s u l t s f o r g e n e r a l nonlinear programming problems and t o c l a r i f y t h e i n t e r p r e t a t i o n of t h e Lagrange multipliers. Promising directions of f u r t h e r r e s e a r c h are indicated.

A. Kurzhanski Chairman System and Decision Sciences Program

CONTENTS

Abstract Introduction

1 Origins of Subgradient Ideas

2 Lagrange Multipliers and Sensitivity 3 Stability of Constraint Systems References

7HPSCEIEZIFL,Y

S Y & L ? Z Y

LN

O P T E I Z A T I 3 K :

TEE ROLE OF NONSKO3TH Ahr&YSiS

R.T. Rockafellar'

The motivations of nonsmooth znzlysis a r e ciiscussed. Appiications a r e given t o Line sensitivity of optimal vaiues, t h e i n t e r p r e t a t i o n of i a g r a n g e multipliers, a n d t h e s t a b i i i t y of c o n s t r a i n t systems uncier p e r t u r b n t i o n .

it h a s b e e n r e c o g n i z e d f o r some time t h a t t h e tools of c i a s s i c a l anniysis a r e n o t a d e q u a t e f o r a s a t i s f a c t o r y t r e a t m e n t of problems of optimization. Tnese toois work f o r t h e c h a r a c t e r i z a t i o r , of locaily optimal solutions t o problems w h e r e a smooth (i.e.

continuously d i f f e r e n t i a b l e ) function is minimized o r maximized. s u b j e c t t o finiteiy mar,)' smooth equality c o n s t r a i n t s . They a i s o s e r v e in t h e study of p e r t u r b a t i o n s of such con- s t r a i n t s , namely t h r o u g h t h e implicit function t h e o r e m a n d i t s consequences. A s soon as inequality c o n s t r a i n t s a r e e n c o u n t e r e d , however, t h e y begin t o fail. One-sided d e r i v a t i v e conditions start t o r e p l a c e two-sided conditions. Tangent c o n e s r e p i a c e t a n g e n t s u b s p a c e s . Convexity a n d convexification e m e r g e as more n a t u r a l t h a n l i n e a r - ity a n d linearization.

i n problems w h e r e inequality c o n s t r a i n t s actualiy predominate o v e r equations, as i s t y p i c a l in most modern a p p l i c a t i o n s of optimization, a qualitative c n a n g e o c c u r s . Ko l o n g e r i s t h e r e a n y simple way of recognizing which c o n s t r a i n t s a r e a c t i v e in a neigh- borhood of a given point of tine f e a s i b l e set, s u c h as t h e r e would b e if t h e s e t w e r e z c u b e o r simplex, s a y . The b o u n d a r y of t h e f e a s i b l e set d e f i e s e a s y d e s c r i p t i o n a n d may b e s t b e t h o u g h t of as a nonsmooth h y p e r s u r f a c e . I t d o e s not t a ~ e long t o r e a i i z e t o o t h a t t h e g r a p h s of many of t h e o b j e c t i v e functions which n a t u r a l l y a r i s e a r e nonsmooth in a simirar way. Tnis i s t h e motivation f o r much of t h e e f f o r t t h a t n a s gone i n t o

* Research supported in port b y a grant f r o m t h e Nationai Science Foundation a t t h e U n i v e r s i t y oi Washington, S e a t t i e .

introducing a n d deveioping v a r i o u s c o n c e p t s of "tangent cone!', "normal cone", "direc- tional d e r i v a t i v e " a n d "generaiized g r a d i e ~ t " . Tnese c o n c e p t s n a v e c n z n g e d t h e f a c e of optimization t h e o r y and given b i r t h t o a new s u b j e c t , nonsxro~th analysis, which is affecting o t h e r areas of matnematics as weil.

An i m p o r t a n t aim of nonsmooth analysis i s the formuiztion of generaiized neces- s a r y o r s u f f i c i e n t conditions f o r optimality. This in t u r n r e c e i v e s impetus frorr.

r e s e a r c h in numerical methods of optimization t h z t invoive nonsinooth functions gen- e r a t e d by decomposition, e x a c t penalty r e p r e s e n t a t i o n s , a n d t h e iike. The idea essen- tiaily i s t o provide tests t h a t e i t h e r establish ( n e a r ) optimaiity ( p e r h a p s s t z t i o n a r i t y ) of t h e point a l r e a d y a t t a i n e d o r g e n e r z t e a f e a s i b i e d i r e c t i o n of improvement f o r mov- ing t o a b e t t e r point.

Sonsmooth a n z i y s i s z i s o h a s o t h e r i m p o r t a n t aims, however, which shouid not b e overlooked. Tnese include t h e s t u d y of sensitivity a n d s t a b i l i t y with r e s p e c t t o p e r t u r - bations of o b j e c t i v e a n d c o n s t r a i n t s . In a n optimization problem t h a t d e p e n e s on a p a r a m e t e r v e c t o r v , now t o v a r i a t i o n s in ' ~ i a f f e c t t h e optimal value, t h e optimzl soln- tion set, a n d t h e f e a s i b l e solution s e t ? Can anything b e s a i d a b o u t rates of c h a n g e ?

This i s w h e r e Lipschitzian p r o p e r t i e s t a k e on s p e c i a l significance. They a r e i n t e r m e d i a t e between continuity and d i f f e r e n t i a b i l i t y a n d c o r r e s p o n z t o bounds on possible r a t e s of c h a n g e , r a t n e r t h a n rates themselves, which may not e x i s t , at l e a s t in t h e c l a s s i c a l s e n s e . Like convexity p r o p e r t i e s t h e y c a n b e passed along t h r o u g h v a r i - ous c o n s t r u c t i o n s w h e r e t r u e d i f f e r e n t i a b i l i t y , e v e n if one-sided, would b e i o s t . F u r t h - e r m o r e , t h e y c a n b e formulated in geometric t e r m s t h a t s u i t t h e s t u d y multifunctions (set-valued mappings), a s u b j e c t of g r e a t importance in optimization t n e o r y b u t f o r which c l a s s i c a l notions a r e aimost e n t i r e l y lacking.

I t i s in t h i s l i g h t t h a t t h e d i r e c t i o n a l d e r i v a t i v e s and s u b g r a d i e n t s introduced by F.H. C l a r k e

[I]

[2] snould b e judged. C l a r k e ' s t h e o r y empnasizes Lipscnitzian p r o p e r - t i e s and s t u r d i l y combines convex anaiysis and c i a s s i c a l smooth a n a i y s i s in a singie framework. A t t h e p r e s e n t s t a g e of development, thanits t o t h e e f f o r t s of many indivi- duals, i t h a s a l r e a d y h a d s t r o n g e f f e c t s on almost e v e r y area of optimization, from non- l i n e a r programming t o t h e c a i c u l u s of v a r i a t i o n s , an^ a l s o on mathematicai questions beyond t h e domain of optimization p e r s e .

This is n o t t o s a y , nowever, t h z t C l a r ~ e ' s d e r i v a t i v e s and s u b g r a d i e n t s are t h e only o n e s t h a t h e n c e f o r t h need t o b e c o n s i d e r e z . S p e c i a l s i t u a t i o n s c e r t a i n l y do r e q u i r e s p e c i a i insignts. i n p a r t i c u i a r , t h e r e a r e cases w h e r e s p e c i a l one-sided f i r s t and second d e r i v a t i v e s t h a t a r e more finely tuned t h a n C i a r k e ' s are worth introducing.

Significant and useful r e s u i t s c a n be oblained. ir, such manner. But s u c h r e s u l t s a r e likely t o b e r e l a t i v e l y limited in s c o p e .

I ; h e power anC g e n e r a i i t y of t h e kind of nonsmooth anaiysis t h a t i s b a s e d or, C i a r k e ' s i i e a s c a n b e c r e d i t e d t o t h e foliowing f e a t u r e s , in summary:

( a ) Applicability t o a huge c l a s s of functions and o t h e r o b j e c t s , such as sets acd.

mxitifunctions.

(b) Emphasis on geometric c o n s t r u c t i o n s and. i n t e r p r e t a t i o n s .

( c ) Reduction t o c l a s s i c a l analysis in t h e p r e s e n c e of smoot'nness a n d t o convex analysis in t h e p r e s e n c e of convexity.

(d) Unified formulation of optimality conditions f o r a wide v a r i e t y of probiems.

( e ) Comprehensive calculus of s u b g r a d i e n t s a n d normal v e c t o r s which makes pos- s i b l e a n e f f e c t i v e specialization t o p a r t i c u i a r c a s e s .

(f) Coverage of sensitivity and s t a b i l i t y questions and t h e i r r e l a t i o n s h i p t o L a g r a n g e multipliers.

(g) Focus on iocal p r o p e r t i e s of a "uniform" c h a r a c t e r , which are l e s s likely t o b e u p s e t o y slight p e r t u r b a t i o n s , f o r i n s t a n c e in t h e s t u d y of d i r e c t i o n s of d e s c e n t .

(h) Versatility in infinite as well as finite-dimensional s p a c e s a n d in t r e a t i n g t h e

. .

i n t e g r a l functionals and d i f f e r e n t i a i inciusions t h a t a r i s e in optima: c o n r o i , s t o c h a s t i c programming, a n d e i s e w n e r e .

In t h i s p a p e r we a i r , at putting t h i s t'neory in a n a t u r a l p e r s p e c t i v e , f i r s t by dis- cussing i t s foundations in analysis and geometry a n d t h e way t h a t Lipschitzian p r o p e r - t i e s come t o occupy t h e s t a g e . Tnen we s u r v e y t h e r e s u l t s t h a t h a v e b e e n o b t a i n e d r e c e n t l y on sensitivity a n d stability. Such r e s u i t s a r e n o t y e t famiiiar t o many r e s e a r c n e r s who c o n c e n t r a t e o n optimality c o n l i t i o n s and. t h e i r u s e in aigorithms.

N e v e r t h e l e s s t h e y s a y much t h a t b e a r s on numerical m a t t e r s , and t h e y d e m o n s t r a t e well t h e s o r t of challenge t h a t nonsmooth a n a i y s i s i s now a b l e t o meet.

1. CXIGZ<S OF SUBGRADIEhT DEBS

i n o r d e r t o gain a foothold on t h i s new t e r r i t o r y , i t i s b e s t t o begin by thinking a b o u t functions f : Rn

+R

t h a t a r e not n e c e s s a r i l y smooth S x t h a v e s t r o n g one-sided.

d i r e c t i o n a l d e r i v a t i v e s in t h e s e n s e of

Examples a r e ( f i ~ i i e ) convex functions [ 3 ] an6 subsmoctiz functions, t h e i a t t e r being by definition re;reser;table ioca:iy e s

w n e r e S is a compact s p a c e (e.g., a f i n i t e , d i s c r e t e index s e t ) and ff,

1

s ES

1

i s a family of smooth functions whose vaiues and d e r i v a t i v e s depend continuously on s znd z jointly. Subsmooth functions w e r e i n t r o d u c e d in [4]; a l l smooth functions a n d a l l finite convex functions on

R~

a r e in p a r t i c u l a r subsmooth.

The formula given h e r e f o r f ' ( z ; A ) d i f f e r s from t h e more common one in t h e l i t e r a t u r e , w h e r e t h e iimit A'-A i s omitted (weak one-sided d i r e c t i o n a l d e r i v a t i v e ) . I t corresponcis in s p i r i t t o t r u e ( s t r o n g ) differentiability r a t h e r t h a n weak d i f f e r e n t i a - bility. Indeed, uncier t h e assumption t h a t f ' ( z , h ) e x i s t s f o r a l l h ( a s in (1.1)), one h a s f d i f f e r e n t i a b l e at z if and only if f ' ( z ; h ) i s l i n e a r in A. Then t h e one-sided limit

t

&O i s a c t u a l l y r e a l i z a b l e as a two-sided Limit t -9.

The c l a s s i c a l c o n c e p t of g r a d i e n t a r i s e s from t h e duality between l i n e a r functions on

R n

a n c v e c t o r s in

R n .

To s a y t h a t f ' ( z ; h ) i s l i n e a r in A i s t o s a y t h a t t h e r e i s a v e c t o r y E

R n

with

f ' ( z ; h )

=

y .A f o r a l l A. (1.3)

Tnis y i s c a l l e d t h e g r a d i e n t of f at z a n d i s denoted by Of ( 2 ) .

In a similar way t h e modern c o n c e p t of s u b g r a d i e n t a r i s e s f r o m t h e duality between s u b l i n e a r functions o n

R n

and convex s u b s e t s in

R n .

^A function L i s said t o b e s u b l i n e a r if i t s a t i s f i e s

when Al 2 0 ,

. .

,A, 2 0.

I t i s known from convex anaiysis [3, $131 t h a t t h e finite s u b l i n e a r functions L on

R~

are p r e c i s e l y t h e s u p p o r t functions of t h e nonernpty compact s u b s e t s Y of

R n :

e a c h L c o r r e s p o n d s t o a unique I' by t h e formuia

L ( h ) = m a x y . h f o r a l l A.

Y EY

(1.5)

Linearity c a n b e identified with t h e case w h e r e 1' consists of just a single v e c t o r y . I t t u r n s o u t t h a t when f is c o w e x , and more g e n e r a l l y when f is subsmooth [4], t h e d e r i v a t i v e f ' ( z , A ) i s always s u b l i n e a r in A. E e n c e t h e r e is a n o n e n p t y compact s u b s e t Y of

R"

: uniqueiy d e t e r m i n e e , such t h a t

f f ( z ; h ) = m a x p . h f o r a l l h. ( I . 6)

y EY

This s e t i' i s denoted by

af

( z ) , a n d i t s elements y a r e called s u b g r a d i e n t s of f a t z . With r e s p e c t t o any l o c a i r e p r e s e n t a t i o n (1.4), o n e h a s

Y = c o t V f s ( z ) : s - { , w n e r e S , = a r g m a x f , ( z ) ( I . 7)

s € s

( c o

=

convex hull), b u t t h e set Y

= Zf

( z ) i s of c o u r s e by i t s definition independent of t h e r e p r e s e n t a t i o n used.

In t h e case of f convex [3, $231 o n e c a n define s u b g r a d i e n t s at z equivalently as t h e v e c t o r s y such t h a t

f ( 2 ' ) 2 f ( z )

+

y . ( z f - z ) f o r a l i z'. (1.8)

F o r f subsmooth t h i s g e n e r a l i z e s t o

f ( z f ) 2 f ( z )

+

~ . ( z ' z )

+

o ( j 2'-z i i ), ( I . 9)

b u t caution must b e e x e r c i s e d h e r e a b o u t f u r t h e r g e n e r a l i z a t i o n t o functions f t h z t are not subsmooth. Although t h e v e c t o r s y satisfying (1.9) d o always form a ciosed convex s e t I' at z , r e g a r d l e s s of t h e n a t u r e of f

,

t h i s set

Y

d o e s not yieid a n e x t e n s i o n of formula (1.5), n o r d o e s i t c o r r e s p o n d in g e n e r a l t o a r o b u s t c o n c e p t of d i r e c t i o n a l d e r i v a t i v e t h a t c a n be used as a s u b s t i t u t e f o r f ' ( z ; h ) in (1.6). F o r a number of y e a r s , t h i s i s wnere s u b g r a d i e n t t h e o r y came t o a halt.

A way a r o u n d t h e impasse w a s d i s c o v e r e d by C l a r k e in h i s t h e s i s in 1973. C l a r k e took up t h e study of functions f : Rn + R t n a t a r e l o c a l l y L t p s c h i t z i a n

:n

t h e s e n s e of t h e d i f f e r e n c e quotient

being bounded on some neighborhood, of e a c h point z . This c l a s s of h n c t i o n s i s of i n t r i n s i c value f o r s e v e r a l r e a s o n s . F i r s t , i t includes a l l subsmooth functions and. co2- sequently a l l smooth functions a n d a!! finite convex functions; i t a l s o inciudes a l l f i n i t e c o n c a v e functions a n d a l l f i n i t e saddie functions (which a r e convex in o n e v e c i o r a r g u - n e s t and. c o n c a v e in a n o t h e r ; see [3, $351). Second, i t is p r e s e r v e d u n d e r taking l i n e a r combinations, pointwise maxima and minima of coliections of functions (with c e r t z i n mild assumptions), i n t e g r a t i o n nnd o t h e r o p e r a t i o n s of ob-.:ious i m p o r t a n c e in optimiza- tion. ThirC, i t e x h i b i t s p r o p e r t i e s that a r e closely r e l a t e d t o differentiabiLity. T h e loczl b o ; l n d e ~ n e s s of t h e d i f f e r e n c e quotient (1.12) is such z p r o p e r t y i t s e l f . In f a c t when f i s iocoi!:: Lipschitzinc, t h e gra2ier.t Cf ( z ) e x i s t s f o r aii S c t z negiigi'zie set of points z ir. R n ( t h e c i a s s i c a l theorern of Xzdemacher, s e e

51).

C l a r k e d i s c o v e r e d t h a t wnen f is ioczlly Lipscnitzian, t h e s p e c i a l d e r i v a t i v e expressior.

i s always a f i n i t e s u b i i n e a r function of h . Hence t h e r e e x i s t s a unique nonempty com- p a c t convex set

Y

s u c h t h a t

f " ( ~ ; h ) = m a x y . h f o r a l l h.

Y EY

Moreover

Q " ( x ; h )

=

f ' ( z ; h ) f o r a:i h when f i s subsmooth. (1.13)

Thus in denoting t h i s s e t 'I' by

af

^{( x )} and cailing its elements s u b g r a d i e n t s , o n e a r r i v e s at c n a t u r a l e x t e n s i o n of nonsmooth a n a l y s i s t o t h e c i a s s of a l l locally Lipschitzizn functions. Many powerful f o r m ~ l a s z n 6 r u i e s h e v e b e e n e s t a b l i s h e d f o r caiculating or estimating L3f ( x ) in t h i s b r o a d c o n t e x t , b u t i t is n o t o u r aim t o go into them h e r e ; see [2] a n d [ S ] , f o r instance.

I t should b e mentioned t h a t C l a r k e himself did not i n c o r p o r a t e t h e limit h f + n i n t o t h e definition of f " ( z ; h ) , b u t b e c a u s e of t h e Lipschitzian p r o p e r t y t h e value obtained f o r f " ( z ; h ) i s t h e same e i t h e r way. By writing t h e formuia with h '+n o n e is a b l e t o s e e more c l e a r l y t h e r e l a t i o n s h i p between f " ( x ; h ) a n d Q ' ( x ; h ) a n d a l s o t o p r e p a r e t n e ground f o r f u r t h e r e x t e n s i o n s t o functions Q t h a t a r e mereiy lower sem- icontinuous r a t h e r t h a n Lipschitzian. (For s u c h functions o n e w r i t e s x ' ^-f z in p i a c e of x'

-

^x t o i n d i c a t e t h a t x i s t o b e a p p r o a c h e d by x' only in s u c h a way t h a t f ( s f ) ^-+ Q ( x ) . More will b e said a b o u t t h i s i a t e r . )

Some p e o p l e , naving gone aiong with t h e developments up until t h i s point, begin t o balk a t t h e " c o a r s e " n a t x r e of t h e C l a r k e d e r i v a t i v e f " ( z ; i L ) in certair; c a s e s w h e r e f is not subsmooth and n e v e r t h e l e s s i s being minimized. F o r exampie, if

/ I I

f ( x )

= -

z i 1 l2 one h z s f O(S;h j

=

^{! h} I, w h e r e a s f ' ( C ; h ) e x i s t s t o o b u t

I I

f '(9;i;)

= -

h . Thzs f ' r e v e a i s t h a t e v e r y h +O gives a d i r e c t i o n of d e s c e n t f r o x 0 , in t h e s e n s e of yie!tin,n f '(C;n)<O, b u t f " r e v e z i s no such thing, inasmuch as f " ( 3 ; h )

>

3. Becznse of t h i s i t is f e a r e d t h a t f O does not embody zs muck i n f o ~ m a t i o r : as f ' anci t h e r e f o r e may n o t b e eaiire'ly s u i t a b l e f o r t h e statement of n e c e s s a r y condi- tions f o r a minimur,, i e t alone for e m ~ i n y m e n t ir; aigoritnms of d e s c s n t .

Clearly f " cannot repLace f ' in e v e r y situation where t h e two may d i f f e r , n o r k a s t h i s e v e r been suggested. E u t even in f a c e of this c a v e a t t h e r e a r e zrguments t o b e made in f z v o r of ^{f O} t h a t may heip t o iliumilnate its n a t u r e a n e t h e sxpporting motiva- tion. The Ciarke derivative ^{f O} i s o r i e n t e d towards minimizztion probiems, in c o n t r n s t t o f ', which is n e u t r a i between minimization and mzxinizztior,. 1;: n?ditior,, it.

emphasizes a c e r t a i n uniformity. A v e c t o r

n

witk f " ( x ; h )

<

C provides a d e s c e n t direction in a s t r o n g s t a b l e sense: t h e r e i s a n E

>

⁰ such t h a t f o r all z ' n e a r z , h' n e a r h , and positive t n e a r 0 , one has

f ( z ' f t h ' )

<

f ( z ' )

- t ~ .

A v e c t o r h with f '(z;h)

<

0 , or, t h e o t h e r hand, provides descent oniy from x ; at points x ' a r b i t r a r i l y n e a r t o z i t may give a direction of a s c e n t instead. This instabil- ity is not without numericai consequences, since z might b e r e p i a c e d by z' b e t o round-of f

.

An algorithm t h a t r e l i e d on finding a n h with f ' ( x ; h )

<

0 in c a s e s where f " ( x ; h ) 2 0 f o r ail h (such an x is said t o b e s u b s t a t i o n a r y point) seems unlikely t o b e v e r y r o b u s t . Anyway, i t must b e realized. t h a t in executing a method of descent t h e r e is v e r y little c h a n c e of actually a r r i v i n g aiong t h e way at a point x t h a t is subs- tationary but not a local minimizer. One is easily convinced from examples t h a t such 2

mishap can oniy be t h e consequence of an unfortunate cnoice of t h e s t a r t i n g point and d i s a p p e a r s under t h e slightest p e r t u r b a t i o n . The situation resembles t h a t of cycling in t h e simplex method.

F u r t h e r m o r e i t must b e understood t h a t because of t h e orientation of t h e defini- tion of f o towards minimization, t h e r e is no justice in holding t h e notion of substa- tionarity up t o any i n t e r p r e t a t i o n o t h e r t h a n t h e following: a substationary point is e i t h e r a point where a locai m i n i m u m is attained o r one where p r o g r e s s towards a local minimum is "confusedv. Sometimes, f o r instance, one n e a r s cited a s a failing of f o t h a t f ' is a b l e t o distinguish between a iocal minimum and a local maximum in having f ' ( x ; h ) ² 0 f o r all h in tine f i r s t c a s e , but f ' ( x ; h )

s

0 f o r ail ir ir, t h e second, whereas f " ( z ; h ) r 0 f o r all h in both cases. But t h i s is unfair. A one-sided orientation in nonsmooth anaiysis is merely a r e f l e c t i o n of t h e f a c t t h a t in virtually all applications of optimization, t h e r e is unambiguous i n t e r e s t in e i t h e r maximization o r minimization, but not both. For t h e o r e t i c a l p u r p o s e s i t might as well b e minimization.

Certainly t h e idea t h a t a f i r s t - o r d e r concept of derivative, such as we a r e dealing with h e r e , i s obliged t o provide conditions t h a t distinguish effectively between 2 Local minimum and a local maximum i s out of line f o r o t h e r reasons. Classical anaiysis maites no attempt in t h a t d i r e c t i o n , without second derivatives. Presumably: second

d e r i v a t i v e c o n c e p t s in nonsmooth analysis will eventually f u r n i s h t h e a p p r o p r i a t e ais- tinctions, c f . Chaney

[?I.

A final n o t e o n t h e question of f a v e r s u s f' i s t h e r e m i n d e r t h a t f " ( x ; h ) i s defined f o r any locally Lipschitzian funciion f and even more g e n e r a l l y , w h e r e a s f ' ( x ; h ) i s only defined f o r functions f in a n a r r o w e r c l a s s .

An important goal of nonsmooth analysis i s not only t o make full u s e of Lipschitz continuity when i t i s p r e s e n t , b u t a l s o t o p r o v i d e c r i t e r i a f o r Lipschitz continuity in c a s e s where i t c a n n o t b e known a p r i o r i , along with c o r r e s p o n d i n g e s t i m a t e s f o r t h e local Lipschitz c o n s t a n t . F o r t h i s p u r p o s e , i t i s n e c e s s a r y t o e x t e n d s u b g r a d i e n t t h e o r y t o functions t h a t might n o t b e locally Lipschitzian o r e v e n continuous e v e r y - w h e r e , b u t merely lower semicontinuous. Fundamental examples of s u c h functions in optimization are t h e so-called m a r g i n a l functions, which give t h e minimum value in a p a r a m e t e r i z e d problem as a function of tine p a r a m e t e r s . Such functions c a n e v e n t a k e on cm.

E x p e r i e n c e with convex a n a l y s i s a n d i t s a p p l i c a t i o n s shows f u r t h e r t h e d e s i r a b i l - ity of being a b l e t o treat t h e i n d i c a t o r functions of sets, which p l a y a n e s s e n t i a l r o i e iri t h e p a s s a g e between analysis a n d geometry.

In f a c t , t h e i d e a s t h a t h a v e b e e n d e s c r i b e d s o f a r c a n b e extended. in a powerful, consistent manner t o t h e c l a s s of a l l lower semicontinuous functions f :

R n -, R ,

w h e r e

R - =

[-=,-I (extended r e a l number system). T h e r e are two compiementary ways of doing t h i s , with t h e same r e s u l t . In t h e continuation of t h e a n a l y t i c a p p r o a c h w e h a v e b e e n following until now, a m o r e s u b t l e d i r e c t i o n a i d e r i v a t i v e formula

i s introduced and shown t o a g r e e with f " ( z ; h ) whenever f i s locally Lipschitzian a n d indeed whenever f " ( z ; h ) (in t h e e x t e n d e d definition with x ' J ~ Z , as mentioned ear- l i e r ) i s not

+-.

Moreover f ' ( z ; h ) i s p r o v e d always t o b e a lower semicontinuous, sub- l i n e a r function of h (extended-real-valued). From convex analysis, t h e n , i t follows t h a t e i t h e r f '(z;O)

= --

o r t h e r e i s a nonempty closed convex set Y CRn, uniquely determined, witin

f ' ( z ; h )

=

s u y - h f o r a l l h .

't

Y E

T l a i s ' i s t h e a p p r o a c h followed in R o c k a f e l l a r [8], [9]. One t h e n a r r i v e s at t h e c o r r e s p o n d i n g geometric c o n c e p t s by taking f t o b e t h e i n d i c a t o r 6C of a cioseG set C . F o r any z E C , t h e functior, h 4 6 ; ( z l k ) is :tself t'ne i n d i c a t o r of z c e r t a i n ciosed s e t

T C ( z ) which h a p p e n s aiways t o b e a convex c o n e ; t h i s is t h e Clarite t a n g e n t c o n e t o C at x . The suhgraciient set

on t h e o t h e r h a n d , is a ciosed convex s e t t o o , t h e C l a r k e n o r m a l c o n e t o C t o x . The two c o n e s a r e p o l a r t o e a c h o t h e r :

In a more g e o m e t r i c a p p r o a c h t o t h e d e s i r e d extension, t h e t a n g e n t c o n e T C ( z ) a n d normal c o n e NC(z) c a n f i r s t b e defined in a d i r e c t manner t h a t accorcis with t h e p o l a r i t y r e i a t i o n s (1.16). Then f o r a n a r b i t r a r y lower semicontinuous function f :

R" +R -

a n d point x at which f i s ? k i t e , o n e c a n f o c u s on TE(z, f ( z ) ) and NE(x ,f ( x ) ) , w h e r e E i s t h e e p i g r a p h of f (a closed s u b s e t of

R n

+I). The c o n e TE ( z , f ( x ) ) i s itself t h e e p i g r a p h of a c e r t a i n function, nameiy t h e s u b d e r i v a t i v e h '4

f '(x ;h), w h e r e a s t h e c o n e NE ( x , f ( z ) ) p r o v i d e s t h e subgradients:

Tine p o l a r i t y between TE (x , f ( z )) a n d NE ( z , f ( z )) yieids t h e subderivative-subgraciient r e l a t i o n (1.14). ( C l a r k e ' s original extension of i3f t o lower semicontinuous functions

[I]

followed t h i s g e o m e t r i c a p p r o a c h in defining normal c o n e s d i r e c t l y a n d t h e n invok- ing (1.17) as a definition f o r s u b g r a d i e n t s . He did n o t f o c u s much o n t a n g e n t c o n e s , however, o r p u r s u e t h e idea t h a t TE (x , f (x )) might c o r r e s p o n d t o 2 r e i a t e d c o n c e p t of d i r e c t i o n a l d e r i v a t i v e . )

The d e t a i l s of t h e s e equivaient forms of extensior, need n o t occupy u s h e r e . The main thing t o u n d e r s t a n d i s t h a t t h e y yield a b a s i c c r i t e r i o n f o r Lipschitzian con- tinuity, as follows.

TEEOREM 1 ( R o c k a f e l i a r [lo]). For a Lower s e m i c o n t i n u o u s f u n c t i o n f : Rn ->h'

-

actuaLLy t o be L i p s c n i t z i a n o n some n e i g h b o r h o o d of t n e p o i n t z , i t i s sufficierct ( a s weLL as n e c e s s a r y ) t n a t t n e s z b g r a d i e n t s e t

af

(z) be rconempty a n d b o u n d e d . T h e n o n e has

f ( x " > - f ( , - ' 1

-

^{I I}

iim s a y _{r -r *}

-

^"

_..

_* ^'

-

_Y ^rriax _{E s f )} ^{y ..}

Tnis c r i t e r i o n c a n b e a p p l i e 2 without e x a c t ~ n o w i e d g e of

Bf

( x ) b u t only ar, esti- mate t h a t

4

^f Gf ( z ) ^C I' f o r some s e t Y. If Y i s boundec, one may conciude t h a t f is locally i i p s c h i t z i a n around. x . if i t is known t h a t y

< ^X

f o r a i l y EY, o n e h a s from (1.19)

j f (z")

-

f (2') ! S X ~ X ' ~ - X ~ ! f o r Z ' a n d x" n e a r z.

2. LAGRANCE KULTPLIERS

AND

SENSITIVITY

Many ways h a v e been found f o r cieriving optimzLity conciitions f o r probiems with c o n s t r a i n t s , b u t not a l l of them p r o v i d e full information a b o u t t h e L a g r a n g e multipliers t h a t are obtained. The test of a good method i s t h a t i t should l e a d t o some s o r t of i n t e r p r e t a t i o n of t h e multiplier v e c t o r s in t e r m s of sensitivity o r g e n e r a l i z e d rates of c h a n g e of t h e optimal value in t h e problem with r e s p e c t t o p e r t u r b a t i o n s . Unti! q u i t e r e c e n t l y , a s a t i s f a c t o r y i n t e r p r e t a t i o n along s u c h lines w a s a v a i l a b l e only f o r convex programming a n d s p e c i a l c a s e s of smooth nonlinear programming. Now, however, t'nere a r e g e n e r a l r e s u l t s t h a t a p p l y t o a l l kinds of probiems, a t l e a s t in R n . These r e s u l t s d e m o n s t r a t e well t h e power of t h e new nonsmooth analysis and a r e n o t matched b y any- thing achieved by o t h e r techniques.

Let u s f i r s t c o n s i d e r a nonlinear programming probiem in i t s canonical p a r a m e t e r - izatior,:

( p ,

>

^{minimize g (z )} s u b j e c t t o x ^E K a n d gi ( z ) + u i 5 0 f o r -I =l,

...,

r ,

=

!If o r i =s+l,

...,

m ,

w h e r e g ,gl,...,gm are iocally Lipscnitzinn f ~ n c t i o n s on R n znd K i s a closed s u b s e t of R n ; t h e ui ' S are p a r a m e t e r s and. form a v e c t o r u ERm. Q y anniogy with what i s known in p a r t i c u l a r c a s e s of (P,), o n e can formulate t h e p o t e n t i a l optimality condition on a f e z s i b l e solution z , namely t h a t

m m

0 E a g ( z )

+ t i

=iyi Bgi ( x )

+

N K ( x ) with yi 2 0 and yiLgi

( z ) - u i l =

0 f o r i = i ,

...,

s ,

a n d a corresponciing constraint qualification at x

t h e only v e c t o r y = ( y ;,

. . .

, y,) satisfying t h e version of (2.1) in which t h e term 6g (z) i s omitted is p =C.

. .

In s m o o t h p r o g r a m m i n g , w h e r e t h e f u n c t i o n s g , g l ,

. . .

, g , are ail cont.~i~uo;;z!)r d i f f e r e n t i a b l e a n d t h e r e i s n o a b s t r a c t c o n s t r a i n t z E K , t h e f i r s t r e l a t i o n i n (2.1) r e d u c e s t o t n e g r a d i e n t e q u a t i o n

0 = Vg ( z )

-+ C E l y i

^{t g i ( z ) ,}

a n d o n e g e t s t h e c l a s s i c a l Kuhn-Tucker conditions. T h e c o n s t r a i n t q u a l i f i c a t i o n i s t h e n e q u i v a l e n t (by d u a l i t y ) t o t h e well known o n e of M a n g a s a r i a n a n d Fromovitz.

In c o n v e z p r o g r a m m i n g , w n e r e g , g l , ...,g, a r e ( f i n i t e ) c o n v e x f a c t i o n s , g, + l , . . . , g , are a f f i n e , a n d K i s a c o n v e x set, condition (2.1) i s a l w a y s s u f f i c i e n t f o r optimality. U n d e r t h e c o n s t r a i n t q u a l i f i c a t i o n (2.2), which in t h e a b s e n c e of e q u a l r t y c o n s t r a i n t s r e d u c e s t o t h e Slater c o n d i t i o n , i t i s also n e c e s s a r y f o r o p t i m a l i t y .

F o r t h e g e n e r a l case of (P,) o n e h a s t h e following r u i e a b o u t n e c e s s i t y .

THEOREM 2 ( C l a r k e [ I l l ) . S u p p o s e z is a LocaLLy optimaL s o L u t i o n to

(F,)

a t w h i c h t h e c o n s t r a i n t q u a L i f i c a t i o n (2.2) is s a t i s f i e d . T h e n t h e r e is a muLtipLier v e c t o r y s u c h that t h e o p t i m a L i t y c o n d i t i o n (2.1) is s a t i s f i e d .

This i s n o t t h e s h a r p e s t r e s u l t t h a t may b e s t a t e d , a l t h o u g h i t i s p e r h a p s t h e sim- p l e s t . C l a r k e ' s p a p e r [ll] p u t s a p o t e n t i a l l y s m a i l e r set in p i a c e of N K ( z ) a n d p r o v i d e s a l o n g s i d e of (2.2) a less s t r i n g e n t c o n s t r a i n t q u a l i f i c a t i o n in terns of "caimness" of (P,) with r e s p e c t t o p e r t u r b a t i o n s of u . H i r i a r t - U r r u t y [12] a n d R o c k a f e l l a r [13]

c o n t r i b u t e s o m e a l t e r n a t i v e ways of writing tine s u j g r a d i e n t r e l a t i o n s . F o r o u r F u r - p o s e s h e r e , let i t s u f f i c e t o mention t h a t T h e o r e m 2 r e m a i n s t r u e when t h e optimality condition (2.1) i s given in t h e s l i g h t l y s h a r p e r a n d m o r e e i e g a n t f o r m :

0 E a g ( z )

+

y a G ( z ) + N K ( z ) with y € N C ( G ( z ) i u ) ,

w h e r e G ( z )

=

( g l ( z ) , . . . , g , ( z ) ) a n d

C = ~ W ~ ~ / W ~ S O f o r i = L ,..., s and wi=O f o r - I = s + l ,

...,

^{m ] .} _(2.4)

T h e n o t a t i o n 5 G ( z ) r e f e r s t o C l a r k e ' s g e n e r a l i z e d J a c o b i a n [2] f o r t h e mapping G ; o n e h a s

Theorem 2 h a s t h e shining v i r t u e of combining t h e n e c e s s a r y conditions f o r smooth programming and. t h e o n e s f o r convex programming into a single statement. Moreover i t c o v e r s subsmooth programming a n d much m o r e , a n d i t aliows f o r a n a b s t r a c t con- s t r a i n t in t h e form of x ^€ K f o r a n a r b i t r a r y ciosed set K. Formuias f o r caiculating t h e normal c o n e NK(x) in p a r t i c u l a r c a s e s c a n t h e n b e used t o a c h i e v e additional s p e - c ializations.

W'nat Theorem 2 d o e s n o t d o i s p r o v i d e any i n t e r p r e t a t i o n f o r t h e muitipliers y i . In o r d e r t o a r r i v e at such a n i n t e r p r e t a t i o n , i t i s n e c e s s a r y t o look m o r e ciosely zi t h e p r o p e r t i e s of t h e marginai function

p ( u )

=

optimai value (infimum) in(P, ). (2.6)

Tinis i s a n extended-real-valued function on R m which i s lower semicontinuous when t h e following mild i n f - b o u n d e d n e s s c o n d i t i o n is fulfilled:

F o r e a c h

2L

€ R m , a c R a n d E > 0 , t n e s e t o f ali x ^€ K (2.7) satisfying g ( x )

s

c , g i ( x ) 5 u i

-

^{+ E} f o r i

=I, ..

. , s , a n d

u i

-

-E 5 gi ( 2 ) 5

Gi

+& f o r i =S

+I, ...,

m , i s bounded in R n .

This condition a l s o implies t h a t f o r e a c h u with p ( u )

<

(i.e. with t h e c o n s t r a i n t s of (P,) c o n s i s t e n t ) , t h e set of all (globally) optimal solutions t o (P,) i s nonempty and com- p a c t .

In o r d e r t o state t h e main g e n e r a l r e s u l t , w e let

Y(u )

=

set of aii multiplier v e c t o r s y t h a t s a t i s f y (2.1) f o r some optimal soiution x t o (P, ).

THEOREM 3 ( Z o c k a f e l l a r r13-j). S u p p o s e t h e i n f - b o u n d e d n e s s c o n d i t i o n (2.7) i s s a t i s f i e d . Let u be s u c h t h a t t h e c o n s t r a i n t s of (P,) a r e c o n s i s t e n t and e v e r y o p t i m a l s o l u t i o n x to (P,) s a t i s f i e s t h e c o n s t r a i n t q u a l i f i c a t i o n (2.2). Then 8 p ( u ) is a n o n e m p t y compact s e t w i t h

8 p ( u ) ~ c o Y ( u ) and e x t a p ( u ) c Y ( u ) . (2.9)

(where "ext" d e n o t e s e z t r e m e points]. In p a r t i c u l a r p is Locally L i p s c h i t z i a n a r o u n d u w i t h

p 0 ( u ; h ) S s u p y . h f o r a l l h.

Y EY(u )

I I

I n d e e d , a n y

X

s a t i s f y i n g ^' y i

< ^X

f o r a l l y EY(u) s e r v e s as a l o c a l L i p s c h i t z ccn- s t u n t :

I

i p ( u " ) p ( u ' ) is h !u"-ii'l w h e n u ' a n d u " a r e n e a r u. (2.11)

F o r smooth programming, t h i s r e s u l t was f i r s t proved. by Cauvin [14]. S e demon- s t r a t e 2 f c r t h e r t h z t when (P,) h a s a unique optimal solution z , f o r which t h e r e i s a unique multiplier v e c t o r y , s o t h a t Y ( u )

=

^y j , t h e n actually p i s d i f f e r e n t i a b l e at u with V p ( u ) = y

.

F o r convex programming one knows ( s e e [3]) t h a t ap ( u )

=

'17(u ) always ( u n d e r o u r inf-boundedness assumption) a n d consequently

Minimax formulas t h a t give p f ( u ; k ) in c e r t a i r . c a s e s of smooth programming w h e r e 'IP(u) i s not just a singleton car, b e f o r exampie found, in Demyanov znd Yaiozemov ;:5] a n d Rocirafeliar [IS]. Aside from s u c n s p e c i a l csses t h e r e a r e no formxias iznowr, f o r p f ( u ; n ) . S e v e r t h e l e s s , Theorem 3 d o e s p r o v i d e z n e s t i m a t e , b e c z c s e p ' ( u ;h ) I p ^"(21 ;i; ) whenever p ' ( u ; h ) e x i s t s . ( i t i s i n t e r e s t i n g t o n o t e in t h i s cor,nec- tion t h a t b e c a u s e p i s Lipscnitzian a r o u n d u by Theorem 3, i t i s actua!!y d i f f e r e n t i a b l e aimost e v e r y w h e r e a r o u n d u by R a a e m a c h e r ' s theorem.)

Theorem 3 h a s r e c e n t l y b e e n b r o a d e n e d in [ 6 j t o include more g e n e r a l kines o i p e r t u r b z t i o n s . Consider t h e p a r a m e t e r i z e d problem

(Qv

>

minimize f (v ,z ) o v e r a l i z satisfying

F ( v , z )

c

C a n 6 (v , z ) E D ,

where v i s a p a r a m e t e r v e c t o r in R d , t i e functions f : Rd x Rn -R and F: Rd x Rn -Rm a r e locally Lipschitzian, a n d t h e s e t s C cRm a n 6

D c

R d xRn a r e closed. H e r e C couid b e t h e c o n e in (2.4), in which e v e n t t h e c o n s t r a i n t F ( v ,,-) E C would r e d u c e t o

f i ( 2 , ~ ) I 0 f o r i =l, ..., S ,

=

0 f o r i =s

+I, ...,

m ,

b u t t n i s c h o i c e of C i s n o t r e q u i r e d . The condition ( v ,z j E

D

may e q u i v a i e z t ! ~ 5 e writ- t e n as z E I'(v), where F is tile ciosed multifunction whose g r a p h is D. I t r e p r e s e n t s t h e r e f o r e a n a b s t r a c t c o n s t r a i n t t h a t c a n v a r y with v . A fixed z c s t r a c t c o n s t r a i n t

x E K c o r r e s p o n d s t o r ( v ) = K , D = R d X K .

In t h i s m o r e g e n e r a l s e t t i n g t h e a p p r o p r i a t e optimality condition f o r a f e a s i j l e solution x t o ( 9 , ) i s

f o r some y a n d z with y m c ( F ( v , x ) ) ,

and t h e c o n s t r a i n t qualification i s

t h e oniy v e c t o r p a i r ( y ,z ) satisfying t h e v e r s i o n of ( 2 . 1 3 ) in which t h e t e r m af _{( v} , z ) i s omittee is ( y ,z ) = ( 0 , 0 ) .

TIYEOREI! 4 ( R o c k a f e l l a r [ 6 , $81). S u p p o s e t n a t x is a l o c a l l y o p t i m a l s o l u t i ~ n to (9,) a t w h i c n t h e c o n s t r a i n t q u a l i f i c a t i o n ( 2 . 1 4 ) is s a t i s f i e d . Then t h e r e i s a m u l t i p l i e r p a i r ( y ,z) s u c h t h a t t h e o p t i m a l i t y c o n d i t i o n ( 2 . 1 3 ) i s s a t i s f i e d .

Theorem 4 r e d u c e s t o t h e v e r s i o n of Theorem 2 having ( 2 . 3 ) in p l a c e of ( 2 . 1 ) when

( Q , ) i s t a k e n t o b e of t h e form

( P )

namely when

f ( v , x ) = g ( x ) , F ( v , x ) = G ( x ) + v , D = R m Y. K ( R ~ = R ~ ) , anci C is t h e c o n e in ( 2 . 4 ) . F o r tine c o r r e s p o n d i n g v e r s i o n of Theorem 3 in terms of tine marginal function

q ( v )

=

optimal value in (Q, ) , ( 2 . 1 5 )

w e t a k e inf-boundeaness t o mean:

F o r e a c h

-7 md

^: a ER a n d E >0, t h e set of a!; z satisfying f o r some v wiik

!

^v

-C !

5 E

t h e c o n s t r s i n t s F ( v , x ) EC, ( v , z ) E D , and having f ( v ,z) 5 a, i s bounded in

R n

Again: t h i s p r o p e r t y e n s u r e s t h a t q i s lower semicontinaous, and t h n t f o r e v e r y 2; f o r which t h e c o n s t r a i n t s of ( Q , ) a r e c o n s i s t e n t , t h e set of optimai soiutions t o ( Q , ) i s nonempty and compact. Let

Z ( v )

=

set of a l l v e c t o r s z t h a t s z t i s f y t h e multiplier ( 2 . : 7 )

-

15 -

condition (2.13) f o r some optimal solution z t o (Q,) and v e c t o r y .

THEOREX 5 (Rockafellar i6, $81). Suppose t h e i n f - b o u n d e d n e s s c o n d i t i o n (2.16) i s s a t i s f i e d . Let v be s u c h t h a t t n e c o n s t r a i n t s of (Q,) are c o n s i s t e n t a n d e v e r y optimal s o l u t i o n z to (Q,) s a t i s f i e s t n e c o n s t r a i n t quaLifzcation (2.14). Ther, B q ( v ) i s a n o n e m p t y compact set w i t h

Bq(v) c c o Z ( v ) a n d e x t 6 q ( v ) c Z ( v ) . (2.18)

I n p a r t i c u l a r q i s Locally L i p s c h i t z i a n a r o u n d v w i t h q 0 ( 2 ; ; h ) I s u p z - h for all n.

z E Z ( V )

, 8

A n y A s a t i s f y i n g i z i

<

h for aLL z E Z ( v ) serves a s a Local L i p s c h i t z c o n s t a n t : : q ( v " ) - q ( v ' ) ~ ~ h l v " - - v r ; w h e n v ' a n d v " a r e n e a r v. (2.20)

The g e n e r a l i t y of t n e c o n s t r a i n t s t r u c t u r e in Theorem 5 will make possibie in t h e n e x t section a n application t o t h e study of multifunctions.

3. STABILITY OF CONSTRkr'NT SYSTEKS

The sensitivity r e s u l t s t h a t h a v e just been p r e s e n t e d are c o n c e r n e d with wnat h a p p e n s t o t h e optimal vaiue in a probiem when p a r a m e t e r s v a r y . I t t u r n s o u t , though, t h a t they car, be applied t o t h e study of what h a p p e n s t o t h e f e a s i b l e solution s e t and t h e optimal solution s e t . In o r d e r t o explain t h i s and indicate t h e main r e s u l t s , w e must c o n s i d e r t h e kind of Lipschitzian p r o p e r t y t h a t p e r t a i n s t o multifunctions (set-valued mappings) and t h e way t h a t t h i s car, b e c h a r a c t e r i z e d in t e r m s of a n a s s o c i a t e d dis- t a n c e function.

Let

Y: R d 3"

b e a closed-valued multifunction, i.e. r ( v ) i s f o r e a c h ^2; E

R d

a closed s u b s e t of

R n ,

possibly empty. The motivating exampies a r e , f i r s t , r ( v ) t z k e n t o b e t h e s e t of a l l f e a s i b l e soiutions t o t h e pnrameterized optimization problem

(9,)

a b o v e , and s e c o n i , ? ( v ) t a k e n t o b e t h e s e t of a l l optimal s o i ~ t i o n s t o (Q,).

One s a y s t h a t r ( v ) i s Locally L i p s c h i t z i a n arocr.6 ^2; if f o r a i l ^2;' arid v " irl some neighborhood of v one h a s T(z; ') an2 T(v ") nonempty and bounded with

f i e r e B d e n o t e s t h e cioseci unit ball in Rn and

X

i s a Lipschitz c o n s t a n t . This p r o p e r t y c a n b e e x p r e s s e d equivalently by means of t h e c l a s s i c a l Hausdorff m e t r i c on t n e s p a c e of a l l nonempty compact s u b s e t s of Rn :

h a u s ( r ( v "), r ( v ')) 5

X !

v " -u '

i

when v ' a n d v " a r e n e a r v . (3.2)

I t is i n t e r e s t i n g t o n o t e t h a t t h i s i s a "differential" p r o p e r t y of s o r t s , inasmuch as it d e a l s with rates of c h a n g e , o r at l e a s t bounds o n s u c h rates. Until r e c e n t l y , however, t h e r e h a s n o t b e e n any v i a b l e p r o p o s a l f o r "differentiation" of

r

t h a t might b e associ- a t e d with i t . A c o n c e p t investigated by Aubin [I71 now a p p e a r s promising as a candi- d a t e ; see t h e end of t h i s s e c t i o n .

Two o t h e r definitions a r e needed. The multifunction

r

i s l o c a l l y b o u n d e d at v if t h e r e i s a neighbornood V of v a n d a bounded set S cRn s u c h t h a t r ( v ') cS f o r a l l V ' E V . I t i s closed at v if t h e e x i s t e n c e of s e q u e n c e s ivk

1

and izk

!

with v k 4 0 , zk E r ( v k ) a n d z k -+z impiies z ~ r ( v ) . Finaily, w e i n t r o d u c e f o r

7

t h e d i s t a n c e f u n c t i o n

d r ( v , w )

=

d i s t ( r ( v ) . ~ . )

=

^{min - z -zu}

.

I Er(v )

The following g e n e r a l c r i t e r i o n f o r Lipschitz c ~ n t i n u i t y c a n t h e n b e s t a t e d .

THEOREM 6 ( R o c k a f e l l a r [ 1 8 ] ) . The m u l t i f u n c t i o n

r

^{i s} l o c a l l y L i p s c h i t z i a n a r o u n d v i f a n d o n l y i f

r

i s closed a n d l o c a l l y b o u n d e d at v w i t h r ( v )

+ 6 ,

a n d i t s d i s t a n c e f u n c t i o n d i s l o c a l l y L i p s c h i t z i a n a r o u n d ( v ,z ) f o r e a c h x c F(v ).

The c r u c i a l f e a t u r e of t h i s c r i t e r i o n i s t h a t i t r e d u c e s t h e Lipschitz continuity of

I?

t o t h e Lipschitz continuity of a function d r which i s actually t h e marginal function f o r a c e r t a i n optimization problem (3.3) p a r a m e t e r i z e d by v e c t o r s v a n d w

.

This p r o b - lem f i t s t h e mold of

(Q,),

with v r e p i a c e d by ( v , w ) , and i t t h e r e f o r e comes uncier t h e c o n t r o l of Theorem 5: in a n adapted. f o r m . One is r e a d i l y a b l e by t h i s r o u t e t o cierive t h e following.

TKEOREK 7 ( R o c k a f e i l a r [18]). Let

I?

be t h e m u l t i f u n c t i o n t h a t a s s i g n s t o e a c h v E R d t h e set of a l l feasible s o l u t i o n s to problem ( 8 , ) :

r ( z i )

=

{ Z ! ~ ( v , z ) E C a n d ( v , z ) E

D f .

_(3.4)

S u p p o s e for a g i v e n v t h a t

r

i s Locally b o u n d e d a t z:, a n d t h a t r ( v ) i s n o n e m p t y w i t h t h e c o n s t r a i n t q u a l i f i c a t i o n (2.14) s a t i s f i e d b y e v e r y x c r ( z ; ) . T h e n

r

i s l o c a l l y L i p s c n i t z i a n a r o u n d v

.

COROLLARY. Let

r : R d = R n

be a n y m u l t i J u n c t i o n w n o s e g r a p h D

= I

^{( v ,x )}

!

z ^{E ~ ( V )}

1

i s closed. S u p p o s e for a g i v e n v t h a t

r

^{i s} l o c a l l y b o u n d e d a t v , a n d t h a t r ( v ) i s n o n e m p t y w i t h t h e f o l l o w i n g c o n d i t i o n s a t i s f i e d j ' o r e v e r y z € ? ( v ) : t h e o n l y v e c t o r z w i t h ( z ,0 ) E ND(v ,z) i s z

=

0. ( 3 . 5 )

T h e n

r

i s l o c a l l y L i p s c h i t z i a n a r o u n d v .

The c o r o l l a r y i s just t h e c a s e of t h e t'neorem w h e r e t h e c o n s t r a i n t F ( v ,z) E C i s triviaiized. I t c o r r e s p o n d s closely t o a r e s u l t of Aubin [ l i ' j , a c c o r d i n g t o whicn

r

^{i s}

"pseudo-Lipschitzian" r e i a t i v e t o t h e p a r t i c u l a r p a i r ( v , z ) with z E r ( v ) if t h e p r o j e c t i o n of t h e t a n g e n t cone T D ( v , z ) c Rd x R n

on Rd i s a l l of R d .

Conditions (3.5) a n d (3.6) are equivalent t o e a c h o t h e r by t h e duality between ND(v , z ) a n d T D ( v , z ) . The "pseudo-Lipschitzian" p r o p e r t y of Auhin, which will n o t b e defined h e r e , i s a s u i t a b l e localization of Lipschitz continuity which f a c i l i t a t e s t h e t r e a t m e n t of multifunctions

I'

with r ( v ) unbounded, as i s highly d e s i r a b l e f o r o t h e r p u r p o s e s in optimization t h e o r y ( f o r i n s t a n c e t h e t r e a t m e n t of e p i g r a p h s d e p e n d e n t on a p a r a m e t e r v e c t o r v ) . As a matter of f a c t , t h e r e s u l t s in Rocisafellar :I83 build on t i i s c o n c e p t of Aubin and are n o t limited t o locally bounded multifunctions. Only z s p e c i a l c a s e h a s b e e n p r e s e n t e d in t h e p r e s e n t p a p e r .

This t o p i c is a l s o c o n n e c t e d with i n t e r e s t i n g icieas t h a t Aubin h a s p u r s u e d t o w a r d s a d i f f e r e n t i a l t h e o r y of multifunctions. Aubin defines the multifunction whose g r a p h is t h e C l a r k e t a n g e n t c o n e T D ( v , z ) , w h e r e D i s t h e g r a p h of I', t o b e t h e d e r i v a t i v e of

r

at v r e l a t i v e t o t h e point x ^E r ( v ) . In denoting t h i s d e r i v a t i v e muitifunction by

r;,,

^,

we h a v e , b e c a u s e T D ( v , z ) i s a ciosed convex c o n e , t h a t

r;,,

^{i s z} closed c o n v e z p r o c e s s from Rd t o Rn in t h e s e n s e of convex annlysis :3, 5391. Convex p r o c e s s e s are v e r y much a k i n t o i i n e a r t r a n s f o r m a t i o n s , a n d t h e r e i s q u i t e z c o n v e z a l g e b r a f o r them ( s e e

[3, $391,

[iq,

and 120:). In p a r t i c u l a r , ,, h a s a n a d j o i n t L"; :, : Rn Z R d , which t u r ~ s o u t in t h i s c a s e t o b e t h e closed convex p r o c e s s with

In t h e s e t e r m s Aubin's condition (3.6) c a n b e written as ciorr,

r;,,

= R": w h e r e a s t h e dual condition (3.5) is

T ; ; ( "

= !!3 j . The l c t t e r i s equivaient t o , being iocally bounded a t t h e o r i g i n .

T h e r e i s t o o much in t h i s vein f o r u s t o b r i n g f o r t h h e r e , but t h e f e w f a c t s we h a v e c i t e d may s e r v e t o indicate some new d i r e c t i o n s in which nonsmooth a n a i y s i s i s now going. W e may soon h a v e a hignly deveioped a p p a r a t u s t h a t can b e a p p l i e d t o t h e s t u d y of a l l kinds of multifunctions a n d t h e r e b y t o s u b d i f f e r e n t i a l multifunctions in p a r t i c z - iar

.

F o r example, as an a i d in t h e analysis of t h e s t a b i l i t y of optimal solutions a n d mul- t i p l i e r v e c t o r s in problem (Q,), o n e c a n t a k e up t h e s t u d y of t h e Lipschitzian p r o p e r - t i e s of t h e multifunction

r ( v ) = s e t of a l l (x , y ,z ) suck t h a t x i s f e a s i b l e in (Q,) a n d t h e optimality condition (2.13) i s satisfied.

Some r e s u l t s on s u c h l i n e s are given in Aubin [I71 a n d R o c k a f e l l a r [2;<.

REFERENCES

[ I ]

F.H.

C l a r k e , "Generalized g r a d i e n t s a n d applications", Trans. Amer. S o c . 205 (1975), pp.247-262.

i23 r . H . C l a r k e , m t i m i z a t i o n a n d Nonsmoo t h A n a l y s i s , Wiley-icterscience, Sew York, 1983.

13) R.T. R o c k a f e l l a r , Convex A n a l y s i s , P r i n c e t o n University P r e s s , P r i n c e t o n X J , 1970.

[4] R.T. Rockaf e l l a r , "Favorable c l a s s e s of Lipschitz continuous functions in s u b g r a - dient optimization", P r o c e s s e s i n Nondifierentiable O p t i m i z a t i o n , E. Surminsiri (ed.), IIASA Collaborative P r o c e e d i n g S e r i e s , I n t e r n a t i o n a l I n s t i t u t e f o r Appiied Systems Analysis, L a x e n b u r g , Austria, 1 9 8 2 , pp. 125-143.

[5] S . Sairs, Theory of t h e I n t e g r a l , Monografie Matematyczne S e r . , no. 7 , 1937; 2nZ r e v . e d . Dover Press, Few York, 1954.

i6;

^R.T. Xockafeliar, "Sxtensions of s u b g r a d i e n t caiculus with a p p l i c a t i o n s t o optimi- zatior.", ;. Xonlinear Ana;., t o z p p e a r in 1985.

C'72 R.VJ. Cnaney, h!ath. C)per. Res. 9 (19S4).

[8] X.T. R o c k a f e l l a r , "Generalized d i r e c t i o n a l d e r i v a t i v e s and s u b g r z d i e n t s of non- convex functions", Canaciian 2 . Yath. 32 (1980), ~ p . 157-180.

[9] R.T. R o c ~ a f e l i a r , T h e T h e o r y of S u b g r a d i e n t s a n d i t s A p p l i c a t i o n s t c P r o b l e m s of D p t i m i z a t i o n : C o n v e z a n d N o n c o n v e z P u n c t i o n s , Eeidermann Verlag, West Berlin, 1961.

[ l o ] R.T. R o c k a f e l l a r , "Clarke's tangent c o n e s and t h e b o u n d a r i e s of closed sets in R ~ " , J. Xoniin. Anal. 3 (1972), pp.145-154.

[I13 P.E. Ciarite, "A new a p p r o a c h t o Lagrange muitipliers", Math. Oper. Res. 1 (1976), pp. 165-1'74.

[I21 J-B H i r i a r t - U r r u t y , "Refinements o: n e c e s s a r y optimaiity conditions in nondif- f e r e n t i a b i e programming,

I,"

Appl. Yzth. Opt. 5 (1979), pp.63-C2.

[I31 R.T. R o c k a f e l l a r , "Lzgrange multtpiiers and s u b d e r i v a t i v e s G? cptimai value func- tions in nonlinear progrzmming", J!ath. P r o g . Study 1 7 (1982), 28-66.

[I41 J . Gauvin, T h e g e n e r a l i z e d g r z d i e n t cf a marginal function in mathematical p r o - gramming problem", Math. Oper. Res. 4 (1979), pp.458-453.

[I51 V.F. Demyanov a n d V.N Malozemov, "On t h e t h e o r y of n o n l i n e a r minimax p r o b - lems", Russ. Kath. S u r v . 25 (1971), 57-115.

[16] R.T. Rockafel!ar,"Directional differentiability of t h e optimal v a l u e in a nonlinear programming problem", Math. P r o g . StuZies 2 1 (1984), pp. 213-226.

[17] J.P. Aubin, "Lipschitz b e h a v i o r of solutions t o convex minimization problems", Math. O p e r . Res. 9 (1984), p p . 87-111.

[18] R.T. R o c k a f e l l a r , "Lipschitzian p r o p e r t i e s of multifunctions", J . Nonlin. A ~ a l . , t o a p p e a r in 1985.

;13] R.T. R o c k a f e l l a r , " C o ~ v e x a l g e b r a and duality ia dynamic models of productior,", in Mathematical Models of Economic (J. L o s t , e d . ) , h'orth-Holland, 1973, pp.351- 378.

[20] R.T. Rockafelinr, M o n o t o n e P r o c e s s e s of C o n v e z a n d C o n c a v e T y p e , Memoir no.77, Amer. Math. Soc., P r o v i d e n c e RI, 1967.

[213 R.T. R o c k a f e l l a r , "Maximal monotone r e l a t i o n s and t h e second d e r i v a t i v e s of nonsmooth functions", Ann. Inst. H. P o i n c a r d , Anaiyse Non L i n d a i r e 2 (1985), pp.167-184.