An Algorithm for Minimizing a Certain Class of Quasidifferentiable Functions

NOT FOR QUOTATION WITHOUT P E R M I S S I O N O F THE AUTHOR

AN ALGORITHPl FOR M I N I M I Z I N G A CERTAIN CLASS OF Q U A S I D I F F E R E N T I A B L E FUNCTIONS

V . F . D e m y a n o v S . G a m i d o v T . I . S i v e l i n a D e c e m b e r 1 9 8 3 W P - 8 3 - 1 2 2

K o r k i n g P a p e r s a r e i n t e r i m r e p o r t s o n w o r k of t h e I n t e r n a t i o n a l I n s t i t u t e f o r A p p l i e d S y s t e m s A n a l y s i s a n d have received o n l y l i m i t e d r e v i e w . V i e w s o r o p i n i o n s expressed h e r e i n do n o t n e c e s s a r i l y repre- s e n t t h o s e of t h e I n s t i t u t e o r of i t s N a t i o n a l Member O r g a n i z a t i o n s .

INTERNATIONAL I N S T I T U T E FOR A P P L I E D SYSTEMS ANALYSIS A - 2 3 6 1 L a x e n b u r g , A u s t r i a

AN ALGORITHM FOR M I N I M I Z I N G A C E R T A I N CLASS OF QUASIDIFFERENTIABLE FUNCTIONS

V.F. Demyanov, S. Gamidov, T . I . S i v e l i n a

1 . I N T R O D U C T I O N

One i n t e r e s t i n g and i m p o r t a n t c l a s s of n o n d i f f e r e n t i a b l e f u n c t i o n s i s t h a t p r o d u c e d by smooth c o m p o s i t i o n s o f max-type f u n c t i o n s . Such f u n c t i o n s a r e o f p r a c t i c a l v a l u e and have been s t u d i e d e x t e n s i v e l y by s e v e r a l r e s e a r c h e r s [ I - 3 1 . W e t r e a t them a s q u a s i d i f f e r e n t i a b l e f u n c t i o n s and a n a l y z e them u s i n g q u a s i - d i f f e r e n t i a l c a l c u l u s .

One s p e c i a l s u b g r o u p o f t h i s c l a s s o f f u n c t i o n s ( n a m e l y , t h e sum of a max-type f u n c t i o n and a min-type f u n c t i o n ) h a s been s t u d i e d by T . I . S i v e l i n a [ 4 ] . The main f e a t u r e o f t h e a l g o r i t h m d e s c r i b e d i n t h e p r e s e n t p a p e r i s t h a t a t e a c h s t e p i t i s n e c e s - s a r y t o c o n s i d e r a b u n d l e o f a u x i l i a r y d i r e c t i o n s and p o i n t s , of which o n l y one c a n b e c h o s e n f o r t h e n e x t s t e p . T h i s r e p u i r e - ment seems t o a r i s e from t h e i n t r i n s i c n a t u r e o f n o n d i f f e r e n t i a b l e

f u n c t i o n s .

2 . THE UNCONSTRAINED CASE L e t

where

x € E n ' y . ¹ ( x ) max' m i j ( x ) , ^{I i}

-

⁼ l:Ni j €1 i

and f u n c t i o n s F ( x r y l r , y m ) and

m i j

( x ) a r e c o n t i n u o u s l y d i f f e r - e n t i a b l e on Entm and E n , r e s p e c t i v e l y .

Take any g E En. Then f o r a

-

> 0 w e have

where

ayi ( x ) - yi ( x + a g )

-

yi ( x )

= l i m = max ( 4 :

- ^{( X I}

^{t g ) I}

a9 a++ 0 a jERi ( x ) 1 7

T h i s l e a d s t o

where

and

I t i s c l e a r t h a t c o n v e r g e n c e i n ( 2 ) and ( 4 ) i s u n i f o r m w i t h r e s p e c t

* +

For x €En to be a local minimum point of f it is sufficient that

- * *

-af(x ) cint

-

af(x )

.

The following lemmas can be derived from the above necessary and sufficient conditions:

Lemma _{1 .} For any set of coefficients

there exists another set of coefficients

such that

* *

(I£ aF(y(x ) =

o

put iij =

o

Y j E R ~ ( X ) . I ay:

Condition (6) is a multipliers rule

-

note the difference between it and the Lagrange multipliers rule for mathematical programming.

It follows from (6) that x is a stationary point of the

*

smooth function

and if

-

af(x

*

) consists of more than one point then the set { A }

i

j is not unique. (Of course, it may not be unique even if (x*) is a singleton.)

+

For an arbitrary quasidifferentiable function condition (7)

is sufficient for a minimum only with certain additional assumptions.

However condition (7) is sufficient for functions described by ( 1 ) .

Lemma 2. I f f o r any w ~ B ( x

*

) t h e r e e x i s t s e t s

r.

^{n+ 1}

i € l : n + l and a 0

1

a i = l

i= 1

?

such t h a t t h e v e c t o r s i v i l form a s i m p l e x ( i . e . , v e c t o r s {vi-vncl

1

n+ 1

*

i € l : n ) a r e l i n e a r l y i n d e p e n d e n t ) and w =

1

^aivi, t h e n x i s a i= 1

l o c a l minimum p o i n t of f on En.

We s h a l l now i n t r o d u c e t h e f o l l o w i n g s e t s , where E

-

> 0 , ^1-1 - > 0:

i

^aF ( y ( x ) ) +

a

= c o V E E ~ / V =

C

a F ( y ( X ) ) $ ! . ( x ) , ~ E R ~ ~

-E ax i E I + ( x ) ayi 1 3

L e t f be d e f i n e d by ( 1 ) . A p o i n t x € E n

*

w i l l be c a l l e d an E-inf- stationary point o f f on E i f

n

We s h a l l now d e s c r i b e an a l g o r i t h m f o r f i n d i n g an E - i n f - s t a t i o n a r y p o i n t , w i t h ^E > 0 and ^p > 0 f i x e d .

Choose an a r b i t r a r y x o E E ~ . Suppose t h a t x k h a s been found.

t h e n xk i s a n E - i n f - s t a t i o n a r y p o i n t and t h e p r o c e s s t e r m i n a t e s . I f , on t h e o t h e r hand, ( 7 ) i s n o t s a t i s f i e d t h e n f o r e v e r y W E B ( x )

1-I k

we f i n d

min ll w+vll =

I

w+vk ( w )

I I .

~ €( x k ) 2 ~ ~

w+vk ( W )

I f w

+

vk ( w ) ^f 0 t h e n l e t g k ( w ) =

-

a n d compute

I I

^{W + V k} ( w )

I I

min f ( x k + a g k ( w ) ) = f ( x k + a k ( w ) g k ( w ) a > o

I f w

+

vk ( w ) = 0 t h e n t a k e a k ( w ) = 0 and f i n d

W e t h e n s e t

X k+ 1 ^{= X k}

+

^c% _k^{( w} _k^{) g} _k^{( w} _k⁾

.

I t i s c l e a r t h a t

By r e p e a t i n g t h i s p r o c e d u r e w e o b t a i n a s e q u e n c e o f p o i n t s { x k } . I f i t i s a f i n i t e s e q u e n c e ( i . e . , c o n s i s t s o f a f i n i t e number o f p o i n t s ) t h e n i t s f i n a l e l e m e n t i s a n E - i n £ - s t a t i o n a r y p o i n t by c o n s t r u c t i o n . O t h e r w i s e t h e f o l l o w i n g r e s u l t h o l d s .

T h e o r e m I . I f t h e s e t D ( x o ) = { x E En

1

^{f ( x )}

-

^< f ( x o )

1

i s bounded t h e n a n y l i m i t p o i n t o f t h e s e q u e n c e { x k } i s a n E - i n f - s t a t i o n a r y p o i n t o f f on E n .

P r o o f . The e x i s t e n c e o f l i m i t p o i n t s f o l l o w s from t h e bounded-

* *

n e s s o f D ( x o ) . L e t x be a l i m i t p o i n t o f i x k } , i . e . , x = l i m xk

.

I t i s c l e a r t h a t kS+w s

Assume t h a t x

*

i s n o t a n & - i n £ s t a t i o n a r y p o i n t . Then t h e r e

* *

e x i s t s a w EBo ( x ) s u c h t h a t

min Ilw+vll

*

= a > 0

.

vEa f ( x * )

-E

We s h a l l d e n o t e by wk

*

t h e p o i n t i n B ( x k ) which i s n e a r e s t t o

* * *

^lJ

*

w and by p ( w k ) t h e d i s t a n c e of wk from w

.

^{I t i s} o b v i o u s t h a t

*

P(W; 1 > G. It may a l s o be s e e n t h a t t h e mapping

a

f ( x )

s kS+m -E

i s u p p e r - s e m i c o n t i n u o u s . From ( 1 0 ) and t h e above s t a t e m e n t s i t f o l l o w s t h a t t h e r e e x i s t s a K such t h a t

* * *

a

min llwk +vll llwk +v(wk ) I l ⁼ a k

2 2

Yks > K . ( 1 1 )

v € a E f ( x k ) s ^{S S S}

S

Now we have

where

* *

i

⁺

^I

^{I ~ ~ ) I ( ~}

^*

^{( ~}

^*

X k -X + agk n i n +!. ( X xk -x + a g k

j.

⁽¹³¹

s S ; i E 1

-

( x * ) ~ E ( x * ) R ~^,, ayi ¹ I s s

S i n c e

max a

+

min bi

5

max [ a . + b . l < max a

+

max bi i E 1 i

i ~ 1 i € I ^{1 1 -} i ~ 1 ⁱ i~ I min a i

+

rnin bi

5

rnin [ai+bi] - < rnin a

+

max bi

i ~ 1 i ~ 1 i ~ 1 i ~ 1 i

i~ I

i t f o l l o w s from ( 1 3 ) t h a t

3. THE CONSTRAINED CASE Let us consider the set

where

yi(x) = max

4

(x) ^I I - = 1 : ^I iE(m+l):p ^I jEIi ij

and the functions H ( ~ , y ~ + ~ ~ . . . ~ y ~ ) and mij(x) are continuously differentiable on En-m+p and En, respectively. Let the function f be of the form (1). The function h is quasidifferentiable and its quasidifferential can be 'described analogously to that of f in Section 2. The set R defined by (15) is called quasidiffer- entiable.

The problem is to find min f (x)

.

As in (3) we have

XER

where

Let

Now we have

a H ( ? ( x ) a H ( F ( x ) ^{) 1}

h ( x + a g ) = h ( x )

+

a

1

^max

( ⁺

i € I ; ( x ) j€Ri ( x ) ax ayi

where

W e now i n t r o d u c e t h e s e t s

where ^E

-

> 0 , ~ 1 0 .

S e v e r a l e q u i v a l e n t n e c e s s a r y c o n d i t i o n s f o r a minimum h a v e been o b t a i n e d [ 6 , 7 , 8 ] . Here we t a k e t h e n e c e s s a r y c o n d i t i o n i n t h e form p r o p o s e d by A. S h a p i r o [ 8 ] :

I n o r d e r t h a t x

*

E R b e a minimum p o i n t o f a q u a s i d i f f e r e n t i - a b l e f u n c t i o n f d e f i n e d on a q u a s i d i f f e r e n t i a b l e s e t R , it i s n e c e s s a r y t h a t

- *

-

a f ( x

c

a f ( x * )

*

-

f o r h ( x ) <

o

( 1 6 )

8 - *

* *

- [ a f c X * ) + ~ h ( x * ) l C C O { ~ ( X

-

a h ( x ⁾ , a h ( x ⁾

-

a f ( x 1 1

( 1 7 ) f o r h ( x )

*

= 0

.

Take ^E - > 0 , T

-

> 0. W e s h a l l c a l l x

*

E R a n ( E , T ) - i n f - s t a t i o n a r y p o i n t o f f on R i f

f o r ^{- T} - < h ( x

*

⁾ - < 0

W e s h a l l now d e s c r i b e a n a l g o r i t h m f o r f i n d i n g a n ( ~ , ~ ) - i n f - s t a t i o n a r y p o i n t w i t h ^E > O I ^p > 0 and ^T > 0 f i x e d .

Choose a n a r b i t r a r y x o E R . Suppose t h a t x k E R h a s been f o u n d . I f c o n d i t i o n ( 1 6 ) o r ( 1 7 ) i s s a t i s f i e d a t xk t h e n x k i s a n ( E , T )

-

i n £ - s t a t i o n a r y p o i n t and t h e p r o c e s s t e r m i n a t e s . T h e r e a r e two o t h e r p o s s i b i l i t i e s :

( a ) h ( x k ) < - T

( b ) - T - < h ( x k )

-

^< 0

.

I n c a s e ( a ) w e p e r f o r m o n e s t e p i n t h e m i n i m i z a t i o n o f t h e f u n c t i o n fr u s i n g t h e same a l g o r i t h m a s i n S e c t i o n 2 e x c e p t t h a t

min f ( x k

+

a g k ( w ) )

a>O

must b e r e p l a c e d by

min f ( x k

+

a g k ( w ) )

a > o - xk+agk ( w ) En

I n c a s e ( b ) w e h a v e t o f i n d

-

-

miniIIwl+w2+vlI ( v E c o { a E f ( x k )

-

a E h ( x k ) ( x k )

- a E f

( x k )

= I I W l + W ₂+ v _k( w ₁+W ₂)

I I

I

f o r e v e r y w l E B ( x k ) and w2 E B ( x k )

.

1-1 1-I

Compute

min f ( x k ( a ) ⁾ = f ( x k ( w l , w 2 ) )

a > o - h ( x k ( a ) )(0 where

x k ( a ) ⁼ xk

-

^a ( w , + w + V ( W + W 2 ) )

2 k l

We t h e n f i n d

S e t t i n g x ~ = +x ~(w k l l ~ k 2 )

,

i t i s c l e a r t h a t

R e p e a t i n g t h i s p r o c e d u r e , we c o n s t r u c t a s e q u e n c e o f p o i n t s { x k } . I f i t i s a f i n i t e s e q u e n c e t h e n t h e f i n a l e l e m e n t i s an ( E , T ) -

i n f - s t a t i o n a r y p o i n t o f f on R; o t h e r w i s e it c a n be shown t h a t t h e f o l l o w i n g t h e o r e m h o l d s .

T h e o r e m 2. I f t h e s e t

D

( x o ) = { x E Q

1

f ( x ) - < f ( x o )

1

i s hounded t h e n any l i m i t p o i n t o f t h e s e q u e n c e { x k } i s a n ( E . T ) - i n f - s t a t i o n - a r y p o i n t o f f on Q.

P r o o f . Theorem 2 c a n b e proved i n t h e same way a s Theorem 1 . Remark 1 . I f t h e i n i t i a l p o i n t x d o e s n o t b e l o n g t o Q i t i s

0

n e c e s s a r y t o t a k e a few p r e l i m i n a r y s t e p s i n t h e m i n i m i z a t i o n o f f u n c t i o n h u n t i l a p o i n t b e l o n g i n g t o Q i s o b t a i n e d .

Remark 2 . To f i n d an i n f - s t a t i o n a r y p o i n t ( i . e . , an ( E , T ) -

i n f - s t a t i o n a r y p o i n t where E = T = 0 ) i t i s n e c e s s a r y f o r ^E t o t e n d t o z e r o ( t h i s c a n be a c h i e v e d u s i n g t h e s t a n d a r d m a t h e m a t i c a l programming t e c h n i q u e s ) .

Remark 3 . I t i s p o s s i b l e t o e x t e n d t h e proposed a p p r o a c h t o t h e c a s e where

max H . (x,z (x)

,...,

z . (XI) jEJ

I

j 1

I

mj

and the functions Fi(x,yil1...,yim )

,

H.(x,z

,...,

^{z . )}

,

i

I

j 1 7mj

@ i k R ( ~ ) are continuously differentiable.

Remark 4. Instead of the one-dimensional minimization proposed in (18) it is possible to take

where

03

The authors wish to thank cordially Professor A.M. Rubinov for his valuable advice and Helen Gasking for her careful editing.

REFERENCES

1. G. Papavassilopoulos. Algorithms for a class of nondiffer- entiable problems. J. Optimization Theory and Applications, Vol. 34(1), 1981. pp. 41-82.

2. A. Ben-Tal and J. Zowe. Necessary and sufficient optimality conditions for a class of nonsmooth minimization problems.

Mathematical Programming, Vol. 24, 1982, pp. 70-91.

3. R. Fletcher and G.A. Watson. First and second order con- ditions for a class of nondifferentiable optimization prob- lems. Mathematical Programming, Vol. 18, 1980, pp. 291-307.

4. T.I. Sivelina. Minimizing a certain class of quasidiffer- entiable functions. Vestnik of Leningrad Univ., Vol. 7,

1983, pp. 103-105.

5. V.F. Demyanov and A.M. Rubinov. On quasidifferentiable

functionals. Soviet Math. Dokl., Vol. 21 (I), 1980, pp. 14-17.

6. V.F. Demyanov and L.N. Polyakova. Minimization of a quasi- differentiable function on a quasidifferentiable set. USSR Comput. Math. and !lath. Phys., Vol. 20(4), 1930, pp. 34-43.

7. V.F. Demyanov. Quasidifferentiable Functions: Necessary Conditions and Descent Directions. WP-83-64, International Institute for Applied Systems Analysis, Laxenburg, Austria, 1983.

8. A. Shapiro. On optimality conditions in quasidifferentiable optimization (forthcoming)

.