On Minimizing the Sum of a Convex Function and a Concave Function

NOT FOR QUOTATION WITHOUT PERMISSION OF THE AUTHOR

ON M I N I M I Z I N G THE SUM OF A CONVEX FUNCTION AND A CONCAVE FUNCTION

L,N, POLYAKOVA

J u n e 1 9 8 4 CP-84-28

CoZZaborative Papers r e p o r t work which h a s n o t been p e r f o r m e d s o l e l y a t t h e I n t e r n a t i o n a l I n s t i t u t e f o r A p p l i e d Systems A n a l y s i s and which h a s r e c e i v e d o n l y l i m i t e d r e v i e w , V i e w s o r o p i n i o n s e x p r e s s e d h e r e i n do n o t n e c e s s a r i l y r e p r e s e n t t h o s e o f t h e I n s t i t u t e , i t s - N a t i o n a l Member O r g a n i z a t i o n s , o r o t h e r o r g a n i - z a t i o n s s u p p o r t i n g t h e work,

INTERNATIONAL INSTITUTE FOR APPLIED SYSTEMS ANALYSIS A-2361 L a x e n b u r g , A u s t r i a

PREFACE

I n t h i s p a p e r , t h e a u t h o r p r e s e n t s a n a l g o r i t h m f o r m i n i - m i z i n g t h e sum o f a convex f u n c t i o n and a concave f u n c t i o n . The f u n c t i o n s i n v o l v e d a r e n o t n e c e s s a r i l y smooth and t h e r e - s u l t i n g f u n c t i o n i s q u a s i d i f f e r e n t i a b l e . The main p r o p e r t y of s u c h f u n c t i o n s i s t h e non-uniqueness of d i r e c t i o n s of s t e e p e s t d e s c e n t ( a n d a s c e n t ) , and t h e r e f o r e s p e c i a l p r e c a u t i o n s must be t a k e n t o g u a r a n t e e t h a t t h e a l g o r i t h m c o n v e r g e s t o a s t a t i o n a r y p o i n t .

T h i s .paper i s a c o n t r i b u t i o n t o r e s e a r c h on n o n d i f f e r e n t i a b l e o p t i m i z a t i o n c u r r e n t l y underway w i t h i n t h e System and D e c i s i o n S c i e n c e s Program.

A N D R Z E J W I E R Z B I C X I Chairman

System and D e c i s i o n S c i e n c e s

ON MINIMIZING THE SUM OF A CONVEX FUNCTION AND A CONCAVE FUNCTION

L.N. POLYAKOVA

D e p a r t m e n t o f A p p l i e d M a t h e m a t i c s , L e n i n g r a d S t a t e U n i v e r s i t y , U n i v e r s i t e t s k a y a n a b . 7 / 9 , L e n i n g r a d 199164, U S S R

Received 27 December 1 9 8 3 Revised 2 4 Rarch 1 9 8 4

We consider here the problem of minimizing a particular subclass of quasidifferentiable functions: those which may be represented as the sum of a convexfunctionanda

concave function. It is shown that in an n-dimensional space this problem is equivalent to the problem of

minimizing a concave function on a convex set. A successive approximations cethod is suggested; this

makes use-of'some of the principles of E-steepest-descent- type approaches.

Key w o r d s : Quasidifferentiable Functions, Convex Functions,

Concave Functions, E-Steepest-Descent Methods.

1

.

Introduction

The problem of minimizing nonconvex nondifferentiable func- tions poses a considerable challenge to specialists in mathe- matical programming. Most of the difficulties arise from the fact that there may be several directions of steepest descent.

To solve this problem requires both a new technique and a new approach. In this paper we discuss a special subclass of non- differentiable functions: those which can be represented in the form

where f is a finite function which is convex on En and f2 is a 1

finite function which is concave on E

.

Then f is continuous n

and quasidifferentiable on En

,

with a quasidifferential at x E En which may be taken to be the pair of sets

where

In other words, af(x) is the subdifferential of the convex

function f at x E En (as defined in convex analysis) and af (x) 1

is the superdifferential of the concave function f2 at x E En

.

Consider the problem of calculating

inf f (x)

.

xEEn

Quasidifferential calculus shows that for x E E tobe

*

a minimum n

point of f on En it is necessary that

We shall now show that the problem of minimizing f on the space En can be reduced to that of minimizing a concave function on a convex set.

L e t R d e n o t e t h e e p i g r a p h o f t h e convex f u n c t i o n f l

,

i . e . ,

R = e p i f = { z = [ x , ~ ] E En x E l h ( z )

-

^{f l ( X I}

^-

^{11 4 01 t}

and d e f i n e t h e f o l l o w i n g f u n c t i o n on En x E l :

S e t R i s c l o s e d and convex and f u n c t i o n $ i s q u a s i d i f f e r e n - t i a b l e a t any p o i n t z E E

n ^{x E l}

.

Take a s i t s q u a s i d i f f e r e n t i a l a t z = [ x , p ] t h e p a i r o f s e t s D $ ( z ) = [ { O }

,

a f 2 ( x ) ^x i l l ]

,

where 0 E E n + l

.

L e t u s now c o n s i d e r t h e problem o f f i n d i n g

I t i s well-known ( s e e , e . g . , [ 31) t h a t i f a concave f u n c t i o n a c h i e v e s i t s i n f i m a l v a l u e on a convex s e t , t h i s v a l u e i s a c h i e v e d on t h e boundary o f t h e s e t .

Theorem 1 . For a p o i n t x * t o b e a s o l u t i o n o f p r o b l e m ( I ) , i t i s b o t h n e c e s s a r y a n d s u f f i c i e n t t h a t p o i n t [ x * , p * ] be a s o l u - t i o n t o p r o b l e m ( 3 ) , w h e r e P * = f ( x * )

.

Proof

N e c e s s i t y . L e t x* b e a s o l u t i o n o f problem ( 1 )

.

^Then

11 + f ( x ) ? f l ( x ) + f 2 ( x ) > f l ( x * ) + f 2 1 x * ) V p 2 f l ( x ) , VxGEn= ( 4 ) 2

But ( 4 ) i m p l i e s t h a t

where p* = f l ( x * )

.

Thus t h e r e e x i s t s a z * = [ x * , p * ] E R such t h a t

$ ( z ) 1 )I ( z * ) Vz E R

.

T h i s p r o v e s t h a t t h e c o n d i t i o n i s n e c e s s a r y .

S u f f i c i e n c y . T h a t t h e c o n d i t i o n i s a l s o s u f f i c i e n t c a n b e proved i n a n a n a l o g o u s way by a r g u i n g Backwards from i n e q u a l i t y ( 5 ) .

2 . A n u m e r i c a l a l g o r i t h m

S e t E 2 0

.

^A p o i n t x o E E i s c a l l e d an E - i n f - s t a t i o n a r y n

p o i n t o f t h e f u n c t i o n f on En i f

where

i . e .

, a

^f ( x o ) i s t h e E - s u b d i f f e r e n t i a l o f t h e convex f u n c t i o n

'E

f l a t x O

.

F i x g E _En and s e t

Theorem 2. For a p o i n t xo t o be an E - i n f - s t a t i o n a r y p o i n t o f t h e f u n c t i o n f on E~

,

i t i s b o t h n e c e s s a r y and s u f f i c i e n t t h a t

a,£ (x,)

min a4 ^{2 0 .}

Il

g ll = 1

N e c e s s i t y . Let xo be an E-inf-stationary point of f on En

.

Then from (6) it follows that

Eience

min max ( z , ~ )

V w

sf(xol

,

I I ~ I I = I

z&+a,f(~~)

-

and thus for every g E En, lgll=l we have

min w a f (xo)

However, this means that

proving that the condition is necessary. That it is also suf- ficient can be demonstrated in an analogous way, arguing back- wards from the inequality ( 9 )

-

-6-

Note that since the mapping

is Hausdorff-continuous if E

>

0 (see, e.g., [I]), then the following theorem holds.

Theorem 3. If E

>

0 t h e n t h e f u n c t i o n max (v,g) i s con- vaaEf (x)

t i n u o u s i n x o n E _n f o r any f i x e d g E En

.

Assume that xo is not an E-in£-stationary point. Then we can describe the vector

aEf(xO) gE (x0) = arg min

Il

g ll = 1

a

⁹

as a direction of E-steepest-descent of function f at point

Xo

It is not difficult to show that the direction

where vOE E CEf(xO) ^WO E gf(x0) and

-

^{max min Ilv+wl = -llv + w}

^ll

= a E ( x o )

,

WET£ (x") fiaEf (x,) O E

o

is a direction of E-steepest-descent of function f at point x 0

-

Now let us consider the following method of successive ap- proximations.

F i x E

>

0 a n d c h o o s e an a r b i t r a r y i n i t i a l a p p r o x i m a t i o n x O ^E En

.

Suppose t h a t t h e Lebesque s e t

i s bounded. A s s u m e t h a t a p o i n t x E E~ h a s a l r e a d y b e e n f o u n d . If - a f ( x k )

-

^C a_,£ ( x k )

,

t h e n xk i s a n E - i n f - s t a t i o n a r y p o i n t of f o n E~ ; i f n o t , t a k e

X k+ 1 = ^Xk + a k g E k a k = a r g min f ( x k + a g

,

a 2 0 & k

where g E k = g , ( x k ) i s a n E - s t e e p e s t - d e s c e n t d i r e c t i o n o f f a t Xk ^'

Theorem 4 . T h e f o l l o w i n g r e l a t i o n h o l d s :

l i m a , ( x k ) = 0

.

k-

P r o o f . W e s h a l l p r o v e t h e t h e o r e m by c o n t r a d i c t i o n . Assume t h a t a s u b s e q u e n c e i x k 1 of s e q u e n c e { x k } a n d a number a>O e x i s t

S

s u c h t h a t

(The r e q u i r e d s u b s e q u e n c e must e x i s t s i n c e D ( x o ) i s c o m p a c t . ) W i t h o u t l o s s o f g e n e r a l i t y , w e c a n assume t h a t x -x

*

I: ks

( c l e a r l y , x ^E D ( x O ) ⁾

.

Then

where

The t e r m o ( a t g E k ) a p p e a r s i n t h e above e q u a t i o n due t o t h e con- s

c a v i t y of f 2

.

The f a c t t h a t f u n c t i o n f 2 i s concave i m p l i e s t h a t o ( a t g E k s ) 5 0 ' d a > O t WgEk E En t

S

and t h e r e f o r e

f ( x k + a g E k ) ( x k 1 +

la

max ( v t g E k ) d r

+

s s S O v E a f l (xk +Tg,k ) s

S S

+

^{a min} (wt g

6 a f 2 ( x k E k S S

S i n c e

a

f ( x ) 3 a f ( x ) f o r e v e r y x E En

,

^{we have}

E 1 1

max ( v t g ~ k s 2 max t

e a E f l

( x k +TgEk )

S S

and t h u s

f ( x k +ag s f ( x k )

+ l a

max ) d ~

+

s ^{E k S} s O v E a E f 1 ( x k + r g E k )

S S

+

a min

~ ~ ^{( X} a 1 f(w,gEks ks

S i n c e t h e mapping a E f l i s H a u s d o r f f - c o n t i n u o u s a t t h e p o i n t x

* ,

t h e r e e x i s t s a 6

>

⁰ s u c h t h a t

where S r ( z ) = {x E _{E n ] llx-zll} 6 r }

.

A l s o , t h e r e e x i s t s a number

K

>

⁰ s u c h t h a t

and hence

T h e r e f o r e

Inequality (10) contradicts the fact that sequence {f(xk)} is bounded, thus proving the theorem.

References

[I] E.A. Nurminski, "On the continuity of &-subgradient map- pings", C y b e r n e t i c s 5 (1977) 148-1 49.

[2] L.N. Polyakova, "Necessary condtitions for an extremum of a quasidifferentiable function", V e s t n i k L e n i n g r a d s k o g o

U n i v e r s i t e t a 13 ( 1 980) 57-62 (translated in V e s t n i k L e n i n g r a d U n i v . Math. 13 (1981) 241-247).

[3] R. T. Rockafellar

,

Convex Ana Z y s i s (Princeton University Press, Princeton, New Jersey, 1970).