NOT FOR QUOTATION WITHOUT PERMISSION OF THE AUTHOR
ON M I N I M I Z I N G THE SUM OF A CONVEX FUNCTION AND A CONCAVE FUNCTION
L,N, POLYAKOVA
J u n e 1 9 8 4 CP-84-28
CoZZaborative Papers r e p o r t work which h a s n o t been p e r f o r m e d s o l e l y a t t h e I n t e r n a t i o n a l I n s t i t u t e f o r A p p l i e d Systems A n a l y s i s and which h a s r e c e i v e d o n l y l i m i t e d r e v i e w , V i e w s o r o p i n i o n s e x p r e s s e d h e r e i n do n o t n e c e s s a r i l y r e p r e s e n t t h o s e o f t h e I n s t i t u t e , i t s - N a t i o n a l Member O r g a n i z a t i o n s , o r o t h e r o r g a n i - z a t i o n s s u p p o r t i n g t h e work,
INTERNATIONAL INSTITUTE FOR APPLIED SYSTEMS ANALYSIS A-2361 L a x e n b u r g , A u s t r i a
PREFACE
I n t h i s p a p e r , t h e a u t h o r p r e s e n t s a n a l g o r i t h m f o r m i n i - m i z i n g t h e sum o f a convex f u n c t i o n and a concave f u n c t i o n . The f u n c t i o n s i n v o l v e d a r e n o t n e c e s s a r i l y smooth and t h e r e - s u l t i n g f u n c t i o n i s q u a s i d i f f e r e n t i a b l e . The main p r o p e r t y of s u c h f u n c t i o n s i s t h e non-uniqueness of d i r e c t i o n s of s t e e p e s t d e s c e n t ( a n d a s c e n t ) , and t h e r e f o r e s p e c i a l p r e c a u t i o n s must be t a k e n t o g u a r a n t e e t h a t t h e a l g o r i t h m c o n v e r g e s t o a s t a t i o n a r y p o i n t .
T h i s .paper i s a c o n t r i b u t i o n t o r e s e a r c h on n o n d i f f e r e n t i a b l e o p t i m i z a t i o n c u r r e n t l y underway w i t h i n t h e System and D e c i s i o n S c i e n c e s Program.
A N D R Z E J W I E R Z B I C X I Chairman
System and D e c i s i o n S c i e n c e s
ON MINIMIZING THE SUM OF A CONVEX FUNCTION AND A CONCAVE FUNCTION
L.N. POLYAKOVA
D e p a r t m e n t o f A p p l i e d M a t h e m a t i c s , L e n i n g r a d S t a t e U n i v e r s i t y , U n i v e r s i t e t s k a y a n a b . 7 / 9 , L e n i n g r a d 199164, U S S R
Received 27 December 1 9 8 3 Revised 2 4 Rarch 1 9 8 4
We consider here the problem of minimizing a particular subclass of quasidifferentiable functions: those which may be represented as the sum of a convexfunctionanda
concave function. It is shown that in an n-dimensional space this problem is equivalent to the problem of
minimizing a concave function on a convex set. A successive approximations cethod is suggested; this
makes use-of'some of the principles of E-steepest-descent- type approaches.
Key w o r d s : Quasidifferentiable Functions, Convex Functions,
Concave Functions, E-Steepest-Descent Methods.
1
.
IntroductionThe problem of minimizing nonconvex nondifferentiable func- tions poses a considerable challenge to specialists in mathe- matical programming. Most of the difficulties arise from the fact that there may be several directions of steepest descent.
To solve this problem requires both a new technique and a new approach. In this paper we discuss a special subclass of non- differentiable functions: those which can be represented in the form
where f is a finite function which is convex on En and f2 is a 1
finite function which is concave on E
.
Then f is continuous nand quasidifferentiable on En
,
with a quasidifferential at x E En which may be taken to be the pair of setswhere
In other words, af(x) is the subdifferential of the convex
function f at x E En (as defined in convex analysis) and af (x) 1
is the superdifferential of the concave function f2 at x E En
.
Consider the problem of calculating
inf f (x)
.
xEEn
Quasidifferential calculus shows that for x E E tobe
*
a minimum npoint of f on En it is necessary that
We shall now show that the problem of minimizing f on the space En can be reduced to that of minimizing a concave function on a convex set.
L e t R d e n o t e t h e e p i g r a p h o f t h e convex f u n c t i o n f l
,
i . e . ,R = e p i f = { z = [ x , ~ ] E En x E l h ( z )
-
f l ( X I-
11 4 01 tand d e f i n e t h e f o l l o w i n g f u n c t i o n on En x E l :
S e t R i s c l o s e d and convex and f u n c t i o n $ i s q u a s i d i f f e r e n - t i a b l e a t any p o i n t z E E
n x E l
.
Take a s i t s q u a s i d i f f e r e n t i a l a t z = [ x , p ] t h e p a i r o f s e t s D $ ( z ) = [ { O },
a f 2 ( x ) x i l l ],
where 0 E E n + l
.
L e t u s now c o n s i d e r t h e problem o f f i n d i n g
I t i s well-known ( s e e , e . g . , [ 31) t h a t i f a concave f u n c t i o n a c h i e v e s i t s i n f i m a l v a l u e on a convex s e t , t h i s v a l u e i s a c h i e v e d on t h e boundary o f t h e s e t .
Theorem 1 . For a p o i n t x * t o b e a s o l u t i o n o f p r o b l e m ( I ) , i t i s b o t h n e c e s s a r y a n d s u f f i c i e n t t h a t p o i n t [ x * , p * ] be a s o l u - t i o n t o p r o b l e m ( 3 ) , w h e r e P * = f ( x * )
.
Proof
N e c e s s i t y . L e t x* b e a s o l u t i o n o f problem ( 1 )
.
Then11 + f ( x ) ? f l ( x ) + f 2 ( x ) > f l ( x * ) + f 2 1 x * ) V p 2 f l ( x ) , VxGEn= ( 4 ) 2
But ( 4 ) i m p l i e s t h a t
where p* = f l ( x * )
.
Thus t h e r e e x i s t s a z * = [ x * , p * ] E R such t h a t$ ( z ) 1 )I ( z * ) Vz E R
.
T h i s p r o v e s t h a t t h e c o n d i t i o n i s n e c e s s a r y .
S u f f i c i e n c y . T h a t t h e c o n d i t i o n i s a l s o s u f f i c i e n t c a n b e proved i n a n a n a l o g o u s way by a r g u i n g Backwards from i n e q u a l i t y ( 5 ) .
2 . A n u m e r i c a l a l g o r i t h m
S e t E 2 0
.
A p o i n t x o E E i s c a l l e d an E - i n f - s t a t i o n a r y np o i n t o f t h e f u n c t i o n f on En i f
where
i . e .
, a
f ( x o ) i s t h e E - s u b d i f f e r e n t i a l o f t h e convex f u n c t i o n'E
f l a t x O
.
F i x g E En and s e tTheorem 2. For a p o i n t xo t o be an E - i n f - s t a t i o n a r y p o i n t o f t h e f u n c t i o n f on E~
,
i t i s b o t h n e c e s s a r y and s u f f i c i e n t t h a ta,£ (x,)
min a4 2 0 .
Il
g ll = 1N e c e s s i t y . Let xo be an E-inf-stationary point of f on En
.
Then from (6) it follows that
Eience
min max ( z , ~ )
V w
sf(xol,
I I ~ I I = I
z&+a,f(~~)-
and thus for every g E En, lgll=l we have
min w a f (xo)
However, this means that
proving that the condition is necessary. That it is also suf- ficient can be demonstrated in an analogous way, arguing back- wards from the inequality ( 9 )
-
-6-
Note that since the mapping
is Hausdorff-continuous if E
>
0 (see, e.g., [I]), then the following theorem holds.Theorem 3. If E
>
0 t h e n t h e f u n c t i o n max (v,g) i s con- vaaEf (x)t i n u o u s i n x o n E n f o r any f i x e d g E En
.
Assume that xo is not an E-in£-stationary point. Then we can describe the vector
aEf(xO) gE (x0) = arg min
Il
g ll = 1a
9as a direction of E-steepest-descent of function f at point
Xo
It is not difficult to show that the direction
where vOE E CEf(xO) WO E gf(x0) and
-
max min Ilv+wl = -llv + wll
= a E ( x o ),
WET£ (x") fiaEf (x,) O E
o
is a direction of E-steepest-descent of function f at point x 0
-
Now let us consider the following method of successive ap- proximations.
F i x E
>
0 a n d c h o o s e an a r b i t r a r y i n i t i a l a p p r o x i m a t i o n x O E En.
Suppose t h a t t h e Lebesque s e ti s bounded. A s s u m e t h a t a p o i n t x E E~ h a s a l r e a d y b e e n f o u n d . If - a f ( x k )
-
C a_,£ ( x k ),
t h e n xk i s a n E - i n f - s t a t i o n a r y p o i n t of f o n E~ ; i f n o t , t a k eX k+ 1 = Xk + a k g E k a k = a r g min f ( x k + a g
,
a 2 0 & k
where g E k = g , ( x k ) i s a n E - s t e e p e s t - d e s c e n t d i r e c t i o n o f f a t Xk '
Theorem 4 . T h e f o l l o w i n g r e l a t i o n h o l d s :
l i m a , ( x k ) = 0
.
k-
P r o o f . W e s h a l l p r o v e t h e t h e o r e m by c o n t r a d i c t i o n . Assume t h a t a s u b s e q u e n c e i x k 1 of s e q u e n c e { x k } a n d a number a>O e x i s t
S
s u c h t h a t
(The r e q u i r e d s u b s e q u e n c e must e x i s t s i n c e D ( x o ) i s c o m p a c t . ) W i t h o u t l o s s o f g e n e r a l i t y , w e c a n assume t h a t x -x
*
I: ks
( c l e a r l y , x E D ( x O ) )
.
Thenwhere
The t e r m o ( a t g E k ) a p p e a r s i n t h e above e q u a t i o n due t o t h e con- s
c a v i t y of f 2
.
The f a c t t h a t f u n c t i o n f 2 i s concave i m p l i e s t h a t o ( a t g E k s ) 5 0 ' d a > O t WgEk E En tS
and t h e r e f o r e
f ( x k + a g E k ) ( x k 1 +
la
max ( v t g E k ) d r+
s s S O v E a f l (xk +Tg,k ) s
S S
+
a min (wt g6 a f 2 ( x k E k S S
S i n c e
a
f ( x ) 3 a f ( x ) f o r e v e r y x E En,
we haveE 1 1
max ( v t g ~ k s 2 max t
e a E f l
( x k +TgEk )S S
and t h u s
f ( x k +ag s f ( x k )
+ l a
max ) d ~+
s E k S s O v E a E f 1 ( x k + r g E k )
S S
+
a min~ ~ ( X a 1 f(w,gEks ks
S i n c e t h e mapping a E f l i s H a u s d o r f f - c o n t i n u o u s a t t h e p o i n t x
* ,
t h e r e e x i s t s a 6
>
0 s u c h t h a twhere S r ( z ) = {x E E n ] llx-zll 6 r }
.
A l s o , t h e r e e x i s t s a numberK
>
0 s u c h t h a tand hence
T h e r e f o r e
Inequality (10) contradicts the fact that sequence {f(xk)} is bounded, thus proving the theorem.
References
[I] E.A. Nurminski, "On the continuity of &-subgradient map- pings", C y b e r n e t i c s 5 (1977) 148-1 49.
[2] L.N. Polyakova, "Necessary condtitions for an extremum of a quasidifferentiable function", V e s t n i k L e n i n g r a d s k o g o
U n i v e r s i t e t a 13 ( 1 980) 57-62 (translated in V e s t n i k L e n i n g r a d U n i v . Math. 13 (1981) 241-247).
[3] R. T. Rockafellar