Singularity Theory for Nonlinear Optimization Problems

NOT FOR QUOTATION WITHOUT THE PERMISSION OF THE AUTHOR

m G U L A R m THEORY FVR NONLZNEAR OPITMIZATION PROBIXBB

J. C a s t i

November 1985 WP-85-79

W o r k i n g P c r p e r s are interim r e p o r t s on work of t h e International Institute f o r Applied Systems Analysis and h a v e r e c e i v e d only limited review. Views o r opinions e x p r e s s e d h e r e i n do not necessarily r e p r e s e n t t h o s e of t h e Institute o r of i t s National Member Organizations.

INTERNATIONAL INSTITUTE FOR APPLIED SYSTEMS ANALYSIS 2361 Laxenburg, Austria

Abstract

Techniques f r o m t h e t h e o r y of s i n g u l a r i t i e s of smooth mappings are employed t o study t h e r e d u c t i o n of n o n l i n e a r optimization p r o b l e m s t o s i m p l e r forms. I t is shown how s i n g u l a r i t y t h e o r y i d e z s c a n b e used t o : 1) r e d u c e decision s p a c e dimensionality; (2) t r a n s f o r m t h e c o n s t r a i n t s p a c e t o s i m p l e r f o r m f o r primal algo- r i t h m s ; (3) p r o v i d e s e n s i t i v i t y analysis.

SINGULARITY THEORY FOR NONLINEAR OPTIMIZATION PROBLJDIS

J. C a s t i

I. Background

Consider a smooth ( C m ) function f : R n ^-, R m a n d assume t h a t f h a s a c r i t i c a l point a t t h e o r i g i n , i . e . d f ( 0 )

=

0 . The t h e o r y of s i n g u l a r i t i e s as developed by Thorn, M a t h e r , Arnol'd a n d o t h e r s [I-31 a d d r e s s e s t h e following b a s i c questions:

A. What i s t h e l o c a l c h a r a c t e r of f in a neighborhood of t h e c r i t i c a l poir,t?

Basiczlly, t h i s questior, amounts t o a s k i n g "at what p o i ~ t i s i t s a f e t o t r u n c a t e t h e T a y l o r s e r i e s f o r f ?" This i s t h e d e t e r m i n a c y problem.

B. What a r e t h e "essential" p e r t u r b a t i o n s of f ? That i s , what p e r t u r b a t i o n s of f car, o c c u r which c h a n g e t h e q u a l i t a t i v e n a t u r e of f a n d which c a n n o t b e t r a n s f o r m e d away by a c h a n g e of c o o r d i n a t e s ? This is t h e u n f o l d i n g problem.

C. Can we classify t h e t y p e s of s i n g u l a r i t i e s which f c a n h a v e u p t o dif- feomorphism? This i s t h e c l a s s i p t c a t i o n problem.

E l e m e n t a r y c z t a s t r o p h e t h e o r y l a r g e l y s o l v e s t h e s e t h r e e p r o b l e m s (when m = I ) ; i t s g e n e r a l i z a t i o n t o s i n g u l a r i t y t h e o r y s o l v e s t h e f i r s t two, a n d g i v e s rela- tively complete information o n t h e t h i r d f o r m , n small. H e r e we outline a p r o g r a m f o r t h e utilization of t h e s e r e s u l t s in a n a p p l i e d s e t t i n g t o d e a l with c e r t a i n t y p e s of n o n l i n e a r optimization problems. In t h e following s e c t i o n we give a b r i e f sum- mary of t h e main r e s u l t s of s i n g u l a r i t y t h e o r y f o r p r o b l e m s A-C f o r j k t n c t i o n s ( m

=I)

and t h e n p r o c e e d t o a discussion of how t h e s e r e s u l t s may b e employed f o r n o n l i n e a r optimization.

II.

Determinacy. Unfoldings, and Classifications Equivalence of Germs

In i t s l o c a l v e r s i o n , e l e m e n t a r y c a t a s t r o p h e t h e o r y d e a l s with f u n c t i o n s f

:U+

R w h e r e

U

i s a neighborhood of 0 in R n . The c l e a n e s t way t o h a n d l e s u c h functions is t o p a s s t o germs, a g e r m being a class of functions which a g r e e o n s u i t a b l e n e i g h b o r h o o d s of 0. All o p e r a t i o n s on g e r m s a r e defined b y p e r f o r m i n g s i n i l a r o p e r a t i o n s on r e p r e s e n t a t i v e s of t h e i r c l a s s e s . In t h e sequel, we s h a l l usu- ally make n o distinction between a g e r m a n d a r e p r e s e n t a t i v e function.

W e l e t En b e t h e s e t of all smooth g e r m s R n ⁺ R , a n d l e t En, b e t h e set of all smooth g e r m s Rn ^-, R m . Of c o u r s e En,1

=

En

.

These sets are v e c t o r s p a c e s o v e r R , of infinite dimension. We a b b r e v i a t e ( z l ,

.. .

,zn ⁾ E Rn t o z

.

If f E En, t h e n

f

( 2 )

=

CPl(z),

- . .

, f m ( z ) ) and t h e f i are t h e components of f .

A d i f f e o m o r p h i s m g e r m 4p:Rn -+Rn s a t i s f i e s ~ ( 0 )

=

0 , a n d h a s a n i n v e r s e q' such thaL y ( y f ) ( z ) )

=

z

=

4p'(q(z)) f o r z n e a r 0. I t r e p r e s e n t s a smooth, i n v e r t i - b l e l o c a l c o o r d i n a t e c h a n g e . By t h e I n v e r s e Function Theorem, 4p i s a diffeomor- phism g e r m if a n d only if i t h a s a n o n z e r o J a c o b i a n , t h a t i s ,

Two g e r m s f , g : R n ^-+ R are r i g h t e q u i v a l e n t if t h e r e i s a diffeomorphism g e r m y a n d a coristant y E R s u c h t h a t

This i s t h e n a t u r a l e q u i v a l e n c e f o r studying topological p r o p e r t i e s of t h e g r a - d i e n t Of (Poston a n d S t e w a r t [4]). I f f , r a t h e r t h a n O f , i s i m p o r t a n t , t h e t e r m y i s omitte?.

A t y p e of g e r m is a r i g h t e q u i v a l e n c e c l a s s a n d t h e c l a s s i f i c a t i o n of g e r m s up to r i g h t e q u i v a l e n c e amounts t o a c l z s s i f i c a t i o n of t y p e s . E a c h t y p e f o r m s a s u b s e t of E n , a n d t h e c e n t r a l o b j e c t of s t u d y is t h e way t h e s e t y p e s f i t t o g e t h e r .

A p r e c i s e d e s c r i p t i o r , i s comp!icated by t h e f a c t t h a t most t y p e s h a v e infinite dimension; b u t t h e r e is a m e z s u r e of t h e corfiplexity of a t y p e , t h e c o d i m e n s i o n , which i s g e n e r a l l y f i n i t e . H e u r i s t i c a l l y , i t is t h e d i f f e r e n c e between t h e dimension of t h e t y p e a n d t h a t of En ( e v e n though b o t h are infinite). A p r e c i s e definition i s given below.

T h e l a r g e s t t y p e s h a v e codimension 0 and form o p e n sets in

En.

T h e i r boun- d a r i e s c o n t a i n t y p e s of codimension 1 ; t h e b o u n d a r i e s of t h e s e in t u r n c o n t a i n t y p e s of codimension 2, a n d so o n , with h i g h e r codimensions r e v e a l i n g p r o g r e s s i v e - ly m o r e complex t y p e s . T y p e s of infinite codimension e x i s t , b u t f o r m a v e r y small s e t in a r e a s o n a b l e s e n s e .

Codimension and the Jacobian ideal

Let En b e t h e set of g e r m s R n ^-+ R , a n d let

F

b e t h e s e t of f o r m a l p o w e r s e r i e s in zl,

...,

z,. T h e r e i s a map j:E + F d e f i n e d by

w h e r e t h e r i g h t - h a n d s i d e i s t h e T a y l o r s e r i e s , or j e t , o f f . Note t h a t i t e x i s t s as a f o r m a l power s e r i e s f o r al! smooth f : c o n v e r g e n c e i s n o t r e q u i r e d in what follows.

T h e map j i s o n t o , l i n e a r o v e r R , and p r e s e r v e s p r o d u c t s (i.e., j C P . 9 ) = j u g )

=

( j f . j g ) ) .

L e t mn b e t h e s e t of f E En s u c h t h a t f (0)

=

0 . This is a n i d e a l of En (mean-' ing t h a t if f E mn a n d g E E ther, f g E mn , which we w r i t e b r i e f l y a s m n E n C m,).

I t s k t h p o w e r _m,

'k

c o n s i s t s of all f e E n s u c h t h a t 0

=

f (0)

=

df (0.)

=

dZf (0)

= - =

dk-If (0). In p a r t i c u l z r , f is a s i n g u l a r i t y if a n d only i f f E m,

.

The i d e a l s mk form a d e c r e a s i n g s e q u e n c e .

T h e r e is a s i m i l a r c h a i n in Fn.

Let

M'k ⁼

^{j ( m ) :} t h i s is t h e set of f o r m a l power s e r i e s with z e r o c o x s t a n t t e r m . Then h&

=

j(m,) is t h e

?

set of formal p o w e r series without t e r m s of d e g r e e S k -1

.

The i n t e r s e c t i o n of a l l M: i s 0 ; t h e i n t e r e s e c t i o n of a l l mk is t h e set m," of f l a t g e r m s , having z e r o Taylor s e r i e s .

The J a c o b i a n i d e a l of a s i n g u l a r i t y f i s t h e set of 211 g e r m s e x p r e s s i b l e in t h e form

f o r a r b i t r a r y g e r m s gi

.

W e d e n o t e i t by A V ) , o r merely A when f i s u n d e r s t o o d . I t s image j ACf) C Fn h a s a n analogous definition, w h e r e t h e p a r t i a l d e r i v a t i v e s a r e defined formally. S i n c e f i s a s i n g u l a r i t y , A V )

c

m,. T h e c o d i m e n s i o n of f i s defined t o b e

cod C f )

=

dimRmn / AV).

Similarly, at t h e formal power s e r i e s l e v e l , we define

The codimension of a n o r b i t i s t h e same as t h a t of i t s t a n g e n t s p a c e T. This i s t h e same as t h e dimension of t h e q u o t i e n t v e c t o r s p a c e E/T. In E n , t h e analog of t h i s t a n g e n t s p a c e i s t h e Jacobian i d e z l , s o t h e codimension shon!d b e d i m En / A V ) . This m e z s u r e s t h e number of independent d i r e c t i o n s in En "missing" f r o m ACf) , o r equivalently missing from t h e o r b i t o f f

.

The computation of cod C f ) i s e f f e c t e d by means of t h e following r e s u l t : if ei- t h e r cod Cf ) o r cod (j f ) i s f i n i t e t h e n s o i s t h e o t h e r , a n d t h e y a r e e q u a l . Thus, t h e computation may b e c a r r i e d o u t o n t h e formal power s e r i e s l e v e l w h e r e i t i s a com- b i n a t o r i a l calculation. F o r e x a m p l e s ir, c l a s s i c a l notation, see P o s t o n a n d Stewart [&>.

Determinacy

L e t f e E n , and d e f i n e t h e k - j e t j k V ) t o b e t h e Tzylor s e r i e s of f u p t o and including t e r m s of o r d e r k . F o r e x a m p l e ,

W e s a y t h a t f i s k - d e t e r m i n a t e ( o r k-determined) if f o r a n y g e En s u c h t h a t j k g

=

jk f , i t follows t h a t g i s r i g h t equivalent t o f .

A germ i s 1-determined if- i t s l i n e a r p a r t i s n o n z e r o , t h a t i s , i t s d e r i v a t i v e d o e s n o t vanish. S o t h e non-1-determined germs a r e t h e singularities: If f i s a s i n g u l a r i t y a n d f (0)

=

0 ( a s we c a n assume) t h e n t h e s e c o n d d e r i v a t i v e g i v e s t h e 2-jet o f f in t h e f o r m

j 2

f

( z l n - - - p z n )

=

~ i , j ~ i j z i z ' w h e r e t h e Hessian m a t r i x

i s symmetric. I t c a n b e shown t h a t f is 2-determined if a n d only if d e t (H) # 0 ; in t h i s c a s e f i s r i g h t equivalent to

This i s a r e f o r m u l a t i o n i n d e t e r m i n a c y t e r m s of t h e Morse Lemma (Milnor

151).

A g e r m equivalent t o (*) i s s z i d t o b e Morse. Morse g e r m s are p r e c i s e l y t h o s e of codimension 0 . T h e number L of n e g a t i v e signs in (*) is t h e i n d e z of f , a n d f i s a n 1-saddle. Morse t h e o r y (Milnor [5]) d e s c r i b e s t h e g l o b a l p r o p e r t i e s of a function

f : X

-. R

w h e r e X i s a smooth manifold, and f h a s only Morse s i n g u l a r i t i e s . (See Casti [9] f o r more d e t a i l s ) .

T h e r e e x i s t r u l e s f o r computing t h e d e t e r m i n a c y of a given germ: a n e a s y n e c e s s a r y condition, a n e a s y ( d i f f e r e n t ) s u f f i c i e n t condition, a n d a h a r d e r necessary-and-sufficient condition.

Let A b e t h e J a c o b i a n i d e a l of f

.

Then:

(i) If mk

c

m, A t h e n f i s k-determined.

(ii) If f i s k - d e t e r m i n e d t h e n mk s m, A.

(iii) f i s k - d e t e r m i n e d if a n d only if rn:

s

m, A(J +g ) f o r a l l g E m:

.

T h e r e i s z slightly s t r o n g e r f o r m of (i), namely ( i s ) If mk+l

s mi^

^{t h e n f} i s k-determined.

Nurcerous examples in P o s t o n a n d S t e w a r t [4] and Gibson [3] show how t o com- p u t e t h e determinacy of a given f . F o r example, s u p p o s e f i s in Morse f o r m (*).

the^ A

= <

+ 2 z l , . .

.

,

+

22,

> =

m, a n d m:

=

m, A. By (i), f is 2-determined as as- s e r t e d a b o v e .

A g e r m in f i n i t e l y determined if i t i s k - d e t e r m i n e d f o r some f i n i t e k

.

The fol- lowing are equivalent:

(iv) f h a s f i n i t e codimension (v) f i s finitely d e t e r m i n e d (vi) m i

s

A f o r some t

.

The solution t o t h e Determinacy Problem i s t h u s t h a t i t i s s a f e (up t o r i g h t equivalence) t o t r u n c a t e a k - d e t e r m i n e d germ at d e g r e e k of i t s Taylor s e r i e s . F o r a germ s u c h as zZy ^E E2, which is not finitely d e t e r m i n e d , i t is not s a f e t o t r u n c a t e h i g h e r o r d e r p e r t u r b i n g terms (and indeed z 2 y t y t h a s a t y p e t h a t d e p e n d s on t ) . Germs t h a t are not finitely d e t e r m i n e d e i t h e r a r i s e in a c o n t e x t w h e r e some symmetry is a c t i n g (and should b e analyzed by methods similar t o t h o s e a b o v e b u t which t a k e symmetry i n t o a c c o u n t

-

which c a n b e done) o r must b e viewed with suspicion. By (iv), w e may summarize: "nice" g e r m s h z v e f i n i t e codi- mension.

S u p p o s e t h a t f is n o t 2-determinate, s o t h a t d e t (H)=O. L e t t h e r a n k of t h e matrix

H

b e r and c a l l n -r i t s c o r a n k . A useful r e s u l t . c a l l e d t h e S p l i t t i n g Lem- ma, s z y s t h a t f is r i g h t equivalent t o a germ of t h e f o r m

F o r many p u r p o s e s , t h e q u a d r a t i c t e r m s may b e i g n o r e d . So t h e Splitting Lemma r e d u c e s t h e e f f e c t i v e n u m b e r of v a r i a b l e s t o n --r. A simple proof f o r f i n i t e di- mensions i s in Poston a n d S t e w a r t 141.

The determinacy c a l c u l a t i o n s , a n d t h e a p p l i c a t i o n of t h e Splitting Lemma, may b e c a r r i e d o u t equally well on j k f in

F,

, provided t h e codimensior, of f i s finite.

The formal power s e r i e s s e t t i n g i s b e t t e r f o r computations.

Unfoldings

An unfolding of a s i n g u l a r i t y i s a "parametrized family of p e r t u r b a t i o n s . " The notior. i s useful mainly b e c a u s e , f o r f i n i t e codimension s i n g u l a r i t i e s , t h e r e e x i s t s a

"universal unfolding" which i s a s e n s e c a p t u r e s a l l possible unfoldings.

More r i g o r o u s l y , l e t f c E n . Then ar: I - p a r a m e t e r u n f o l d i n g of f i s a germ F ' F n + i l t h a t i s , a real-valued g e r m of a f u n c t i o z F ( z l ,..., z,. E ~ , .

. .

, E ~ )

=

F ( x , E ) , such t h a t F ( z , O )

=

f ( 2 ) .

An unfolding F i s i n d u c e d from F if F ( z , 6 )

=

F ( P ~ ( z ) ~ # ' ( ~ ) ) + ~ ( 6 ) w h e r e

6 = ( d l , .

. .

,6,) c R m p6:Rn ⁺ R n

$:Rm -, R 1 7:RL -, R.

Two ur,fo!dings a r e e q u i v a l e n t if e a c h czr. b e induced f r o x t h e o t h e r . An 1- p a r a m e t e r unfolding i s v e r s a 1 if all o t h e r unfoldizgs c a n b e induce2 from it;

u n i v e r s a l if in addition, 1 i s as small a s possible.

S u p p o s e t h a t f h a s f i n i t e codimension c . Let u l ,

. . .

, u c b e a b a s i s f o r mn / ACf). Then i t i s a t h e o r e m t h a t z u n i v e r s a l u n f o l d i n g i s given by t h e germ

F ( z ) , E )

=

f ( z )

+

e l u l ( z )

+ - . +

c C u c ( z ) , c i E R

(**>

While d i f f e r e n t c h o i c e s of t h e ui c a n b e made, a universzl unfolding i s unique u p t o e q u i v a l e n c e . The e x i s t e n c e of u n i v e r s a l unfoldings in finite codimension, and t h e method f o r computing them, is p r o b a b l y t h e most significant a n d u s e f u l r e s u l t in e l e m e n t a r y c a t a s t r o p h e t h e o r y . (Note t h a t (**) i s l i n e a r in t h e unfolding v a r i a b l e s

E

.

This i s a t h e o r e m , and i s n o t built i n t o t h e definition of a n unfolding.)

F o r example, if f ( z , y ) = z 3 + y 4 , t h e n a b a s i s f o r m 2 / A C f ) i s Iz , y , z y , y 2,zy

1.

S o a u n i v e r s a l unfolding i s given by

T h e codimension of a g e r m f h a s s e v e r z l i n t e r p r e t a t i o n s : (i) T h e codimension of t h e J a c o b i a n i d e a l in mn ,

(ii) T h e number of independent d i r e c t i o n s "missing" from t h e o r b i t of f , (iii) The number of p a r a m e t e r s in any u n i v e r s a l unfolding of f

.

In addition, if t h e codimension of f i s c , i t c a n b e shown t h a t a n y small p e r t u r - b a t i o n of f h a s at most c

+I

c r i t i c a l points.

Classification

W e s k e t c h how t h e s e i d e a s may b e used t o classify g e r m s of codimension at most 4.

L e t f e En

.

If f i s n o t a s i n g u l a r i t y t h e n f ( z ) i s r i g h t equivalent t o z l . If f i s a s i n g u l a r i t y , a n d i t s Hessian h a s n o n z e r o d e t e r m i n a n t , t h e n f i s r i g h t e q u i v a l e n t t o

*

²¹²

* ^{. . .} *

^z:

^.

Otherwise, d e t (H)=O. Let k =n -r b e t h e c o r a n k of H , a n d s p l i t f as

I t c a n b e p r o v e d t h a t t h e c l a s s i f i c a t i o n of possibilities f o r f d e p e n d s only on t h e similar classification f o r g .

The Taylor s e r i e s of g begins with c u b i c o r h i g h e r t e r m s . F i r s t s u p p o s e t h a t k =I, a n d l e t t h e f i r s t n o n z e r o jet of g b e a t z t

.

This i s t - d e t e r m i n e d , a n d s c a l e s t o 5 z ( t e v e n ) , z t ( t odd). The codimension i s t -2, s o t

=

3 , 4, 5 o r 6.

Next, l e t k =2, and l e t

By a l i n e a r c h a n g e of v a r i a b l e , t h i s c u b i c may b e b r o u g h t t o t h e f o r m z 3

+

^{z y 2}

(one r e a l r o o t ) , z 3

-

^zy2 ( t h r e e d i s t i n c t r e a l r o o t s ) , z2y ( t h r e e r e a l r o o t s , o n e r e p e a t e d ) , z 3 ( t h r e e r e a l r o o t s , a l l r e p e a t e d ) , o r 0 .

The f o r m s z 3 i zy2 a r e 3-determined, and of codimension 3.

The f o r m zZy i s not 3-determined, s o w e corisider h i g h e r t e r m s . A s e r i e s of c h a n g e s of v a r i a b l e b r i n g any h i g h e r o r d e r expznsion t o t h e form z2y

+

^{y t , which}

i s t-determined and of codimension t

.

Only t =4 i s re1evar.t t o o u r problem h e r e . No h i g h e r t e r m added t o z 3 p r o d u c e s a codimension 4 r e s u l t ; z n d no h i g h e r t e r m a d d e d t o 0 does.

Finally, let k r 3. Then t h e codimension c a n b e p r o v e d t o b e at least 7, s o t h i s c a s e d o e s not a r i s e .

Thus, w e h a v e classified t h e g e r m s of codimension 5 4 i n t o t h e canonical f o r m s

z: + (MI

2 1 3 2,222 + (N)

213

+

zlz;

+

(N)

213

+

224

+

^(N)

w h e r e

(M)

=

5222

+ . . .

i z , , 2 (N)

=

+ z 3 2 - c . . . i z n . 2

The c e l e b r a t e d e l e m e n t a r y c a t a s t r o p h e s of Thom are t h e u n i v e r s a l unfoldings of t h e s i n g u l a r i t i e s o n t h i s l i s t , o r i t s extension t o h i g h e r codimensions. The u n i v e r s a l unfolding arises when we t r y t o classify not g e r m s , b u t I - p a r a m e t e r fami- l i e s of germs. F o r I 5 4, "almost all" such a r e given b y u n i v e r s a l unfoldings of g e r m s of codimension S4.

Table 1 s u m x a r i z e s t h e l i s t of g e r m s and t h e i r unfoldings u p t o codimension 5 , to- g e t h e r with t h e i r customzry name a n d symbol in t h e systemztic n o t z t i o n of Arnol'd [I].

The t e r m s

(M)

and (N) a r e omitted f o r c l a r i t y , x a n d y r e p l z c e z l a n d z2; a n d unfolding p a r a m e t e r s are listed as ( a , b , c , d , e ) r a t h e r t h a n ( c ~ , c ~ , c ~ , E ~ , E ~ ) .

TABLE 1: T h e e l e m e n t a r y c a t a s t r o p h e s of codimension 5 5

.

When t h e

+

s i g n o c c u r s , g e r m s with s i g n (+) a r e c a l l e d s t a n d a r d , (-) a r e c a l l e d d u a l .

symbol

A 2

+ A 3 A 4

3 4 5

A6

D

i-

Di+

* 5

D 6 D6C

*6

name f o l d c u s p swaLlowt.ai1 butt.erfly wigwam

e l l i p t i c umbilic h y p e r b o l i c umbilic p a r a b o l i c umbilic s e c o n d e l l i p t i c umbilic s e c o n d h y p e r b o l i c umbilic s y m b o l i c umbilic

g e r m

- - -. . - . -- - . - - -

u n i v e r s a l unfolding

x 3 + a x

CO-

r a n k

codi- mension

1 2

3 4

5 3

3 4

5

5

T h e a b o v e s k e t c h shows how t h e classification p r o b l e m r e d u c e s to t h e d e t e r m i n a c y a n d unfoldfng p r o b - l e m s ( a n d i s r e l a t i v e l y e a s y o n c e t h e s e a r e s o l v e d ) . In a p p l i c a t i o n s , t h e main i n f l u e n c e of t h e c l a s s i f i c a t i o n i s a n o r g a n i z i n g o n e : t h e d e t e r m i n a c y a n d unfolding t h e o r e m s p l a y a m o r e d i r e c t role.

I I t .

Singularity Theory and Nonlinear Programming We c o n s i d e r t h e problem

max

f

( z o v e r a l l z E

R n

s u c h t h a t

Q ( Z ) 0 ,

w h e r e f ,g E m n

.

T h e r e a r e at least t h r e e d i f f e r e n t a s p e c t s of t h i s s t a n d a r d nonlinear optimization problem which s i n g u l a r i t y t h e o r y c a n s h e d some l i g h t upon: ( 1 ) r e d u c t i o n of dimensionality in t h e d e c i s i o n s p a c e f o r dual, p e n a l t y , a n d b a r r i e r t y p e algorithms [?I; (2) t r a n s f o r m a t i o n of t h e c o n s t r a i n t s p a c e i n t o s i m p l e r f o r m f o r primal t y p e algo- r i t h m s [?I a n d (3) s e n s i t i v i t y a n a l y s i s . Let u s examine e z c h of t h e s e areas in t u r n .

Dimensionality Reduction and the Splitting Lemma

If t h e optimization p r o b l e a (1)-(2) i s t o b e z p p r o a c h e d using o n e of t h e dual penal- t y , o r b z r r i e r a l g o r i t h ~ s [?], t h e Splitting Lemma c a n b e used t o r e d u c e t h e dirne~sior.

of t h e decisior: v e s t o r in t h e s u r r o g a t e objective functior,. F o r example, c o n s i d e r t h e augmented Lagrangian method, f o r which t h e s u r r o g a t e o b j e c t i v e function i s

w h e r e a is a v e c t o r of multipliers a n d p i s some positive c o n s t a n t . The p z r a m e t e r s a a r e updated a c c o r d i n g t o , s a y , t h e augmented Lagrangian scheme of Hestenes.

Assume t h a t t h e c r i t i c a l p o i n t of G i s located at z

=

^{z m , a}

=

a m , a n d t h a t t h e c o r a n k of G ( z , a )

=

r. Then t h e Splitting L e ~ m a i n s u r e s t h a t t h e r e e x i s t c o o r d i n a t e t r a n s f o r m a t i o n s z -,

5 ,

a -,

6

s u c h t h a t G -, G , w h e r e

w h e r e c

=

codim G while G I ( - ) i s a function O ( ( z j 3, , which i s l i n e a r in Gl,

.

.

.

^{, a,}

.

The function M ( - ) i s a p u r e q u a d r a t i c . The important point h e r e i s t h a t usuzlly r C C n , which implies t h a t most of t h e computational work i s involved in minimizing t h e q u a d r a t - i c

M ,

which c a n b e d o n e v e r y e f f i c i e ~ t l y by any of a number of quasi-Newton schemes.

The essentially n o n l i n e a r p a r t of t h e problem involves t h e minimization of G , which, however, involves only r v a r i a b l e s . Often r

=

1 o r 2, e v e n if n i s v e r y l a r g e , s a y , h u n d r e d s , s o t h e computational savings c a n b e significant.

The p o t e n t i a l d r a w b a c k t o t h e a b o v e scheme i s t h z t in o r d e r t o compute r , t h e c o r a n k of G , we need t o know t h e Hessian

at t h e c r i t i c a l point ( z m , a o ) . S i n c e i t i s p r e c i s e l y z * which w e s e e k , i t a p p e a r s at f i r s t g l a n c e t h a t t h e s i t u z t i o n i s n o t t o o promising. However, t h i s problem c a n b e c i r - cumvented in a t l e a s t two d i f f e r e n t ways:

(i) Often i t c a n b e s e e n t h a t t h e Hessian will b e of c o n s t z n t r a n k in some neigh- b o r h o o d

D

of z m , e v e n if w e don't know z ' e x a c t l y . This s i t u a t i o n comes a b o u t s i n c e we usually h a v e a t l e a s t some i d e a of t h e region

D

containing z '

.

Thus, if we h a v e a n e s t i m a t e of

D

a n d know t h a t r a n k H ( z , a )

=

c o n s t a n t f o r a l l z E

D ,

t h e n we c a n u s e t h i s information in a s u c c e s s i v e approximatior. scheme g e n e r a t i n g a s e q u e n c e zn -, z The idea i s t o a p p l y t h e S p l i t t i n g Lemma t o e a c h a p p r o x i m a t e problem at t h e point zn

.

(ii) if t h e r e i s n o informztion a b o u t t h e r a n k H, t h e n w e car, a p p e z l t o t h e inequa!- ity

r ( r +1)/2 S codim G ,

which always holds. W e c a n t a k e a pessimistic e s t i m a t e of r which, at w o r s t , means only t h a t w e include a few more v a r i a b l e s in o u r nonlinear optimizatior, of GI(.) t h a n might h a v e b e e n n e e d e d . If codim G S 2 , t h e n we c a n see from t h e inequality t h a t r =1 a n d t h e r e is only a single e s s e z t i a l , nonlinear v z r i a b l e , r e g a r d l e s s of w h e r e z ' i s l o c a t e d . Otherwise t h e r e mzy b e s e v e r a l nonlinear v a r i a b l e s , but t h e number will s t i l l b e s e v e r e l y limited b y t h e a b o v e inequality.

An e s s e n t i a l i n g r e d i e n t in making t h e a b o v e scheme work in p r a c t i c e i s t h e e z s e of determining t h e c o o r d i n a t e t r a n s f o r m a t i o n s z -,

2 , a

^-r

6 _.

_{A s} noted in S e c t i o n 11, t h e t h e o r y g u a r a n t e e s s u c h t r a n s f o r m a t i o n s e x i s t a n d , m o r e o v e r , t h a t t h e y are t h e m s e l v e s d i f f e o r ~ o r p h i s r n s . Thus, t h e c o o r d i n a t e c h a n g e s

h a v e c o n v e r g e n t power s e r i e s e_xpansions. Consequently, s i n c e we know t h e o r i g i n a l form of G a n d i t s normal form G , in p r i n c i p l e w e c a n s u b s t i t u t e t h e a b o v e e x p a n s i o n s a n d match c o e f f i c i e n t s in o r d e r t o d e t e r m i n e t h e e x p l i c i t form of t h e t r a n s f o r m a t i o n s . The o p e r a t i o n a l implementation of t h i s i d e a , however, may r e q u i r e a s u b s t a n t i a l amount of z l g e b r a , depending upon t h e e x a c t n a t u r e of G.

Simplifying the Constraint Space

F o r n o n l i n e a r c o n s t r a i n e d optimizatioc problems hzving nonlinear c o n s t r z i c t sets, t h e c o o r d i n a t e c h a n g e s discussed a b o v e cac b e employe? t o "straighten-out" t h e bind- ing c o n s t r a i n t s in a neighborhood of r e g u l a r points, s o t h a t primzl methods f o r solving c o n s t r a i n e d optimization problems c a n b e used, dealing only with l i n e a r s i d e con- s t r a i n t s . The e s s e n c e of t h e primal methods i s t o start with a f e a s i b l e d i r e c t i o n a l o n g which t h e o b j e c t i v e functior, i s improving. A one-dimensional line s e a r c h ( i n t e r v a l b i s e c t i o n , Newton's method, e t c . ) i s t h e n u s e d t o solve t h e one-dimensional optimization p r o b l e ~ a l o n g t h e improving f e a s i b l e d i r e c t i o n , c o n s t r a i n e d s o t h a t t h e r e s u l t i n g solu- tion r e m a i n s f e a s i b l e 173.

A s p e c i f i c e x a m p l e ,of such 2 primal method i s t h e g r a d i e n t p r o j e c t i o n t e c h n i q u e d u e t o Rosen. This method g e n e r a t e s a n improving f e a s i b l e d i r e c t i o n b y p r o j e c t i n g t h e n e g a t i v e of t h e g r a d i e n t v e c t o r of f o n t o t h e a f f i n e s u b s p a c e determined b y t h e i n t e r - s e c t i o n of t h e binding c o n s t r a i n t s , assuming t h e c o n s t r a i n t s a r e l i n e a r . A p r o j e c t i o n m a t r i x P i s f o r m e d f r o m a s u i t a b l e l i n e z r combination of t h e normal v e c t o r s of t h e con- s t r a i n t s u b s p a c e s (i.e. t h e g r a d i e n t s of t h e binding c o n s t r a i n t s ) . The r e s u l t i n g one- dimensional optimization is t h e n g u a r a n t e e d t o remain f e a s i b l e as long as a s u i t a b l e u p p e r bound i s o b s e r v e d on t h e line s e a r c h [7].

In t h e e v e n t t h e c o n s t r a i n t s are n o n l i n e a r , t h e g r a d i e n t of f i s p r o j e c t e d o n t o t h e i n t e r s e c t i o n of t h e t a n g e n t s p a c e s t o t h e binding c o n s t r a i n t s , s o t h a t movement a l o n g t h e improving f e z s i b l e d i r e c t i o n will, in genera!, t a k e t h e solution oxtside t h e f e a s i b l e r e g i o n ( s e e F i g u r e 1 ) . This n e c e s s i t a t e s a c o r r e c t i o n move t o b r i n g t h e solutions b a c k i n t o t h e f e a s i b l e r e g i o n s a f t e r t h e one-dimensional s e a r c h h z s been completed. Singu- l a r i t y t h e o r y a p p e a r s t o o f f e r t h e possibility of materially improving t h e a b o v e p r o - c e d u r e as we now i n d i c a t e .

move

FIGURE ¹ P r o j e c t e d gradient method of Rosen f o r nonlinear c o n s t r z i n t s (From Figure 10.5, pg. 398 Bazaraa and S h e t t y , 1979)

Consider t h e following nonlinear programming problem:

minimize:

f ( z )

s u b j e c t to: gi ( z ) 5 0 i =1,2, ..., m z ^{2 0}

For any x such t h z t z r 0 , if I

=

[ i : g i ( z )

=

O j , t h e n X

=

[z :gi ( x )

=

0 j

= n

(gi ( 2 )

n ^{R n}

h y p e r p l a n e )

i f 1

will b e t h e i n t e r s e c t i o n of a f i n i t e number of manifolds in

R n

and t h u s , with t h e possi- ble exclusion of a set of points of codimension n , ( c o r n e r s ) will i n h e r i t t h e manifold s t r u c t u r e locally. Loczlly, t h e n , a c o o r d i n a t e change could b e e f f e c t e d in X which will c a u s e X t o t a k e t h e form:

X

-, Y

=

ly :O

=

^a'y

+

^{c ,} a ,c constant v e c t o r s j

as long a s t h e g r a d i e n t s of t h e binding c o n s t r a i n t s don't vanish. A t r z n s v e r s a l i t y a r g u - ment c a n b e used t o r u l e out t h e l a t t e r possibility.

Assuming t h a t only t h e c o n s t r a i n t s g i ( 2 )

=

0 is binding, l e t Si

=

T, gi ( z )

n ^Rn

h y p e r p l a n e ,

where

T, gi ( z )

=

tangent s p a c e t o g i at x.

S i n c e codim Tzgi ( z )

=

1 and codim

IRn

h y p e r p l z n e j

=

1, if t h e i n t e r s e c t i o n i s t r a n s v e r s e

coc5im Tz g i ( 2 )

+

^codim

^{[ R n}

^-I h y p e r p l a n e j = codim Si

=

2

Results f r o m d i f f e r e n t i a l topology assert t h a t t h e set of c r i t i c a l p o i n t s Ri f o r gi will b e i s o l a t e d , t h u s t h e dim

Ri =

0 a n d codim Ri

=

n. T h e r e f o r e ,

codim

Ri +

codim Si

=

n +2

>

n.

S o , f o r Vgi ( 2 ) t o b e z e r o zt e x a c t l y t h e same points w h e r e gi ( z ) = 0 c o n s t i t u t e s a non- t r a n s v e r s e i n t e r s e c t i o n a n d i s t h e r e f o r e non-generic. If a n y s u c h p o i n t s should o c c u r , t h e y will b e isolated a n d t h u s not form a c o n s t r a i n t boundary.

In p r a c t i c e , finding X a n d t h e c o o r d i n a t e t r a n s f o r m a t i o n n e c e s s a r y t o make i t look l i k e Y usually r e q u i r e s some e f f o r t . However, if p r o j e c t i o n o n t o only o n e binding con- s t r a i n t i s n e c e s s a r y , t h e c a l c u l z t i o n becomes s i m p l e r , as t h e following e x a m p l e shows:

min f (zi , z 2 )

=

1 / 2 2 ;

+

1 / 2 2 ;

-

z1-z2 ( t h e g e o m e t r y in x s p a c e i s shown in F i g u r e 2) s u b j e c t t o

z f + z ; - l

s

0

- 2 1 5; 0 - 2 2

s

0

Vp

( 2 )

=

(2,-l,z2-1) at (1,O): Vf (1,O)

=

(0,-1)

vg1(2

=

(22:,2z2) Vg 1(1,0)

=

(2,O)

-

binding

Vg2(z)

=

(-1,O) , Vg2(1,0)

=

(-130)

v g 3 ( z )

=

(0, -1) Vg &,O)

=

(0, -1)

-

binding

As c a n b e s e e n , w e want t o p r o j e c t o n t o g l ( z )

.

To s t r a i g h t e n o u t g l , l e t

2 - 2

y l = z l , y 2 - z 2 . In t h e n e w c o o r d i n a t e s , VfneWwiLl b e :

V f ,,,(Y)

=

(Y

p

^{- 1 , ~}

^$

^{-I), Vf} ",,(l.O)

=

(0, -1)

(Note: This is not t h e g r a d i e n t of t h e t r a n s f o r m e d o b j e c t i v e function b u t r a t h e r t h e t r a n s f o r m e d g r a d i e n t of t h e old o b j e c t i v e functior,.)

The new problem is :

m i n Z ( ~ ~ , ~ ~ ) = 1 / 2 y l + 1 / 2 y 2 - Y ? - Y $ ( t h e g e o m e t r y in y s p a c e in shown in Figure 3) s u b j e c t t o

y 1 + y 2 - 1

s

0 - Y l

s

0 - y 2 5 ; 0

Now t h e c o n s t r a i n t is l i n e a r a n d we p r o j e c t

Vp,,,

o n t o g1 by forming t h e p r o j e c t i o n m a t r i x :

FIGURE 2 Configuration in x s p a c e

The o S j e c t i v e functior, i s optimized along t h e c o n s t r a i n t by l e t t i n g

S o t h e minimum i s t a k e n on at

T h a t t h i s i s t h e optimum c a n b e s e e n by t r y i n g to form a n improving f e a s i b l e d i r e c t i o n in

z

s p a c e . T h e r e s u l t w i l l b e t h e z e r o v e c t o r , indicating t h a t t h e optimum h a s b e e n r e a c h e d .

v ~ ( z ) = (47-I,*-I) - _*

1

^P = I - M ~ ( M M ~ ) - ~ M = ;I ^- [$I + ^{(47 47) =}

^'

^{- 2 1}

^{1 2 1 2}

, 2 2

d

=PVf (z) = -

_47-1

I new

FIGURE

3 Configuration in y s p a c e

zs clzimed.

A summzry of t h e algorithm i s given as follows:

I n i t i a l i z a t i o n step: Choose a f e a s i b l e point z, and find Ii

=

f i : g l ( z )

=

O j . L e t u =1 and go t o (1).

(1) If

Ii =

0 , l e t P = I , form d,

=

^{P V f (2,)} a n d go t o (3). Otherwise, form t h e p r o j e c t i o n m a t r i x in x - s p a c e as follows. Let M

=

Dp ^(2,) b e t h e m a t r i x of g r a d i e n t s of t h e binding c o n s t r a i n t s at

I,.

If P

=

I-

M'

(hWt)-

M =

0 , l e t W

= -(hWf )-'

M Vf (2,).

If W r 0 , z, will b e a Kuhn-Tucker point, otherwise, d e l e t e a row c o r r e s p o n d i n g t o Wi r 0 a n d r e p e a t s t e p (1). This h a s t h e e f f e c t of eliminating binding c o n s t r a i n t s from c o n s i d e r a t i o n which won't g e n e r a t e a n improving f e a s i b l e d i r e c t i o n . L e t I

=

f i :gi ( z )

=

O j a f t e r a n o n z e r o

P

h a s been found.

(2) If X

= n

(gi ( z ) r\ f R n -I hyperplane!) i s a l r e a d y l i n e a r , u s e t h e matrix in

i € I

t h e ' foliowing c a l c u l a t i o n s . Otherwise, find a c o o r d i n a t e c h a n g e s u c h t h a t X becomes

Find Vf ,,,(y ( 2 ) )

!

y (z, ) a n d c o n v e r t t h e problem i n t o y c o o r d i n a t e s . Form t h e pro- jection m a t r i x P

=

I

-

a t ( a a t )-la and go t o (3) a f t e r forming d,

=

P V f ,,JY (z

1) l

Y (z,

(3) L e t h, b e a solution t o

M i ~ i m i z e f (z,

+

hd, ) w h e r e z,

=

z,, if in z c o o r d i n a t e s a n d z,

=

y,, if in y c o o r d i n a t e s

0 4 h S h,,,

w h e r e h,,, i s d e t e r m i n e d s o t h a t t h e problem remains f e a s i b l e .

L e t z,

=

z,

+

hd, , c o n v e r t t o z c o o r d i n a t e s , if n e c e s s a r y , a n d r e t u r n t o (1).

F o r m o r e complex p r o b l e m s involving more t h a n o n e binding c o n s t r a i n t , t h e c o o r - d i n a t e c h a n g e s must b e automated and c h e c k s made on t h e neighborhood of validity of t h e t r a n s f o r m a t i o n s . Application t o o t h e r primal methods c a n a l s o b e made using t h e same t y p e s of a r g u m e n t s .

Sensitivity Analysis and Unfoldings

In S e c t i o n 11, we noted t h a t a universal unfolding of a smooth function f ( z ) r e p r e s e n t s t h e most g e n e r a l t y p e of smooth p e r t u r b a t i o n t o which f c a n b e s u b j e c t e d a n d t h a t t h e number of t e r m s needed t o c h a r a c t e r i z e a l l s u c h p e r t u r b z t i o n s e q u a l s codim f

.

F u r t h e r m o r e , if u l(z ),

. . .

, u, (z ) r e p r e s e n t a b a s i s f o r t h e J a c o b i a n i d e a l mn / VCf ), t h e n t h e lui { a l s o r e p r e s e n t a basis f o r t h e s p a c e of a l l s u c h p e r t u r b a t i o n s . S i n c e p e r t u r b z t i o n s in t h e o b j e c t i v e function a n d / o r c o n s t r a i n t s l i e at t h e h e a r t of sensitivity a n a l y s i s f o r n o n l i n e a r optimizztion, i t seems r e a s o n a b l e t o c o n j e c t u r e t h a t t h e c o n c e p t s of unfolding and t r a n s v e r s a l i t y should b e of u s e in c h a r a c t e r i z i n g v a r i o u s i s s u e s a r i s i n g in t h e s e n s i t i v i t y analysis of nonlinear p r o g r a m s . H e r e we s h a l l i n d i c a t e two d i f f e r e n t d i r e c t i o n s t o b e p u r s u e d : 1) c o n s t r a i n t qualification conditions; 2) o b j e c t i v e function stabilizztion and examination of t h e s t a b i l i t y of t h e dual a l g o r i t h m s discussed a b o v e .

T r a n s v e r s a l i t y and the K u b n - T u c k e r Conditions

A s an indication of how singularity t h e o r y arguments can b e employed t o study constraint perturbations, let us examine t h e classical Kuhn-Tucker conditions using transversality arguments.

The Kuhn-Tucker necessary conditions play a n important r o l e in t h e t h e o r e t i c a l development of mathematical programming. These conditions were derived from a more general set of conditions, called t h e Fritz John conditions by assuming t h a t a con- s t r a i n t qualification is in effect. Both t h e Fritz John and Kuhn-Tucker conditions a r e necessary f o r z * t o b e a n optimal solution of t h e constrained optimization problem.

One of t h e most widely used constraint qualifications i s t h a t t h e gradients of t h e bind- ing constraints a t t h e point z * b e linearly independent.

In singularity theory, t h e concept of a t r a n s v e r s e intersection between two mani- folds is a cornerstone f o r s t r u c t u r a l stability arguments. One definition of a t r a n s v e r s e intersection a t a point is t h a t no v e c t o r is perpendicular t o t h e tangent s p a c e s of both manifolds simultaneously [4]. Since t h e gradient v e c t o r of a manifold at a point w i l l also b e t h e normal v e c t o r to t h e tangent hyperplane at t h z t point, i t follows t h a t t h e gradient v e c t o r s of two intersecting manifolds w i l l both b e collinear if and only if t h e intersection is t r a n s v e r s e . Furthermore, and more importantly, t h e Thom Isotropy Theorem [4] s t a t e s t h a t t r a n s v e r s e crossings are s t r u c t u r a l l y stable. This means t h a t s m a l l perturbations of t h e constraints around a Kuhn-Tucker point won't change t h e geometry of t h e intersection much. In f a c t , t h e original constraint confi- guration can b e r e c o v e r e d by a smooth coordinate change around t h e point of interest.

Let us consider a n example demonstrating t h e s t r u c t u r a l instability of a non- t r a n s v e r s e crossing. In t h e example, t h e following definition of a t r a n s v e r s e crossing will b e used:

DEFINITION 1 . Two manifolds, R and S , embedded in R n i n t e r s e c t t r a n s v e r s a l l y if 1 ) R n S

=

# o r

2) codim (T,R)

+

codim (T,S)

<

^{n and}

codim (T,R)

+

codim (T,S)

=

codim T,R nT,S) w h e r e T, is t h e tangest s p a c e a t z

.

Ezample (from 171, s e e Figure 4 f o r geometry) Minimize: f ( z l , z 2 )

=

^-1

Subject to: z 2

-

^{( 1 2 S 0}

Vf

=

(-1,O) at (1,O): Vf (1,O)

=

(-1,O)

v g l

=

(-X1-f) , I ) V g l (1,O)

=

( 0 , l )

-

binding vg 2

=

(0,-1) V g (1,O)

=

(0,-1)

-

^binding

The gradients of t h e binding constraints are not linerly independent. Checking t h e Kuhn-Tucker conditions:

0 = 1 _. , inconsistency, showing t h a t t h e Kuhn-Tucker conditions don't hold.

Transversalit y

Both g l a n d g will b e embedded in R~ s o

T, g l ( z )

=

f(z1,z2,z3): 2 2

-

^3(1*1)2z1

=

^{2 2}

-

^3(1*1)2 z l j T, g 2 ( z )

=

!(z1,z2,z3): 2 2 = z 2 j

at (1,O):

T, g l ( z )

=

f z l , z z z 3 ) : z 2

= oj,

t h u s

7''

g l ( z )

n

^T,g2(z)

⁼

T, g l ( z ) T, g 2 ( z )

=

I(zl,zZ,z3): 2 2 = O j

codim T, g l ( z )

=

1 codim T, g 2 ( z )

=

1

codim (T, g l ( z )

n

Tx g 2 ( z ) )

=

1 Thus,

codim T, g l ( z )

+

codim T, g 2 ( z ) f codim (7'' g l ( z )

n

T, g 2 ( z

1)

s o t h e i n t e r s e c t i o n i s nontransverse.

If t h e cubic c o n s t r a i n t i s p e r t u r b e d slightly:

8 1 ( ~ 1 , ~ Z )

=

YZ

-

^{(1-111)3 + &,}

t h e n T,gl(z)

n

T,g2(z) at (1.0) will b e t h e empty s e t , s o t h e interesection is. by de- finition, t r a n s v e r s e . A t t h e i r point of i n t e r s e c t i o n , z

=

(l+a,O), s o

T, g l ( z )

=

[(z1,z2,z3): z 2

+

3 a 2 z l

=

3 a 2 j Tz g 2 ( z )

=

f(zl,z2,z3): 2 2

= o j

and T, g l ( z )

n ^T,

g 2 ( z )

=

[(z1,z2,z3): z1

=

l j will b e a line in R ~ . Thus,

codim (T, g ( z ))

=

1 codim (T, g ( z ))

=

1

codim (7'' g l ( z )

n

T, g 2 ( z ) )

=

2

s o codim (T, g l ( z ) )

+

codim (T, g 2 ( z )

=

codim (T, g l ( z ))

n

^{T, g l ( z ))} and t h e inter- s e c t i o n i s t r a n s v e r s e .

FIGURE 4 Example of a nontransverse constraint crossing (from Figure 4.5, pg.136, Bazaraa and Shetty, 1979)

The unfolding concept c a n also b e of use in sensitivity analyses in t h e following manner. As a n u n p a m e t r i z e d function, t h e objective function f' ( z ) may b e unstable with r e g a r d t o s m a l l perturbations (i.e. t h e qualitative c h a r a c t e r of t h e c r i t i c a l points of f' may change as a r e s u l t of small changes in

I).

This is clearly a bad situation as f a r as t h e credibility of t h e r e s u l t s obtained from such a n optimization are concerned.

However. if codirn f' = c , a n unfolding of f' involving at least c p a r a m e t e r s w i l l b e s t a b l e r e l a t i v e t o all s t r u c t u r a l perturbations in t h e sense t h a t if f'(z)+p(z) i s a p e r t u r b a - tion of f' , then t h e behavior of f' _{( z )}

+

p ( z ) n e a r i t s c r i t i c a l points c a n b e c a p t u r e d by varying t h e p a r a m e t e r s in a universal unfolding of

I.

A s already noted, t h e elements ul(z.),

...,

u , ( z ) forming a basis f o r m, /

Vw)

constitute a basis f o r exactly t h e type of perturbations we need t o stabilize f'

.

Unfolding can also b e of use f o r studying t h e stabililty of t h e dual optimization al- gorithms, which r e q u i r e t h e formation of a s u r r o g a t e objective function using a compu- tational parameter. For example, t h e augmented Lagrangian method mentioned above r e q u i r e s t h e use of a p a r a m e t e r p . These parameterized functions c a n b e studied t o l e a r n what types of objective functions and constraints may lead t o s u r r o g a t e objective

functions which are s t r u c t u r a l l y unstable, and which may b e h a v e badly as t h e computa- tional p a r a m e t e r i s v a r i e d .

These i d e a s c a n b e illustrated by considering t h e s t a n d a r d l i n e a r programming problem. In a s e n s e , a l i n e a r objective function i s t h e linearization of a g e n e r a l non- l i n e a r f ( z ) , s i n c e no r e a l world p r o c e s s even g e n e r a t e s a completely l i n e a r potential.

DEFINITION 2 . 1 i s s t r u c t u r a l l y s t a b l e if, for sufficiently small smooth p e r t u r b a - tion functions p. t h e c r i t i c a l points of f and f

+

p remain within t h e same neighborhood and h a v e t h e same type (max, min, saddle, etc.).

Consider t h e l i n e a r program:

maximize: f ( z ) = c t z s u b j e c t to:

Az

S 0

Note t h a t t h e Hessian matrix of f will b e identically z e r o f o r all x , so t h a t a l i n e a r p r o - gram h a s a maximum only by v i r t u e of t h e constraints.

If a small l i n e a r p e r t u r b a t i o n i s added t o t h e objective function:

maximize: f ( z )

=

^{( c f}

+

ct ) z , t i

<<

1 s u b j e c t to:

Az

S 0 z =1,2,

..., n

t h e isoclines of t h e o b j e c t i v e function on t h e x h y p e r p l a n e might s h i f t s o t h a t t h e set of isoclines l e a v e s t h e f e a s i b l e region a t a completely d i f f e r e n t e x t r e m e point of t h e con- vex hull of c o n s t r a i n t s . Thus t h e linear programming problem i s not even s t a b l e with r e s p e c t t o l i n e a r p e r t u r b a t i o n s .

In c o n t r a s t , i t i s known t h a t Morse (i.e. q u a d r a t i c ) e x t r e m a are t h e only s t r u c t u r - ally s t a b l e t y p e s f o r nonparameterized functions, although f o r p a r a m e t e r i z e d functions t h e situation i s d i f f e r e n t . Similarly, since adding a small p e r t u r b a t i o n to a Morse func- tion does not d r a s t i c a l l y change t h e location of t h e unconstrained extremum, t h e loca- tion of t h e c o n s t r a i n e d extremum a l s o shouldn't change too much, s i n c e t h e c o n s t r a i n e d extremum usually o c c u r s where t h e c o n s t r a i n t s a r e t a n g e n t to t h e isoclines of j ( z ) .

As a final note, t h e computational implications of t h e a b o v e discussion are not by any means as d i r e as might seem. While t h e g e n e r a l nonlinear programming problem i s computationally difficult, numerical methods f o r q u a d r a t i c p r o g r a m s , both c o n s t r a i n e d and unconstrained, are w e l l developed. Ir: f a c t , since Morse functions a r e t h e only s t r u c t u r a l l y s t a b l e t y p e s of smooth unparametrized functions, a case could b e made f o r transforming even nonquadratic nonlinear programs into q u a d r a t i c form using t h e dif- feomorphic c o o r d i n a t e c h a n g e s guaranteed by singularity t h e o r y . Thus, a q u a d r a t i c program r e p r e s e n t s , in a c e r t a i n sense, t h e canonical problem f o r mathematical pro- gramming.

Acknuwledgement

Much of t h e work r e p r e s e n t e d h e r e a r o s e during t h e c o u r s e of numerous conver- sations with J. Kempf who, in p a r t i c u l a r , is responsible f o r t h e example of t h e use of t h e g r a d i e n t projection method.

REFERENCES

1 . Arnol'd, V . I . (1981) S i n g u l a r i t y T h e o w , Cambridge University P r e s s , Cambridge.

2. Lu, Y.C. (1976) S i n g u l a r i t y T h e o r y a n d an I n t r o d u c t i o n to C a t a s t r o p h e T h e o r y , S p r i n g e r , N e w York.

3. Gibson, G . (1979) S i n g u l a r P o i n t s o f S m o o t h M a p p i n g s , Pitman, London.

4. Poston, T. and I. Stewart. (1978) C a t a s t r o p h e Theory a n d Its A p p l i c a t i o n s , Pitman, London.

5. Milnor, J . (1973) Morse T h e o r y , Princeton U . P r e s s , Princeton,

6. Golubitsky, M. (1978) An I n t r o d u c t i o n to C a t a s t r o p h e T h e o r y a n d i t s A p p l i c a - t i o m , SIAM Review, 20:352-387.

7. Bazaraa, M.S. and C.M. Shetty. (1979) N o n l i n e a r P r o g r a m m i n g , John Wiley and Sons, N e w York, 560 pp.

8. Pavelle, R., M . Rothstein, and J. Fitch (1980) C o m p u t e r a l g e b r a , Scientific Ameri- can, 245, C, pp. 136-153.

9. Casti J . (1984) S y s t e m S i m i l a r i t i e s a n d N a t u r a l L a w s , WP-84-1, International In- s t i t u t e f o r Applied Systems Analysis, Laxenburg, Austria.