Summer2013 Dr.ThomasM.Surowiec NonlinearOptimizationI

11  Download (0)

Full text


Nonlinear Optimization I

Dr. Thomas M. Surowiec

Humboldt University of Berlin Department of Mathematics

Summer 2013




1 A setX⊂Rnis called convex, if for allx,y∈X and allλ∈(0,1)

λx+ (1−λ)y∈X, i.e. the segment[x,y]lies completely inX.

2 LetX⊂Rnbe convex. A functionf :X →Ris called

1 (strictly) convex (inX), if for allx,y ∈X and for allλ∈(0,1) the following holds true:

f(λx+ (1−λ)y)≤λf(x) + (1−λ)f(y) (f(λx+ (1−λ)y)< λf(x) + (1−λ)f(y))

3 f is uniformly convex, if there existsµ >0with

f(λx+ (1−λ)y) +µλ(1−λ)||x−y||2≤λf(x) + (1−λ)f(y),



A Few Remarks

Geometrically speaking convexity of a function implies that the line segment between function values lies above the function’s graph.

Clearly: Uniform convexity→strict convexity→convexity.




Letf:Rn→Rbe a quadratic function, i.e., f(x) + 1

2xTAx+bTx+c withA∈ Sn,b∈Rn,c∈R. Then

1 f is convex iffAis positive semi-definite.

2 f is strictly convex ifffunif. conv. iffApos. def.

These concepts can be extended in some cases to differentiabe convex functions...


Supporting Hyperplanes/Quadratic Functions


LetX⊂Rnbe open and convex, andf :X →Rcontinuously differentiable.

Then the following assertions hold

1 f is convex (onX) iff for allx,y∈Xit holds that f(x)≥f(y) +∇f(y)T(x−y).

2 f is strictly convex (onX) iff for allx,y∈X, withx6=yit holds that f(x)>f(y) +∇f(y)T(x−y).

3 f is uniformly convex (onX) iff there existsµ >0such that f(x)≥f(y) +∇f(y)T(x−y) +µ||x−y||2 for allx,y∈X.


On the board.


Second-Order Properties


LetX⊂Rnbe open and convex, andf :X →Rtwice continuously differentiable. Then the following assertions hold

1 f is convex (onX) iff∇2f(x)is positive semi-definite for allx∈X.

2 If∇2f(x)is positive definite for allx∈X, thenfis strictly convex (onX).

3 f is uniformly convex (onX) iff∇2f(x)is uniformly positive definite on X, i.e., if there existsµ >0such that

dT2f(x)d≥µ||d||2 forx∈X and for alld ∈Rn.


On the board.


Existence of Solutions


Letf:Rn→Rbe continuously differentiable andx0∈Rnarbitrary.

Furthermore, assume that the level set L(x0) :=n



is convex and thatfis uniformly convex inL(x0). Then the setL(x0)is compact.


On the board.


Existence of Solutions


Letf:Rn→Rbe continuously differentiable andX⊂Rnbe convex.

Consider the optimization problem min

x∈Xf(x), (1)

then the following statements hold

1 Iff is convex onX, then the solution set is convex (possibly empty).

2 Iff is strictly convex onX, then(1)has at most one solution.

3 Iff is uniformly convex (onX) andXis non-empty and closed, then(1) has exactly one solution.


On the board.


A Few Cautionary Remarks

1 f(x) =exp(x)is strictly convex onX =R, yet it has no minimimum!

2 X must be closed for a solution to exist. Any ideas?


Consequences I


Letf:Rn→Rbe continuously differentiable,x0∈R, and let the level set L(x0)be convex andf be uniformly convex onL(x0). In addition, suppose thatx∈Rnis the unique global minimizer off. Then there existsµ >0with



Consequences II


Letf:Rn→Rbe a continuously differentiable and convex function. and let x∈Rnbe a stationary point off. Thenxis a global minimizer off inRn.




Related subjects :