# Summer2013 Dr.ThomasM.Surowiec NonlinearOptimizationI

(1)

## Nonlinear Optimization I

Dr. Thomas M. Surowiec

Humboldt University of Berlin Department of Mathematics

Summer 2013

(2)

## Convexity

Definition

1 A setX⊂Rnis called convex, if for allx,y∈X and allλ∈(0,1)

λx+ (1−λ)y∈X, i.e. the segment[x,y]lies completely inX.

2 LetX⊂Rnbe convex. A functionf :X →Ris called

1 (strictly) convex (inX), if for allx,y ∈X and for allλ∈(0,1) the following holds true:

f(λx+ (1−λ)y)≤λf(x) + (1−λ)f(y) (f(λx+ (1−λ)y)< λf(x) + (1−λ)f(y))

3 f is uniformly convex, if there existsµ >0with

f(λx+ (1−λ)y) +µλ(1−λ)||x−y||2≤λf(x) + (1−λ)f(y),

∀x,y∈X,∀λ∈(0,1).

(3)

## A Few Remarks

Geometrically speaking convexity of a function implies that the line segment between function values lies above the function’s graph.

Clearly: Uniform convexity→strict convexity→convexity.

(4)

## Example

Example

Letf:Rn→Rbe a quadratic function, i.e., f(x) + 1

2xTAx+bTx+c withA∈ Sn,b∈Rn,c∈R. Then

1 f is convex iffAis positive semi-definite.

2 f is strictly convex ifffunif. conv. iffApos. def.

These concepts can be extended in some cases to differentiabe convex functions...

(5)

Lemma

LetX⊂Rnbe open and convex, andf :X →Rcontinuously differentiable.

Then the following assertions hold

1 f is convex (onX) iff for allx,y∈Xit holds that f(x)≥f(y) +∇f(y)T(x−y).

2 f is strictly convex (onX) iff for allx,y∈X, withx6=yit holds that f(x)>f(y) +∇f(y)T(x−y).

3 f is uniformly convex (onX) iff there existsµ >0such that f(x)≥f(y) +∇f(y)T(x−y) +µ||x−y||2 for allx,y∈X.

Proof.

On the board.

(6)

## Second-Order Properties

Theorem

LetX⊂Rnbe open and convex, andf :X →Rtwice continuously differentiable. Then the following assertions hold

1 f is convex (onX) iff∇2f(x)is positive semi-definite for allx∈X.

2 If∇2f(x)is positive definite for allx∈X, thenfis strictly convex (onX).

3 f is uniformly convex (onX) iff∇2f(x)is uniformly positive definite on X, i.e., if there existsµ >0such that

dT2f(x)d≥µ||d||2 forx∈X and for alld ∈Rn.

Proof.

On the board.

(7)

## Existence of Solutions

Lemma

Letf:Rn→Rbe continuously differentiable andx0∈Rnarbitrary.

Furthermore, assume that the level set L(x0) :=n

x∈Rn

f(x)≤f(x0)o

is convex and thatfis uniformly convex inL(x0). Then the setL(x0)is compact.

Proof.

On the board.

(8)

## Existence of Solutions

Theorem

Letf:Rn→Rbe continuously differentiable andX⊂Rnbe convex.

Consider the optimization problem min

x∈Xf(x), (1)

then the following statements hold

1 Iff is convex onX, then the solution set is convex (possibly empty).

2 Iff is strictly convex onX, then(1)has at most one solution.

3 Iff is uniformly convex (onX) andXis non-empty and closed, then(1) has exactly one solution.

Proof.

On the board.

(9)

## A Few Cautionary Remarks

1 f(x) =exp(x)is strictly convex onX =R, yet it has no minimimum!

2 X must be closed for a solution to exist. Any ideas?

(10)

## Consequences I

Lemma

Letf:Rn→Rbe continuously differentiable,x0∈R, and let the level set L(x0)be convex andf be uniformly convex onL(x0). In addition, suppose thatx∈Rnis the unique global minimizer off. Then there existsµ >0with

µ||x−x||2≤f(x)−f(x),∀x∈L(x0).

(11)

## Consequences II

Theorem

Letf:Rn→Rbe a continuously differentiable and convex function. and let x∈Rnbe a stationary point off. Thenxis a global minimizer off inRn.

Updating...

## References

Related subjects :