Universität Konstanz Sommersemester 2015 Fachbereich Mathematik und Statistik

Universität Konstanz Sommersemester 2015 Fachbereich Mathematik und Statistik

Prof. Dr. Stefan Volkwein Sabrina Rogg

Optimierung

http://www.math.uni-konstanz.de/numerik/personen/rogg/de/teaching/

Sheet 6

Tutorial: 7th July Exercise 17 (Scaled gradient method)

Consider the quadratic function f : R ² → R , f (x) = 1

2 x ^>

100 −1

−1 2

x + 1 1 x + 3.

Implement a modified version of the Gradient Method where the update is x ^k+1 = x ^k + t _k d ^k with d ^k = M ⁻¹ (−∇f(x ^k ))

and with the exact stepsize t ^k (Exercise 5). M is one of the following matrices:

M = Id = 1 0

0 1

, M = ∇ ² f =

100 −1

−1 2

, M =

f _xx 0 0 f _yy

=

100 0 0 2

. As basis you can use the Gradient Method you implemented for the first program sheet.

Determine the number of gradient steps required for finding the minimum of f with the different matrices M and initial value x0 = [1.5;0.6] (use = 10 ⁻⁹ ). How close is the computed point to the exact analytical minimum? Explain your observations.

Exercise 18 (Cauchy-step property)

The Cauchy step is defined as s ^CP _a = −t _a ∇f(x _a ), where t _a is given by (see the lecture notes)

t _a =

( _∆

k∇f(x

)k if ∇f (x _a ) ^> H _a ∇f(x _a ) ≤ 0,

min

∆

k∇f(x

)k , _{∇f (x} ^k∇f(x

^)k

a

)

H

∇f(x

)

if ∇f (x _a ) ^> H _a ∇f(x _a ) > 0.

Once the Cauchy point x ^CP _a = x _a + s ^CP _a is computed, show that there is a sufficient decreasing in the quadratic model, i.e, the Cauchy point satisfies

m _a (x _a ) − m _a (x ^CP _a ) ≥ 1

2 k∇f(x _a )k min

∆ _a , k∇f(x _a )k 1 + kH _a k

.

Exercise 19 (Dogleg strategy)

Let us consider the quadratic model of the function f in x _a m _a (x) = f (x _a ) + ∇f(x _a ) ^> (x − x _a ) + 1

2 (x − x _a ) ^> B _a (x − x _a ),

with B _a positive definite (Hessian matrix in x _a or an approximation of it). In the trust- region subproblem we approximately solve

kx−x min

k<∆

m _a (x). (1)

If the trust region is big enough, i.e as if there is no constraint kx − x a k < ∆ a , the exact (global) solution to (1) is the (quasi-)Newton point

x ^QN _a = x _a − B _a ⁻¹ ∇f(x _a ).

The idea of the dogleg method is as follows: Minimize m _a along a path consisting of two straight lines: one from the current point x _a to the Cauchy point x ^CP _a and the other one from x ^CP _a to the (quasi-)Newton point x ^QN _a . This path is described as

x(τ ) =

( x _a + τ(x ^CP _a − x _a ) τ ∈ [0, 1], x ^CP _a + (τ − 1)(x ^QN _a − x ^CP _a ) τ ∈ (1, 2].

Show that h(τ) := m _a (x(τ )) is a decreasing function of τ .

Hint: consider m

on the two straight lines separately.