Convergence Analysis

In this section we prove convergence properties of Algorithm 3.1 for the convex MINLP (3.1) subject to an optimality tolerance ε_OA and a feasibility tolerance ε in a finite number of iterations. An important issue, also concerning the linear outer approximation method described by Algorithm 2.2, is the finite termination of the solution process for NLP(y^k) given by (3.6) or F(y^k) given by (3.7) respectively for some y^k ∈ Y in each iteration of Algorithm 2.2. This topic is usually neglected in existing literature but needs to be addressed, in order to show finite termination of Algorithm 3.1. To establish this task, we introduced aε-stationary-point of NLP(y^k) and F(y^k) for fixed y^k ∈Y in Definition 3.1.

The subsequent lemma shows, that an integer value y^k ∈Y is infeasible in the master problem (3.42) for all values of x∈X, if conditions (3.61) and (3.62) are satisfied for some iterate(x^k, y^k). Note, that this also includes the optimal solution ¯x_y^k of NLP(y^k), i.e., (¯x_yk, y^k) does not satisfy the constraints of the master problem (3.42) if the set T_ε^k contains an iterate (x^k, y^k) satisfying conditions (3.61) and (3.62). Furthermore, (x^k, λ^k_c) is a ε-stationary-point of NLP(y^k) given by (3.6), see Definition 3.1, due to Corollary 3.2.

Lemma 3.1. Let Assumption 3.1 hold and let (x^k, y^k) satisfy conditions (3.61) and (3.62). Furthermore, let (x^k, y^k) be included in the set T_ε^k introduced in (3.43).

Then there exists no x^ ∈ X, such that (^x, y^k) satisfies the constraints of the outer approximation master problem (3.42).

Proof. We will prove by contradiction, that there exists no point (^x, y^k,η), that sat-^ isfies the constraints of the outer approximation master problem (3.42), if the set T_ε^k introduced in (3.43) contains (x^k, y^k) satisfying conditions (3.61) and (3.62).

Note, that for a KKT-point (η^k_c, d^k_c,(λ^k_c, λ^k_ηc)) the KKT-conditions of QP(x^k, y^k) are stated in (3.15). Farkas Lemma, see e.g., Jarre and Stoer [66], states that exactly one of the following two possibilities holds:

1.

∇_xf(x^k, y^k) +B^k_c(d^k_c)_x− [∇_xg(x^k, y^k)]^Tλ^k_c = 0 (3.70) and λ^k_j ≥0, ∀j∈J.

2.

∇_xg_j(x^k, y^k)^Ts ≥ 0, ∀j∈J (3.71) and

(∇_xf(x^k, y^k) +B^k_c(d^k_c)x)^Ts < 0, ∀ s∈R^nc. (3.72) Since(x^k, y^k) is a member ofT_ε^k the subsequent conditions are required for (^x, y^k,η),^ since the corresponding constraints are contained in the outer approximation master problem (3.42)

^

η ≤ f(x^k, y^k) −ε_OA, (3.73)

^

η ≥ f(x^k, y^k) +∇_x,yf(x^k, y^k)^T

^x−x^k 0

, (3.74)

0 ≤ g_j(x^k, y^k) +∇_x,yg_j(x^k, y^k)^T

^x−x^k 0

, ∀ j∈J, (3.75) where (3.73) is an upper bound on variable η introduced in (3.45).

For any constraint of QP(x^k, y^k), that is active at((d^k_c)_x, η^k_c), we get from (3.15) gj(x^k, y^k) = −η^k_c−∇x,ygj(x^k, y^k)^Td^k_c. (3.76) Together with (3.76), (3.75) yields

0 ≤ −η^k_c−∇_x,yg_j(x^k, y^k)^Td^k_c+∇_x,yg_j(x^k, y^k)^T

x^−x^k 0

(3.77)

= −η^k_c+∇_x,yg_j(x^k, y^k)^T

^x−x^k− (d^k_c)_x 0

. (3.78)

As a consequence, we obtain 0

(3.15)

≤ η^k_c ≤ ∇_xg_j(x^k, y^k)^T ^x−x^k− (d^k_c)_x

. (3.79)

Since the KKT conditions of QP(x^k, y^k) are satisfied, (3.70) holds with(λ^k_c)_j ≥0, ∀j∈ J. As (3.71) is also satisfied as shown in (3.79), Farkas lemma (3.72) yields

(∇_xf(x^k, y^k) +B^k_c(d^k_c)_x)^Ts ≥ 0. (3.80)

Let s be given by

s := ^x−x^k− (d^k_c)_x. (3.81) then

(∇_xf(x^k, y^k) +B^k_c(d^k_c)_x)^T ^x−x^k− (d^k_c)_x

≥ 0, (3.82)

holds, which gives

∇_xf(x^k, y^k)^T ^x−x^k

≥ ∇_xf(x^k, y^k)^T(d^k_c)_x− (d^k_c)^T_xB^k_cs. (3.83) Exploiting conditions (3.61) and (3.62), which hold by assumption together with (3.83), condition (3.74) yields

^

η ≥ f(x^k, y^k) +∇_x,yf(x^k, y^k)^T

^x−x^k 0

(3.83)

≥ f(x^k, y^k) +∇_xf(x^k, y^k)^T(d^k_c)_x− (d^k_c)^T_xB^k_c x^−x^k− (d^k_c)_x

(3.84)

≥ f(x^k, y^k) +∇_xf(x^k, y^k)^T(d^k_c)_x+ (d^k_c)^T_xB^k_c(d^k_c)_x− (d^k_c)^T_xB^k_c x^−x^k

≥ f(x^k, y^k) −|∇_xf(x^k, y^k)^T(d^k_c)_x|−|(d^k_c)^T_xB^k_c(d^k_c)_x|−|(d^k_c)^T_xB^k_c x^−x^k

|

≥ f(x^k, y^k) −k∇_xf(x^k, y^k)k₂k(d^k_c)_xk₂−k(d^k_c)^T_xB^k_ck₂k(d^k_c)_xk₂

−k(d^k_c)^T_xB^k_ck₂k ^x−x^k k₂

(3.5)

≥ f(x^k, y^k) −k∇_xf(x^k, y^k)k₂k(d^k_c)_xk₂−k(d^k_c)^T_xB^k_ck₂k(d^k_c)_xk₂−k(d^k_c)^T_xB^k_ck₂M_x

(3.62)

> f(x^k, y^k) −ε_OA.

This contradicts condition (3.73) and shows, that no (^x, y^k,^η) exists, that satisfies the constraints of the outer approximation master problem (3.42), if a ε-stationary point of NLP(y^k) is included in T^k.

Note, that Lemma 3.1 applies to Algorithm 3.1 in Step 4, if conditions (3.61) and (3.62) are satisfied for an iterate (x^k, y^k).

To be able to show finite termination of Algorithm 3.1, the same has to be true, if conditions (3.63) and (3.64) are satisfied for the current iterate (x^k, y^k). This is ensured by the subsequent lemma. Note, that, (x^k, η^k,(λ^k_F, λ^k_η))is aε-stationary point of F(y^k), due to Corollary 3.3.

Lemma 3.2. Let Assumption 3.1 hold and let (x^k, y^k) satisfy conditions (3.63) and (3.64). Furthermore, let (x^k, y^k) be included in the set S^k_ε introduced in (3.44).

Then there exists no x^ ∈ X, such that (^x, y^k) satisfies the constraints of the outer approximation master problem (3.42).

Proof. We will prove by contradiction, that there exists no point (^x, y^k,η), that sat-^ isfies the constraints of the outer approximation master problem (3.42), if the set S^k_ε introduced in (3.44) contains(x^k, y^k) satisfying conditions (3.63) and (3.64).

Since (x^k, y^k) is included in the set S^k_ε, the subsequent conditions are required for (^x, y^k,^η) ∈ X× Y × R that is feasible for the outer approximation master prob-lem (3.42), since it contains the corresponding constraints:

g_j(x^k, y^k) +∇_x,yg_j(x^k, y^k)^T

^x−x^k 0

≥ 0, ∀j∈J. (3.85)

The constraints g_j, j ∈ J can be divided into two sets according to the value of the constraint violation of LPF(x^k, y^k) at the solution ((d^k_F)_x, η^k). We define

W :=

j∈J:g_j(x^k, y^k) +∇_xg_j(x^k, y^k)^T(d^k_F)_x = −η^k (3.86) and

W^⊥ := J\W. (3.87)

For eachj∈W^⊥ the corresponding Lagrangian multiplier (λ^k_F)_j∈R+ associated with the solution((d^k_F)_x, η^k) is zero, i.e.,

(λ^k_F)_j = 0, ∀j∈W^⊥ (3.88)

holds. Furthermore λ^k_η =0 holds due to (3.64). The KKT-conditions of LPF(x^k, y^k) stated in (2.98) yield

1 = X

j∈W

(λ^k_F)_j, (3.89)

0 = X

j∈W

(λ^k_F)j∇_xgj(x^k, y^k). (3.90)

After multiplying all inequalities (3.85) with(λ^k_F)_j, j∈J and summing up, we get X

j∈W

(λ^k_F)jgj(x^k, y^k) + X

j∈W

(λ^k_F)j∇x,ygj(x^k, y^k)

!T

^x−x^k 0

≥ 0, (3.91)

since (λ^k_F)_j=0, ∀j /∈W. Exploiting equation (3.90) yields X

j∈W

(λ^k_F)_jg_j(x^k, y^k) ≥ 0. (3.92)

Since j∈W holds, we get a contradiction due to

0 ≤ P

j∈W

(λ^k_F)jgj(x^k, y^k)

(2.101)

= P

j∈W

(λ^k_F)_j(−∇_xg_j(x^k, y^k)^T(d^k_F)_x−η^k)

≤ P

j∈W

(λ^k_F)_j(|∇_xg_j(x^k, y^k)^T(d^k_F)_x|−η^k)

≤ P

j∈W

(λ^k_F)_j(k∇_xg_j(x^k, y^k)k₂k(d^k_F)_xk₂−η^k)

(3.63),(3.64)

< P

j∈W

(λ^k_F)j(ε−ε)

= 0.

(3.93)

As a consequence, there exists no (^x, y^k,^η) ∈ X×Y ×R, that satisfies the condi-tions (3.85) posed by the outer approximation master problem (3.42), if S^k_ε contains an iterate (x^k, y^k) satisfying (3.63) and (3.64).

Note, that Lemma 3.2 applies to Algorithm 3.1 in Step 4, if the current iterate(x^k, y^k) satisfies (3.63) and (3.64).

Lemmata 3.1 and 3.2 ensure, that the condition requiring either d^k_c =0 and η^k_c = 0 or d^k_F =0 and η^k> 0 whenever conditions (3.61) and (3.62) or conditions (3.63) and (3.64) are satisfied in Step 4, is not needed anymore in Corollary 3.6. Furthermore, we can establish below, that Algorithm 3.1 terminates after a finite number of iterations.

Iff^∗ <∞holds at termination, then the algorithm found aε-solution (x^∗, y^∗)defined as follows.

Definition 3.3. A point (x^∗, y^∗)∈X×Y is a ε-solution of the convex MINLP (3.1), if conditions

kg(x^∗, y^∗)⁻k_∞ ≤ ε (3.94)

and

f(x^∗, y^∗)≤min

y∈V^ max

^

x {f(^x,y) : (^^ x,^λ_c) isε-stationary-point of NLP(^y)}+ε_OA (3.95) are satisfied, whereε_OAis the chosen optimality tolerance andεis the chosen feasibility tolerance and ^λc is the Lagrangian multiplier obtained by solving QP(^x,^y).

Note, that the maximum in (3.95) can be removed, if optimal solutions instead of ε-stationary-points of NLP(y^k), or F(y^k) respectively, are obtained. Then theε-solution defined above corresponds to the global solution of a convex MINLP (3.1) subject to the optimality tolerance ε_OA.

Note, that there exist infinitely many ε-solutions of the convex MINLP (3.1) for any values ofε_OA andε, as long asV 6=∅. There reason is, that there exist infinitely many ε-stationary points for any NLP(y^k) with y^k ∈V for any value of ε > 0.

To carry out the convergence analysis, we have to introduce an additional assump-tion. It is necessary to exclude infeasible stationary points, that possess a constraint violation smaller than the feasibility toleranceε. By choosing the feasibility tolerance sufficiently small, this additional assumption is no severe limitation.

Assumption 3.2. Without loss of generality, we assume, that the solution x¯_y^k of F(y^k) possesses a maximal constraint violation larger than the feasibility tolerance ε for each y^k∈Y\V, i.e.,

kg(¯x_y^k, y^k)⁻k_∞ > ε (3.96) holds for each y^k ∈Y\V.

As a consequence, Corollary 3.6 can be replaced by the subsequent corollary.

Corollary 3.7. Let on^k_MIQP = 0 hold ∀k. Furthermore, let Assumption 3.1 and As-sumption 3.2 hold. Then Algorithm 3.1 terminates after a finite number of iterations either at aε-solution of the convex MINLP (3.1)according to Definition 3.3 or detects, that no feasible point exists, i.e., V =∅, where V is introduced by (3.8).

Proof. Consider some fixed integer value y^k¯. Lety^¯k be inV, then (^x, y^¯k) ∈X×V is infeasible for all values of^x∈Xin the master problem (3.42), if the set T_ε^k contains a ε-stationary point of NLP(y^¯k), see Lemma 3.1. Alternatively lety^k¯ be inY\Vinstead, then(^x, y^¯k)∈X×Y\Vis infeasible for all values ofx^∈Xin the master problem (3.42), if the setS^k_ε contains a ε-stationary point of F(y^¯k), see Lemma 3.2

As a consequence y^k¯ ∈ Y can not be considered twice, if a ε-stationary point of NLP(y^¯k) is contained in the set T_ε^k or respectively a ε-stationary point of F(y^¯k) is contained in the set S^k_ε. In combination with the finiteness of Y this ensures, that Algorithm 3.1 terminates after a finite number of iterations, if aε-stationary point of either NLP(y^¯k) or F(y^¯k) is obtained after a finite number of iterations for ally^¯k∈Y.

Therefore, we prove, that the execution of the Steps 2, 4, 5 and 7 of Algorithm 3.1 yields a ε-stationary point of either NLP(y^¯k) or F(y^¯k) after a finite number of itera-tions starting at an arbitrary iterate(x^k¯, y^k¯).

As long as neither aε-stationary point of NLP(y^¯k) nor F(y^k¯) is obtained,y^k=y^¯kholds for the iteration sequence(x^k, y^k)withk≥k. Due to Corollary 2.5, and Corollary 3.4¯ the iteration sequence (x^k, y^k) with k ≥ ¯k obtained by executing Steps 2, 4, 5 and 7 of Algorithm 3.1 converges towards a stationary point ¯x_y^¯k of NLP(y^¯k), if y^¯k ∈ V or alternatively F(y^¯k), if y^¯k ∈ Y\V. Since the KKT-conditions of either NLP(y^¯k) or F(y^k¯) are satisfied for the accumulation point(¯x_yk^¯, y^¯k), the conditions (3.17) or (3.18) defining a ε-stationary point of NLP(y^¯k) or alternatively F(y^¯k) are satisfied after a finite number of iterations for an arbitrary small fixed value of ε > 0. Therefore, we proved the finite termination of Algorithm 3.1.

Now, we prove by contradiction, that no feasible point exists, i.e., @ (^x,y)^ ∈ X×Y with g_j(^x,y)^ ≥0, ∀j∈J, if Algorithm 3.1 terminates withf^∗ =∞.

Assume, that ∃ (^x,y)^ ∈X×Y with g_j(^x,y)^ ≥0, ∀j∈J. Since g_j(x, y)is concave for all j ∈ J, (^x,y)^ cannot be cut off in the master problem (3.42) by any linearization of some gj(x, y), j∈J. Sincef^∗ =∞holds, no linearization of f(x, y)is contained in the master problem. As a consequence, (^x,^y) is still feasible for the master problem, which contradicts to the termination of Algorithm 3.1.

Moreover, we prove by contradiction, that (x^∗, y^∗) is a ε-solution according to Defi-nition 3.3, if Algorithm 3.1 returns with f^∗ < ∞. Note first, that (x^∗, y^∗) is feasible subject to the feasibility toleranceε, due to the update rule of the best known solution in Algorithm 3.1.

Assume, that there exists a y^ ∈ V with f^+ε_OA ≤ f^∗ at termination, where f^is the maximal function value of all ε-stationary-points of NLP(^y), i.e.,

f^ := max

^

x {f(^x,y) : (^^ x,^λ_c) is ε-stationary point of NLP(^y)},

where ^λ_c is the Lagrangian multiplier obtained by solving QP(^x,y). Since^ y^ ∈ V, there exists a feasible ε-stationary point denoted by ^x_F of NLP(^y), i.e.,

^

For (^x_F,y)^ the subsequent conditions are derived from the linearizations of the con-straints gj(x, y), j∈J, which are contained in the master problem (3.42):

g_j(xⁱ, yⁱ) +∇_x,yg_j(xⁱ, yⁱ)^T

Furthermore, for(^x_F,y)^ also the subsequent conditions are derived from the lineariza-tions of the objective function f(x, y), j ∈ J, which are contained in the master problem (3.42):

by exploiting the convexity of the objective function and f(^x_F,y)^ < f^∗ −ε_OA. As a consequence (^x_F,y)^ is feasible in the master problem MILP(T_ε^k∗, S^k_ε^∗, f^∗, ε_OA), which contradicts to the termination of the algorithm and proves the corollary.

Up to now, we proved, that Algorithm 3.1 terminates after a finite number of iter-ations at a ε-solution according to Definition 3.3 or it detects that no feasible point exists, if no mixed-integer search steps determined by the solution of mixed-integer problem (3.48) are performed, i.e., on^k_MIQP = 0, ∀k. To show, that the convergence properties of Algorithm 3.1 are maintained, if mixed-integer search steps are carried out, we consider two cases separately.

First we assume, that the execution of the Steps 2, 3, 4, 5 and 7 of Algorithm 3.1 always determines an iterate(x^¯k, y^¯k)after a finite number of iterations, which satisfies either conditions (3.61) and (3.62) or conditions (3.63) and (3.64) in Step 4.

Lemma 3.3. Let Assumption 3.1 and Assumption 3.2 hold. Furthermore, let the execution of the Steps 2, 3, 4, 5 and 7 of Algorithm 3.1 always determine an iterate (x^¯k, y^¯k) after a finite number of iterations, that satisfies either conditions (3.61) and (3.62) or conditions (3.63) and (3.64) in Step 4 of Algorithm 3.1 for an arbitrary y^¯k ∈Y.

Then Algorithm 3.1 terminates after a finite number of iterations at a ε-solution according to Definition 3.3 or it detects that no feasible point exists.

Proof. It is sufficient to prove the finite termination of Algorithm 3.1. The proofs given in Corollary 3.7, which ensure that aε-solution is obtained, iff^∗ <∞holds and that no feasible point exists, iff^∗ =∞ holds at termination, remain valid.

By assumption, the execution of the Steps 2, 3, 4, 5 and 7 of Algorithm 3.1 always determine an iterate (x^¯k, y^¯k) after a finite number of iterations, that satisfies condi-tions (3.61) and (3.62) or condicondi-tions (3.63) and (3.64) for somey^¯k ∈Y. Since the set Y is finite, an infinite number of iterations must be caused by infinite repetition of Step 6 of Algorithm 3.1. We show by contradiction, that this is not possible:

Assume, that Step 6 of Algorithm 3.1 is repeated an infinite number of times. Due to the finiteness of the setY, there exists a subsetY^⊂Yof integer values, that are feasible for the master problem at each execution of Step 6. Now we run Algorithm 3.1 until the solution of the master problem (3.42) possesses the integer values of some y^ ∈Y^ for the second time. Then the integer values are fixed to y^ by setting on^k_MIQP =0 in Step 6 of Algorithm 3.1. In this case an iterate(x^¯k,^y)satisfying conditions (3.61) and (3.62) or conditions (3.63) and (3.64) is determined by executing Steps 2, 3, 4, 5 and 7 by assumption, see also Corollary 3.4, Lemma 2.2 and Theorem 2.3. As a consequence of Lemma 3.1 or Lemma 3.2, (x,y)^ is then infeasible in the master problem for all x∈X whenever Step 6 is executed. This contradicts the assumption, thaty^ is in the setY^ and proves the lemma.

We still have to prove that the execution of the Steps 2, 3, 4, 5 and 7 of Algorithm 3.1 converges towards a stationary point according to Definition 2.12 or an infeasible stationary point according to Definition 2.10 of problem NLP(y^¯k) given by (3.6) for some y^¯k ∈ Y. This is established by the following lemma. The convergence ensures that either conditions (3.61) and (3.62) or conditions (3.63) and (3.64) in Step 4 of Algorithm 3.1 are satisfied after a finite number of iterations, see Corollary 3.7.

Lemma 3.4. If Assumption 3.1 and Assumption 3.2 hold, then the successive execu-tion of the Steps 2, 3, 4, 5 and 7 of Algorithm 3.1 yields an iterate (x^¯k, y^¯k) satisfying conditions (3.61) and (3.62) or conditions (3.63) and (3.64) after a finite number of iterations for some integer value y^¯k ∈Y. I.e., Algorithm 3.1 reaches Step 6.

Proof. First we prove convergence, for some special cases, where the number of itera-tions, in which an improving mixed-integer search direction according to Definition 3.2 exists, is finite. The open case, where the number of iterations, in which an improv-ing mixed-integer search direction exists, is infinite, is proved by adaptimprov-ing the proof of Yuan [112] of Theorem 2.3 based on an appropriate redefinition of the iteration sequence of Algorithm 3.1.

Note, that the successive execution of the Steps 2, 3, 4, 5 and 7 yields the same iteration sequence{(x^k, y^¯k)}as the trust region Algorithm 2.1 of Yuan applied to solve NLP(y^k¯) given by (3.6), if no mixed-integer steps are performed, see Corollary 3.4 andy^k=y^¯k holds for all k > ¯k until the conditions (3.61) and (3.62) or conditions (3.63) and (3.64) are satisfied. This is the case, if no improving mixed-integer search direction according to Definition 3.2 exists or if on^k_MIQP =0 holds in iteration k.

The successive execution of the Steps 2, 3, 4, 5 and 7 can either be associated with a bounded or an unbounded sequence of penalty parameters {σ^k}. If the sequence of penalty parameters

σ^k is unbounded, then there exists an iteration k, where^ the value of the penalty parameter σ^{^k} is larger than the threshold ¯σ, specified in Algorithm 3.1. As a consequence, we skip the solution of MIQP(x^k, y^k) given by (3.50) for all k ≥ ^k until either the conditions (3.61) and (3.62) or conditions (3.63) and (3.64) hold. In this case no improving mixed-integer search directions according to Definition 3.2 can be obtained. Convergence is then proved by Corollary 3.4, since the sequence of penalty parameter is monotone increasing until Step 6 is reached.

It remains to prove convergence, if the sequence of penalty parameters{σ^k}is bounded by ¯σ. Without loss of generality, we consider iteration ¯kwith σ^¯k=σ, i.e., the penalty¯ parameter remains constant for all successive iterations until either conditions (3.61) and (3.62) or conditions (3.63) and (3.64) are satisfied in Step 4.

In principle, the successive execution of the Steps 2, 3, 4, 5 and 7 can either con-tain a finite or an infinite number of iterations, where an improving mixed-integer search direction according to Definition 3.2 exists. First we assume, that the number of iterations, in which an improving mixed-integer search direction according to Defi-nition 3.2 exists, is finite. Then we know that the iteration sequence{(x^k, y^k)}, k ≥˜k converges towards a stationary point of NLP(y^˜k) specified in Definition 2.12 due to

Theorem 2.3, see also Corollary 3.4, where ˜k≥k¯ denotes the last iteration, in which an improving mixed-integer search direction according to Definition 3.2 exists.

Now we assume, that there are infinitely many iterations, in which an improving mixed-integer search direction according to Definition 3.2 exists. Since the set Y is finite, there exists a subset Y^ ⊂ Y of integer values, such that every ^y ∈ Y^ occurs infinitely many times in the iteration sequence {(x^k, y^k)} determined by the Steps 2, 3, 4, 5 and 7 of Algorithm 3.1.

We consider an arbitrary ^y∈ Y. We will prove by contradiction in the same way as^ Yuan [112] or Jarre and Stoer [66], that the iteration sequence {(x^k, y^k)} determined by the Steps 2, 3, 4, 5 and 7 of Algorithm 3.1 converges towards a stationary point ¯x_y^ of NLP(^y), see Definition 2.12. Therefore we assume, that such stationary point ¯x_y^ is not an accumulation point of the iteration sequence{(x^k, y^k)}.

Furthermore, we assume without loss of generality, that an improving mixed-integer search directiond^¯k_i according to Definition 3.2 exists in iteration ¯k withy^k¯ = ^y. Since

^

y ∈ Y^ holds, we know, that the integer value y, reoccurs in some iteration denoted^ byk, i.e.,^ ^y= y^k¯ =y^{^k}. As a consequence, we can define the continuous search step d˜^k, which subsumes all search steps inbetween two successive iterations with integer values y, e.g., ˜^ d^¯k given by

x^k+d^k_c, if no improving mixed-integer search direction exists in iterationk andr^k_c > 0holds,

^

x^k, if no improving mixed-integer search direction exists in iterationk andr^k_c ≤0holds,

^

x^k+d˜^k, if an improving mixed-integer search direction exists in iterationk.

Since the sequence of penalty parameters is bounded, we know that

Φ^^k_c(0) − ^Φ^k_c(d^k_c) > δ^kσ¯min{∆^k_c,kg(^x^k,^y)⁻k_∞} (3.102) n-matrixB^k, which is symmetric and positive definite. Otherwise the penalty parameter σ^k does not remain constant due to the penalty parameter update.

For fixed y^ ∈Y let Ωbe defined as the closure of the set of feasible iterates given by Ω := {(˜x,y)^ ∈{(^x^k,y)^ |k >¯k}:kg(˜x,y)^ ⁻k_∞ =0}. (3.104) For (˜x,y)^ ∈Ω we define

Φ(d¯ _x) :=∇_xf(˜x,y)^ ^Td_x+ 1

2M_Bkd_xk²₂+σk(g(˜¯ x,y) + [∇^ _xg(˜x,y)]^ ^Td_x)⁻k_∞, (3.105) for d_x ∈R^nc with d:=

d_x 0

, d∈Rⁿ. M_B is a positive constant satisfying

kB^k_ck₂ ≤ M_B, ∀k, (3.106)

which exists due to Assumption 3.1.

Since we assume, that no stationary point of NLP(^y) is contained in the set Ω, there exists a constant ¯ρ > 0, such that

kdmin_xk_∞≤1(Φ(d¯ _x) −Φ(0)) = −¯¯ ρ. (3.107) As Ω is a compact set, ¯ρ is independent of (˜x,^y)∈Ω. For (^x^k,y)^ we define

Ψ^k(d_x) :=∇_xf(^x^k,y)^ ^Td_x+1

2M_Bkd_xk²₂+σk(g(^¯ x^k,y) + [∇^ _xg(^x^k,y)]^ ^Td_x)⁻k_∞. (3.108) Since Φ^^k_c(d_x)≤Ψ^k(d_x) holds, we obtain

min

kdxk_∞≤∆^k_c( ^Φ^k_c(d_x) − ^Φ^k_c(0)) ≤ min

kdxk_∞≤∆^k_c(Ψ^k(d_x) −Ψ^k(0))

≤ min

kdxk_∞≤1(Ψ^k(d_x) −Ψ^k(0))·min{∆^k_c, 1} (3.109) by exploitingΦ^^k_c(0) =Ψ^k(0)and the convexity of Ψ(λdx) with respect toλ, see Jarre and Stoer [66] for further details.

Due to the compactness ofΩ and the continuity of ∇x,yf(x, y) and ∇x,yg(x, y), there exists a ˜ρ > 0 and a (˜x,^y)∈Ω, such that for all(^x^k,y)^ with dist((^x^k,y), Ω)^ ≤ρ,˜

|Ψ^k(d_x) −Φ(d¯ _x)| ≤ ρ¯

2, ∀d_x ∈R^nc, with kd_xk_∞≤1, (3.110) with

dist(x, Z) := min

z∈Z{kx−zk₂} (3.111)

holds.

If dist((^x^k,y), Ω)^ ≤ρ˜ holds, (3.110) yields min

kdxk_∞≤∆^k_c( ^Φ^k_c(dx) − ^Φ^k_c(0)) ≤ min

kdxk_∞≤1(Ψ^k(dx) −Ψ^k(0))·min{∆^k_c, 1}

≤ min

kd_xk_∞≤1(Φ(d¯ _x) −Φ(0) +¯ ^¯ρ₂)·min{∆^k_c, 1}

≤ −^¯ρ₂ ·min{∆^k_c, 1}.

(3.112)

Due to the boundedness of{∆^k_c}, we obtain min

kd_xk_∞≤∆^k_c( ^Φ^k_c(d_x) − ^Φ^k_c(0)) ≤ −¯δ∆^k_c, (3.113) for some small ¯δ > 0.

For dist((^x^k,y), Ω)^ >ρ, the definition of˜ Ωensures, thatkg(^x^k,y)^ ⁻k_∞≥ρ^holds for some ^ρ > 0. Furthermore, we obtain from (3.102)

Φ^^k_c(d_x) − ^Φ^k_c(0) ≤ −δ¯σmin{∆^k_c,kg(^x^k,^y)⁻k_∞}

≤ −˜δ∆^k_c,

(3.114) with ˜δ > 0.

As a consequence of (3.113) and (3.114)

Φ^^k_c(0) − ^Φ^k_c(d_x) ≥ ^δ∆^k_c, (3.115) holds for all kfor some ^δ > 0.

Let the set of iterations with(^x^k+1,y) = (^^ x^k,y) +^ d˜^k be denoted by K1. Furthermore denote the set of iterations with (^x^k+1,y) = (^^ x^k,y) +^ d^k_c with r^k_c ≥ 0.1 in (3.53) by K₂. The set K:=K₁∪K₂ is considered to be the set of successful iterations yielding a reduction withr^k_c ≥0.1in the merit function. Due to the boundedness ofPσ¯introduced in Definition 2.9, we obtain

by exploiting (3.115), Definition 3.2 and

P_σ¯(^x^k+1,y)^ ≤ P_¯σ((^x^k,y) +^ d^k_i), ∀k∈K₁,

where d^k_i is the improving mixed-integer search direction specified in Definition 3.2.

Therefore P all k sufficiently large, since the ∆^k_c is not decreased for r^k_c ≥ 0.1. This contradicts

P∞ k=1

∆^k_c ≤ ∞. As a consequence, the iteration sequence (^x^k,y)^ is not bounded away from a stationary point of NLP(^y) specified in Definition 2.12, if y^ ∈ Y^ holds and therefore, a ε-stationary point according to Definition 3.1 of NLP(^y) given by (3.6) is obtained after a finite number of iterations for y^ ∈ Y^ for an arbitrary small ε > 0.

This implies, that conditions (3.61) and (3.62) are satisfied after a finite number of iterations.

Subsuming all previous results, yields the final convergence theorem.

Theorem 3.1. If Assumption 3.1 and Assumption 3.2 hold, then Algorithm 3.1 ter-minates after a finite number of iterations at anε-solution of the convex MINLP (3.1) according to Definition 3.3 or it detects that no feasible point exists.

Proof. Together with Corollary 3.7, Lemma 3.3 and Lemma 3.4 prove Theorem 3.1.

3.3 Aspects of Implementation and Future

Im Dokument On Efficient Solution Methods for Mixed-Integer Nonlinear and Mixed-Integer Quadratic Optimization Problems (Seite 100-112)