Calculation of Trial Steps

v_j^(k)g_j(x_k)− 1

2σ_kg_j(x_k)²

)

=L(x_k, v_k) + 1

2σ_k∥g(x_k)∥²₂ . (4.4) Here the weighted value of the constraint violation is added to the Lagrangian function (2.20) of the equality constrained optimization problem.

The subsequent section specifies the subproblems that are solved in each iteration in order to determine a trial step (dk, wk), wheredk denotes the step in the primal vari-ables andw_k denotes the step in the dual variables. Moreover, a feasibility restoration phase is introduced that is entered if inconsistency in the subproblems occurs.

4.1.1 Calculation of Trial Steps

In each iteration k a trial step (d_k, w_k), which consists of a step d_k in the primal variables and a step w_k in the dual variables, has to be calculated. The exact deter-mination ofw_k is explained later. Let(x_k, v_k)be the current iterate. The basic idea is to solve the standard SQP subproblem (3.1) extended by the trust region constraint

∥d∥∞≤ ∆_k. The trust region constraint can easily be transformed into simple bound constraints, i.e., ∥d∥∞≤ ∆_k can be replaced by 2n linear inequality constraints, that is

di+ ∆k ≥0, i= 1, . . . , n , (4.5) and

∆_k−d_i ≥0, i= 1, . . . , n . (4.6) The resulting subproblem is a quadratic problem that can be solved by any available solver. No special strategy for handling the trust region constraint has to be added.

Reformulating the trust region constraint leads to the following subproblem formula-tion

minimize

d∈Rⁿ ∇f(x_k)^Td+1

2d^TB_kd

subject to g_j(x_k) +∇g_j(x_k)^Td= 0 , j ∈ E , g_j(x_k) +∇g_j(x_k)^Td≥0, j ∈ I ,

−∆_k≤d_i ≤∆_k , i= 1, . . . , n ,

(4.7)

where Bk ∈ R^n×n is a symmetric matrix that approximates the Hessian of the La-grangian function (2.20) of the underlying problem.

In the algorithm B_k is required to be positive definite. Positive definiteness is suf-ficient for the global convergence theory. Additional requirements are only necessary for the local convergence theory. Then matrix Bk has to be a good approximation of the Hessian of the Lagrangian function in some sense.

If the feasible region of problem (4.7) is not empty, then the solution of (4.7) is de-noted by(d_k, u_k, µ_k), whereu_k ∈R^mis the Lagrangian multiplier vector corresponding to the linear constraints g_j(x_k) +∇g_j(x_k)^Td_k, j = 1, . . . , m. To simplify the notation in the remainder of this work, µk ∈Ris introduced and defined by

µ_k :=

∑n i=1

(

µ^(k)

i +µ^(k)_i ⁾ , (4.8)

whereµ_k:= (µ^(k)₁ , . . . , µ^(k)_n )^T denotes the multipliers corresponding to the lower bounds (4.5) on step d_k, and µ_k := (µ^(k)₁ , . . . , µ^(k)_n )^T denotes the multipliers corresponding to the upper bounds (4.6), respectively. A more detailed derivation is stated in Section 4.2.

As illustrated by an example in Section 3.4, subproblem (4.7) can be infeasible and no solution exists. In order to overcome this situation, afeasibility restoration phase is introduced. Such a feasibility restoration phase is also used by the filter algorithm, see Algorithm 3.6 by Fletcher, Leyffer, and Toint [43]. This approach differs from other methods by the fact that the standard procedure is to solve the undisturbed subprob-lem (4.7). Approaches as the Vardi-like ones, the Celis-Dennis-Tapia ones or the one by Yuan, see earlier comments in Section 3.4, apply relaxation techniques in each iteration to guarantee consistency of the subproblems during the whole optimization process.

Consequently, one has to take care of additional safeguards to achieve convergence.

The aim of the strategy employed by the new algorithm presented here is to avoid the need of an additional penalty parameter in the subproblems. Therefore, subproblem (4.7) is solved whenever possible. Only in case the problem is inconsistent a switch to a different subproblem is performed. This strategy is also employed by an algorithm addressing equality constrained problems proposed by El-Alem [30]. El-Alem’s algo-rithm also tries to solve the equality constrained formulation of subproblem (4.7) first.

If the problem is infeasible, then a relaxed problem is solved to obtain a trial step.

During the restoration phase the trial steps are determined in two steps. First, the minimum constraint violation that can be achieved within the trust region bound is determined. Thefeasibility restoration problemthat is solved in this situation is defined as

minimize d∈Rⁿ, δ∈R

∑

j∈E∪Ak

g_j(x_k)²δ²

subject to g_j(x_k)(1−δ) +∇g_j(x_k)^Td= 0, j ∈ E , g_j(x_k)(1−δ) +∇g_j(x_k)^Td≥0, j ∈ Ak , g_j(x_k) +∇g_j(x_k)^Td≥0, j ∈ Bk ,

−∆_k≤d_i ≤∆_k , i= 1, . . . , n , 0≤δ≤1 ,

(4.9)

where the sets Ak and Bk stand for A(x_k,0) and B(x_k,0) as defined by (2.15) and (2.16), respectively. This problem always has a solution, since (d, δ) = (0,1) is feasi-ble. The problem determines a relaxation parameter δk. Note that inactive inequality constraints, i.e., constraints in set Bk, are not relaxed and the linearized constraints remain satisfied. The solution of problem (4.9) be denoted byd¯_k and δ_k.

After the required relaxation parameterδ_khas been calculated, a second subproblem is set up, where the violated constraints are relaxed. It is stated as

minimize

In subproblem (4.10) the parameterδ_k remains fixed. Subproblem (4.10) is consistent, asd¯_k is a feasible point for (4.10). The solution of (4.10) is also denoted by(d_k, u_k, µ_k), where uk is the multiplier vector with respect to the m linear approximations of the constraints and µ_k is obtained according to (4.8) with the corresponding multipliers µ_k and µ_k. Since problem (4.7) is inconsistent in this case, there exists at least one linearized constraint that had to be relaxed.

The algorithm employs an additional variable z_k∈R^m that measures the violation of the linearized constraints at the solution of (4.7) or (4.10), respectively. In case the standard subproblem (4.7) is feasible, then the m entries ofz_k are set to zero since all linearized constraints are satisfied. If subproblem (4.10) is solved, then the vector z_k is determined according to the corresponding subproblem, i.e., dk is either the minimizer of problem (4.7), if the subproblem is consistent, or the solution to the relaxed problem (4.10). The step w_k in the dual variables is set to

w^(k)_j := wherezk is either zero, in case problem (4.7) is consistent, or defined by (4.11). Thus, the size of the dual step w_k is also controlled in some sense during the feasibility

restoration phase. Definition (4.12) is motivated by the convergence analysis.

The following section introduces the model that is applied to evaluate the quality of the calculated trial steps.