6.4 Mixed-Integer Optimization Problems
6.4.1 Results for Relaxed Problem Formulation
As already mentioned before, all test problems contained in the collection of mixed-integer nonlinear problems are relaxable. In the remainder of this chapter the per-formance results obtained on the mixed-integer problems are compared to the results achieved on the relaxed formulation of the mixed-integer problems, i.e., the domain of the integer variables is relaxed toR. Again, the performance of MISQPis compared to the one of NLPQLP. The tests are executed with the same termination tolerances as the mixed-integer codes, see Table 6.5.
The specific parameter values set forNLPQLP are listed in Table 6.1.NLPQLPis tested with two different parameter settings. NLPQLP is executed in standard form by setting MODE=0. Additionally, NLPQLP is called in safeguard mode with MODE=17. The codes are denoted by NLPQLP (mode=0)and NLPQLP (mode=17), respectively. The value 17 forMODEstands for initial and repeated scaling every 17th step, i.e., the BFGS matrix is reset to a multiple of the identity matrix.
For the relaxed problems the partial derivatives with respect to the integer and
bi-code nsucc nacc nerr pallf nallf ptime ∅time
MISQP/lag 150 17 8 1.0 1262 1.0 0.07
MISQP/soc 153 16 6 1.4 2300 2.5 0.55
MISQP/com 156 13 6 1.0 1790 1.2 0.35
NLPQLP (mode=0) 140 15 20 1.2 1296 1.0 0.05 NLPQLP (mode=17) 146 12 17 1.0 1042 2.9 0.14 Table 6.7: Results for a Set of 175 Relaxed Mixed-Integer Test Problems
1 2 3 4 5 6 7 8 9 10 11
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
MISQP/lag MISQP/soc MISQP/com
NLPQLP (mode=0) NLPQLP (mode=17)
Figure 6.4: Performance Profiles for a Set of 175 Relaxed Mixed-Integer Problems
nary variables are approximated externally, as described above, i.e., partial derivatives with respect to relaxed binary variables are computed by forward difference formula (6.17), as for the continuous variables. For relaxed integer variables the two-sided formula (6.18) is applied.
Table 6.7 summarizes the numerical results obtained for the relaxed reformulations of the mixed-integer optimization problems. The column titles are the same as for Table 6.6. Note that the returned objective function value of a run on the relaxed problems has to be lower or equal to the best known objective function value of the corresponding mixed-integer problem, in order to be considered a successful return.
More runs are terminated with an error message than in the mixed-integer case.
This indicates that the problems are hard to solve. The codes solve at most 156 prob-lems successfully, i.e., they terminate at a feasible iterate where the KKT optimality conditions are satisfied subject to a tolerance 10−6, and where the objective function value is less than or equal to the known one for the mixed-integer problem formulation.
In all other situations, the codes either fail to solve the problem, or stop at a local solution which is worse than the known one for the mixed-integer formulation. Thus, it is hard to expect that MISQP will ever be able to get significantly higher scores for nsucc for the more complex mixed-integer version of the test set.
The number of problems successfully solved by NLPQLP is lower than the ones ob-tained by the different versions of MISQP. On this set of test problems,NLPQLP is less reliable and robust than MISQP. It is the other way around on the previously tested continuous set, see Section 6.3. The robustness ofNLPQLPcan be improved by changing optionMODEto 17. The price to pay is an increase of the average time spent in solving a problem.
The average number of function evaluations needed by MISQP/soc is twice the one required by MISQP/lag and both versions of NLPQLP. As the value is also higher for MISQP/com, where the augmented Lagrangian merit function is chosen only locally, this difference might be explained by the use of the L∞-penalty function as merit function and the resulting subproblems. Some problems require significantly more function evaluations in case the L∞-penalty function is employed.
The performance profiles for the relaxed problems are shown in Figure 6.4. According to these profiles, NLPQLP (mode=17) is the most efficient code in more than 60% of the problems. On the other hand, MISQP/com is more robust. Again MISQP/com and MISQP/lagperform better thanMISQP/soc. On the relaxed problems the performance profiles indicate a similar behavior of MISQP/soc and NLPQLP (mode=0).
Finally, the results obtained on the relaxed test set, on the one hand, and the mixed-integer version of the test set, on the other hand, are compared. The data are shown in Table 6.8. The average number of function evaluations required byMISQP/soc on the relaxed set is much higher than the one needed by MISQP/com/minlp and MISQP/soc/minlpfor the mixed-integer problems. Regarding priority value pallf, the value calculated forMISQP/socis almost identical to the ones obtained by the mixed-integer solvers. This result is very interesting. Assume MISQP/soc is employed in a basic branch-and-bound framework. Then the branch-and-bound method would
re-code nsucc nacc nerr pallf nallf ptime ∅time MISQP/lag 150 17 8 1.0 1262 1.0 0.07 MISQP/soc 153 16 6 1.6 2300 3.5 0.55 MISQP/com 156 13 6 1.2 1790 2.3 0.35 MISQP/soc/minlp 146 25 4 1.6 1629 224.5 24.17 MISQP/com/minlp 150 21 4 1.5 1506 228.5 23.74 Table 6.8: Comparison Mixed-Integer Problems vs. Relaxed Problems
1 2 3 4 5 6 7 8 9 10 20 30
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
MISQP/lag MISQP/soc MISQP/com
MISQP/soc/minlp MISQP/com/minlp
Figure 6.5: Performance Profiles – Mixed-Integer Problems vs. Relaxed Problems
quire more function evaluations just to obtain a solution to the root problem of the branch-and-bound tree thanMISQP/com/minlpneeds to solve the mixed-integer prob-lem to optimality.
Expectedly, the mixed-integer codes require much more effort than the continuous counterparts with regard to the average time spent to solve the problems. Solving mixed-integer quadratic problems is more complex as the applied branch-and-bound method in the subproblem solver MIQL requires the solution of numerous continuous quadratic problems.
Figure 6.5 shows the corresponding performance profiles with respect to the total number of function evaluations. In almost 40% of the test problemsMISQP/soc/minlp andMISQP/com/minlpare the best solvers subject to the criterion under consideration.
The corresponding values for the continuous versions of MISQP are lower. Surprisingly, MISQPrequires almost the same effort to solve continuous and mixed-integer problems.
6.5 Summary
The FORTRAN subroutine MISQP is tested on continuous and mixed-integer test prob-lems. The numerical results show the efficiency of the proposed Algorithm 4.1 for continuous problems. The implementation is almost as efficient and reliable as the well-established software NLPQLP by Schittkowski [104]. Moreover, the coded Algo-rithm 4.1 outperforms an implementation of AlgoAlgo-rithm 3.3, which applies second order correction steps as proposed by Yuan [130]. The results also indicate that fast local convergence is obtained in practice for Algorithm 4.1.
The aim of this thesis is to find techniques that lead to a reduction of the average number of function evaluations required by an implementation of Algorithm 5.2. The outcome is Algorithm 5.3 that avoids the calculation of second order correction steps.
The test results show that the total number of function evaluations has been reduced about 7% when comparing the implementation of Algorithm 5.3 to the one of Algo-rithm 5.2. Furthermore, the calculation of second order correction steps can be avoided without getting less reliable. Since the numerical solution of the mixed-integer qua-dratic subproblems is time-consuming, the codes should be applied to problems where evaluating function values takes much more time than the internal calculations of the algorithms.
New algorithms are presented that address nonlinear optimization problems. The con-sidered problems are restricted and contain equality and inequality constraints. More-over, mixed-integer nonlinear optimization problems are considered. The proposed algorithms approximate a solution to the considered problem iteratively by solving a sequence of quadratic subproblems. The idea of the mixed-integer algorithms is to extend the continuous SQP approach to mixed-integer optimization. The straightfor-ward way is the substitution of the continuous quadratic subproblems by mixed-integer quadratic subproblems. Hence, all methods belong to the class of sequential quadratic programming methods. In order to stabilize the methods, the trust region approach is used. The size of the primal step computed as a trial step in each iteration is restricted by an additional constraint.
The main part of this thesis discuss the development of an algorithm that is ap-plicable to continuous nonlinear problems, see Algorithm 4.1. The algorithm employs an augmented Lagrangian function as a merit function. Powell and Yuan [93] also use an augmented Lagrangian but their investigation is restricted to equality constrained problems. Niu and Yuan [76] apply the augmented Lagrangian to problems that also contain inequality constraints. In contrast to their algorithm the new trust region SQP algorithm does not transform the inequality constraints into equality constraints by introducing slack variables.
The occurrence of infeasible subproblems, that result from inconsistency of the lin-earized constraints of the problem or inconsistency induced by adding the trust region constraint, is handled by entering a feasibility restoration phase. Adding a feasibility restoration phase instead of permanently relaxing the quadratic subproblems is also applied by filter SQP methods, see Fletcher and Leyffer [42], and Fletcher, Leyffer, and Toint [43]. The advantage of entering a special restoration phase is that no additional penalty parameter in the subproblems is needed. The drawback of the suggested phase is that more subproblems have to be solved. Future work will be the investigation of a modified restoration phase.
Other trust region methods have to apply safety strategies in order to avoid the so-called Maratos effect – a situation where fast local convergence is prevented. In Yuan [130] a trust region algorithm is discussed that is applicable to equality and in-equality constrained problems. The algorithm employs the L∞-penalty function that can suffer the Maratos effect. Thus, additional second order correction steps are cal-culated as proposed by Fletcher [39] and Yuan [129]. This is a commonly used mod-ification to avoid the Maratos effect. The aim of this work was the construction of a trust region algorithm that avoids these additional safeguards. As the local conver-gence analysis indicates the use of an augmented Lagrangian function as merit function leads to fast local convergence without the additional calculation of SOC steps.
Moreover, the theoretical analysis of the proposed algorithm shows that at least
149
one accumulation point of the generated sequence satisfies the Karush-Kuhn-Tucker optimality conditions. The assumptions made for the proof are used frequently by other authors. The extended MFCQ is applied to ensure the convergence to feasible points.
The new trust region SQP algorithm addressing continuous nonlinear problems, cf.
Algorithm 4.1, is developed to find strategies that can be applied to improve the per-formance of the presented mixed-integer Algorithm 5.2 that is based on an algorithm by Yuan [130]. A modified algorithm is stated that locally employs the augmented Lagrangian merit function instead of theL∞-penalty function. The corresponding Al-gorithm 5.3 does not calculate second order correction steps anymore.
It is well known that mixed-integer problems are extremely difficult to solve. Addi-tionally, in highly complex technical simulation codes especially for engineering appli-cations, it is often not possible to evaluate an objective or constraint function value for fractional values of an integer variable. In case integer variables are not relaxable some of the commonly used methods are not applicable. For example, branch-and-bound methods require a relaxation, see Borchers and Mitchell [11]. Outer approximation methods do not need relaxation but the convergence can be shown for convex prob-lems only, see Fletcher and Leyffer [41]. Both mixed-integer algorithm introduced in this thesis are applicable in this situation.
The discussed algorithms are implemented as a FORTRAN subroutine called MISQP.
Numerical results are presented which show the efficiency of the proposed algorithm for continuous problems. The code is almost as efficient as the well-established code NLPQLP by Schittkowski [104], where a line search strategy is used. It has also been shown that the implementation of the new algorithm outperforms an implementation of an algorithm proposed by Yuan [130], which applies second order correction steps.
The numerical results also indicate that the implementation of the proposed continuous algorithm converges close to a solution with a fast rate what proves practically that the theoretical results hold.
The numerical results presented in Chapter 6 also lead to the conclusion that the goal of reducing the average number of function evaluations required by the new mixed-integer Algorithm 5.3 has been achieved. The total number of function evaluations has been reduced about 7% compared to the implementation of Algorithm 5.2. The tests also show that the numerical solution of the mixed-integer quadratic subproblems is costly. Thus, the approach is recommended in situations where the time spent in evaluating function values dominates the internal calculation time of the algorithm.
Although the implemented mixed-integer algorithms are extremely efficient and stop at a feasible solution after very few iterations, in most cases at the optimal solution, a convergence proof does not exist, even not for convex problems. Future work will be the extension of the mixed-integer algorithms to be able to prove convergence, at least for convex problems. A possible extension is already stated in Exler, Lehmann, and Schittkowski [36], where concepts of outer approximation methods are added to Algorithm 5.2.
MISQPis an implementation of the trust region SQP algorithms outlined before.MISQP is coded in FORTRAN. The arising mixed-integer quadratic programming subproblems are solved by theFORTRANcodeMIQLof Lehmann et al. [65].MIQLapplies a branch-and-cut strategy to solve the mixed-integer quadratic problems. The underlying continuous quadratic programs are solved by QL, see Schittkowski [105], a FORTRANcode tracing back to Powell [89]. QL is an implementation of the primal-dual method of Goldfarb and Idnani [50]. An extended documentation of the one presented here can be found in Exler et al. [35].
MISQP requires user-provided model functions and gradients. The subroutine MISQP is called according to the following rules:
1. Choose starting values for the variables to be optimized, and store them inX, first the continuous, then the binary followed by the integer variables. Initialize ROPT, IOPT, and LOPT.
2. Compute objective and all constraint function values at X and store them in F and G, respectively.
3. Compute gradients of objective function and all constraints, and store them in DF and DG, respectively. The j-th row of DG contains the gradient of the j-th constraint,j = 1,. . . , m. Only partial derivatives subject to the continuous variables and the integer variables specified in IDERIV need to be provided.
4. Set IFAIL=0 and execute MISQP.
5. If MISQP terminates with IFAIL=0, the internal stopping criteria are satisfied.
6. In case of IFAIL>0, an error occurred.
7. If MISQP returns with IFAIL=–1, compute objective function values and constraint values for all variables found in X, store them inF and G, and call MISQP again.
8. If MISQP terminates with IFAIL=–2, compute gradient values subject to variables stored in X, and store them in DF and DG. Only partial derivatives subject to the continuous variables and the integer vari-ables specified inIDERIV need to be provided. Then callMISQP again.
Note that by default derivatives subject to integer variables are approximated by MISQP internally using one-sided or two-sided differences at neighboring grid points.
151
The usage of MISQPand the meaning of the parameters are described below. Default values, as far as applicable, are set in brackets. The option arrays contain parameters passed to the subprogram MIQL. They are also presented and highlighted by MIQL.
Usage
The subroutine MISQP can be called by any FORTRAN program by using the following command.
CALL MISQP( M , ME , MMAX , N , NBIN ,
/ NINT , X , F , G , DF ,
/ DG , XL , XU , ACC , MAXIT ,
/ MAXCUT , MAXNDE , IPRINT , IOUT , IFAIL ,
/ IDERIV , ROPT , IOPT , LOPT , RW ,
/ LRW , IW , LIW , LW , LLW )
Parameter Definition
M Input parameter defining the total number of constraints.
ME Input parameter defining the number of equality constraints.
MMAX Row dimension of array DG containing Jacobian of constraints.MMAX must be at least one and greater or equal to M.
N Input parameter defining the total number of optimization variables, continuous, binary and integer ones.
NBIN Input parameter defining the number of binary optimization vari-ables, must be less than or equal to N.
NINT Input parameter for the number of non-binary integer variables, must be less than or equal to N.
X(N) Input and output vector containing starting point at first call. On return, X is replaced by the current iterate. The first N-NBIN-NINT positions are assigned to continuous variables, the subsequent NBIN coefficients to binary variables, and the remaining NINTpositions to non-boolean integer variables.
F Input parameter containing the objective function value at the cur-rent iterate X.
G(MMAX) Input vector containing the values of the constraints at the cur-rent iterate X, first ME equality constraints, then M-ME inequality constraints.
DF(N) Input vector containing the values of the gradient of the objective function at the current iterate X. It is sufficient to determine the gradients subject to the continuous variables and variables specified in IDERIV. See also description of IDERIV below.
DG(MMAX,N) Input matrix containing the values of the Jacobian of the constraints at the current iterate X, first for theMEequality constraints, then for M-ME inequality constraints. In the driving program, the row dimen-sion of DG must be equal to MMAX. It is sufficient to determine the gradients subject to the continuous variables and variables specified in IDERIV. See also description of IDERIV below.
XL(N), XU(N)
On input, the one-dimensional arrays XL and XU must contain the upper and lower bounds of the variables, first for the continuous, then for the binary and subsequently for the integer variables.
ACC Input parameter defining the tolerance for detecting integer values and for termination. If ACC is less than machine precision, thenACC is internally set to machine precision multiplied by 1,000(1.0E-6).
MAXIT Maximum number of iterations (100).
MAXCUT To enable cut generation of the QP solver, row dimension MMAX is enlarged internally by MAXCUT (500).
MAXNDE Maximum number of branch-and-bound nodes for solving MIQP (10,000).
IPRINT Specification of the desired output level:
0 No output of the program.
1 Only a final convergence analysis is given.
2 One line of intermediate results is printed in each iteration.
3 More detailed information is printed for each iteration.
4 In addition, some messages of the QP solver are displayed.
IOUT Integer indicating the desired output unit number, i.e., all write-statements start with WRITE(IOUT,....
IFAIL The parameter shows the reason for terminating a solution process.
Initially IFAIL must be set to zero. On return IFAIL could contain the following values:
-2 Compute new gradient values for continuous variables and variables specified in IDERIV in DF and DG. See description
-2 Compute new gradient values for continuous variables and variables specified in IDERIV in DF and DG. See description