The 306 academic and real-life test problems used to evaluate the performance on continuous optimization problems are published in Hock and Schittkowski [61], and in Schittkowski [102]. The test examples are provided with optimal objective function values f(x⋆), either known from analytical investigations or from the best numerical data found so far.
Partial derivatives are approximated by forward differences,
∂
∂xi
f(x)≈ 1 θi
(
f(x+θiei)−f(x)) , (6.17) where θi :=θ1/2m max(10−5,|xi|) and ei is the i-th unit vector, i= 1, . . . , n. The toler-ance θm represents the machine accuracy. In a similar way, derivatives of constraints are approximated. Note that each time a gradient is approximated, the corresponding function is evaluated n times.
MISQP and NLPQLP are executed within the same test environment and called with the same parameter values shown in Table 6.2. The obtained results are summarized in Table 6.3, where the columns are defined as
nsucc - number of successful test runs according to above definition (6.7)-(6.9), nacc - number of acceptable solutions according to (6.10)-(6.12),
nerr - number of test runs terminated by an error message, nf unc - mean value of number of function evaluations, ngrad - mean value of number of gradient evaluations,
nallf - mean value of number of all function evaluations (function evaluations for gradient approximation included),
time - time for whole test set (in seconds).
The values for nf unc or ngrad are obtained by counting each evaluation of a whole
code nsucc pf unc pgrad pallf ptime MISQP/lag 280 1.0 1.0 1.0 1.2 MISQP/soc 279 2.0 1.3 1.2 2.1 MISQP/com 283 1.0 1.0 1.0 1.4 NLPQLP 284 1.6 1.1 1.2 1.0
Table 6.4: Priority Values for a Set of 306 Continuous Test Problems
1 2 3 4 5 6 7 8 9 10 20
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
MISQP/lag MISQP/soc MISQP/com NLPQLP
Figure 6.1: Performance Profiles for a Set of 306 Continuous Problems
set of function or gradient values, respectively, for a given iterate xk. The additional function evaluations needed for gradient approximations are not counted for nf unc, however, they are included in nallf.
Table 6.3 shows that all codes can solve almost all problems to at least an acceptable solution according to (6.10)-(6.12). All three versions of MISQP terminate with an error message for one test problem. They are not able to terminate at a feasible iterate. MISQP/soc cannot solve a second problem. Again the code converges toward an infeasible point. The number of local solutions, i.e., nacc, is almost identical for all codes.
NLPQLP is obviously the fastest solver with respect to the time needed for the whole test set. As NLPQLP is the only code using line search instead of trust region tech-niques to stabilize the convergence, this can be expected. Each search step in trust region methods corresponds to the solution of a quadratic subproblem. It may require several runs of the quadratic programming solver to obtain a new acceptable iterate, whereas in line search methods only one quadratic program is solved and the next it-erate lies along the calculated search direction. Applying warm start techniques in the quadratic programming solver can improve the performance of trust region algorithms.
MISQP/lag is the only version of MISQP that makes use of the warm start capability of the applied version of the solver QL. Comparing MISQP/lag and MISQP/soc, the difference in the measured time occurs probably due to the warm starts applied by MISQP/lag, since both codes behave identical in the other performance criteria. How-ever, the gap to the line search code NLPQLP still remains.
Comparing the average number of function evaluations – with or without function evaluations for gradient approximations included – Table 6.3 leads to the conclusion thatMISQP/lag andMISQP/com behave almost identically and outperformMISQP/soc significantly. The mean values shown forMISQP/socare dominated by three problems, where MISQP/soc converges very slowly. Nevertheless, the results indicate that the algorithms, that avoid calculating second order correction steps, converge locally with fast rate. NLPQLP also needs less function and gradient evaluations than MISQP/soc, but a few more thanMISQP/lag and MISQP/com.
For the calculation of priority values only runs that are considered to be successful according to (6.7)-(6.9) are taken into account. The calculated priority values are given in Table 6.4, where the columns denote
nsucc - number of runs that obtain successful solutions, pf unc - relative priority of function evaluations,
pgrad - relative priority of gradient evaluations,
pallf - relative priority of all function calls including function calls used for gradient approximations,
ptime - relative priority of execution time.
The results shown in Table 6.4 attest that NLPQLP is the fastest solver with respect to the time needed for all test problems.MISQP/socneeds more than twice the time of
Option Value Description
ACC 10−6 termination accuracy,
ACCQP 10−10 termination accuracy of mixed-integer QP solver, MAXIT 500 maximum number of iterations,
MAXNDE 5,000 maximum number of branch-and-bound nodes in the subprob-lem solver.
Table 6.5: Parameter settings for 175 Mixed-Integer Test Problems forMISQP
NLPQLP. MISQP/lag and MISQP/comperform identically on the other performance cri-teria. Both codes outperformNLPQLP slightly. Comparing MISQP/lagand MISQP/soc, you see that the value of pgrad for MISQP/soc is 1.3 times higher than the one of MISQP/lag, but the value of pf unc is two times the value of MISQP/lag. This can be explained by the additional second order correction steps calculated by MISQP/soc.
Another indication of the need to avoid the SOC steps.
The performance profiles given in Figure 6.1 are based on the total number of function evaluations obtained by summing up the number of function calls required by the algorithm and the number of function calls needed for approximating gradients, i.e., nallf is considered. Again only instances that are solved successfully according to (6.7)-(6.9) are taken into account. All other runs are assumed to fail. In case a run returns an acceptable solution or terminates with an error, then the number of required function evaluations is set to a predefined large constant when determining (6.13) and (6.14). Take a look at the green curve for MISQP/soc. In 28 percent MISQP/soc is the solver with the lowest number of function evaluations. 75 percent of the problems are solved byMISQP/socwithin at most twice as much function evaluations than the best solver. The profile also shows that there are a few problems where MISQP/soc needs more than 20 times the number of function evaluations than the best solver.NLPQLPis the best solver in almost 50 percent of the problems.MISQP/lag follows closely. Once again MISQP/soc has the worst performance of all solvers.
The tests show that problems can be efficiently solved without the calculation of second order corrections steps. This was the intention of the development of Algo-rithm 4.1. The corresponding implementation, referred to as MISQP/lag, outperforms version MISQP/soc significantly. MISQP/soc is the implementation of Algorithm 3.3 with additional second order correction steps.