11.3 Path-based Model
11.4.1 Solving Characteristics of FLOW Configurations
In this section, we will compare the main solving characteristics of the different FLOW configu-rations. By solving characteristics we mean how far the different configurations get when trying to solve VNMP instances, for instance, do they find optimal solutions or do they run out of time before they are able to solve the LP relaxation in the root node. We also consider the standard characteristics like the required run-time or the gap between the lower bound and the best found integer feasible solution after all of the available run-time has been used. We will now discuss the different characteristics that we are going to report on in more detail.
When trying to solve a VNMP instance with one of the configurations of FLOW, we can either succeed (i.e., find an optimal solution) or we can fail to varying degrees. The most serious failure is exceeding the memory limit. Due to technical limitations, we do not get any information about the solving process in this case, even though useful results might have been created before the memory limit was hit. We will denote this type of failure by Mem. The next type of failure is failing to solve the LP relaxation of the root node in the Branch-and-Bound tree, which means we do not get any lower bounds. We will label this type of failure LP. If we succeed in solving the root node and start branching, we can fail to find an integer feasible solution to the VNMP instance that we are trying to solve (denoted by NS). Depending on the configuration of FLOW, it might be possible to buy additional resources. Even if we found an integer feasible solution, it might not be valid for the original VNMP instance because additional resources need to be bought. This type of failure we will denote with AR. If none of these failure reasons apply, then we succeeded in finding an integer feasible solution to the VNMP instance that we are trying to solve. However, we may fail to prove the optimality of this solution (if it is the optimal solution).
We will call this condition Feas. If none of the previous failures apply, then we have found an optimal solution to the VNMP instance we are trying to solve, denoted by Opt. Since FLOW-S does not have an objective to optimize, it counts as Opt if a feasible solution has been found.
In addition to the state that the solving process ended in, we will also report on other character-istics, the first of which is the achieved gap between the final lower bound (lb) on the solution cost and cost of the best found integer feasible solution (ub). The gap (in %) is calculated as 100ub−lbub . A gap of 1% means that the best found solution can be improved by at most 1%. If
we do not have access to lower or upper bounds (i.e., we failed before Feas), we assume a gap of 100%. We also report on the achieved objective values (Obj). However, for the objective we cannot define a useful default value if we fail before Feas, so the reported objective values are only based on instances where at least a feasible solution not requiring additional resources was found. The required run-time t in seconds is not as problematic, if we fail due to Mem we as-sume a run-time of 10000 seconds. For gap, objective, and run-time, we present average values.
Last but not least, we are going to report on the number of variables (Vars) used to model the VNMP instances. This allows us to judge the effectiveness of preprocessing in terms of model size. Note that we report the size of the model after CPLEX has performed its own reductions.
That has two consequences. First of all, this allows us to really see the benefit of preprocessing.
If we would report on the size of the model before the reductions, any reduction in the number of necessary variables caused by preprocessing could in principle be matched by the reductions performed by CPLEX. Preprocessing only adds real value if it reduces the number of variables in addition to the reductions of CPLEX. The second consequence and downside of using the number of variables after the reductions is that the results are not necessarily consistent. We have seen instances where the final number of variables is smaller if weaker preprocessing is used. Of course, the number of variables in a model is only an indicator for the “hardness” of a model but by no means the sole determining factor. We could observe instances where a smaller model for the same instance took far longer to solve to optimality. The reported numbers of variables are based on instances that did not fail due to Mem.
Based on these definitions, we can present the main characteristics of different configurations of FLOW depending on the instance size in Table 11.2.
For the smallest instance sizes, we can see that every configuration is able to solve all of the 120 instances (30 instances with four load levels each). The average time required to to so however is very different. Unsurprisingly, FLOW-B requires the most run-time with more than 100 seconds on average. By adding preprocessing without any pruning and fixing, the run-time requirements can be reduced to one third of the original run-time.
Interestingly, adding additional pruning capabilities increases the required run-time in some cases. The reason is not the run-time required by the enhanced preprocessing methods, as we know from Chapter 9 that they are negligible compared to the total run-time. The reason is the performance variability of ILP [36, 109]. Simple changes to a model which should be perfor-mance neutral (or improve perforperfor-mance) like changing the order of constraints may sometimes cause unexpected degradations. At least the number of variables present in the ILP model is re-duced by using enhanced preprocessing methods. Starting to fix nodes gives another boost to the preprocessing performance. FLOW-F only requires less than 17% of the run-time of FLOW-B, but linking flow with node usage by equality does not give an additional advantage. By adding the possibility of buying additional resources, we slow the solving process down significantly. It also causes a noticeable increase in variables necessary for modeling the VNMP instances. The two configurations that focus on finding a valid solution, FLOW-MA and FLOW-S, unsurpris-ingly are much faster than the optimizing configurations. The solutions that are found by them are about 50% more expensive than the optimal solutions with respect to substrate usage costs.
Starting with instance size 30, the configurations begin to fail. Most noticeable of them is the FLOW-B configuration, which is not able to find an integer feasible solution for one instance.
Table 11.2: Main characteristics of different configurations of FLOW depending on the instance size.
Size Method Mem LP NS AR Feas Opt Gap[%] Obj t[s] Vars
20 FLOW-B 0 0 0 0 0 120 0.0 803.2 102.2 6269.7
FLOW-P1 0 0 3 0 16 101 0.4 1742.4 2006.5 21382.1
FLOW-P2 0 0 1 0 14 105 0.6 1753.8 1718.0 17825.8
FLOW-P3 0 0 0 0 16 104 0.6 1760.0 1676.9 15193.8
FLOW-F 0 0 1 0 15 104 0.5 1752.8 1639.7 15033.7
FLOW-FE 0 0 2 0 12 106 0.3 1746.5 1601.0 15050.5
FLOW-A 0 0 0 1 17 102 0.5 1751.9 1879.9 15971.0
FLOW-MA 0 0 0 0 0 120 0.0 3131.5 2.9 16106.5
FLOW-S 0 0 0 0 0 120 0.0 3179.0 2.7 15288.9
100 FLOW-B 1 45 6 0 27 41 1.4 2939.1 6780.5 212914.1
FLOW-P1 0 8 14 0 48 50 1.7 3291.9 6054.2 71727.4
FLOW-P2 0 3 6 0 53 58 1.6 3385.3 5534.4 57764.0
200 FLOW-B 0 69 4 0 13 34 0.4 3325.1 7410.6 457143.5
FLOW-P1 0 39 9 0 31 41 1.6 3973.1 6856.6 133156.1
FLOW-P2 0 24 13 0 37 46 1.7 4218.3 6514.9 109582.3
FLOW-P3 0 18 17 0 41 44 1.7 4253.3 6455.4 93957.8
FLOW-F 0 17 18 0 38 47 1.5 4264.8 6343.9 92999.5
FLOW-FE 0 15 15 0 41 49 1.5 4342.4 6275.2 92228.6
FLOW-A 0 20 16 1 36 47 1.4 4191.6 6372.2 102392.0
FLOW-MA 0 0 0 0 0 120 0.0 8849.6 28.5 102075.4
FLOW-S 0 0 0 0 0 120 0.0 8872.3 27.1 91624.3
500 FLOW-B 73 6 10 0 5 26 0.2 3171.8 8131.2 433322.5
FLOW-P1 0 84 2 0 7 27 0.2 3448.6 7879.0 316870.9
FLOW-P2 0 68 3 0 15 34 0.7 4423.6 7384.5 250480.3
FLOW-P3 0 67 2 0 18 33 0.7 4503.6 7440.8 243814.4
FLOW-F 0 64 6 0 17 33 0.6 4488.7 7385.6 241183.3
FLOW-FE 0 63 2 0 22 33 0.7 4706.8 7334.0 240455.4
FLOW-A 0 67 6 0 14 33 0.4 4293.6 7452.2 279763.4
FLOW-MA 0 0 0 0 0 120 0.0 13381.6 146.0 278027.4
FLOW-S 0 0 0 0 0 120 0.0 13774.0 174.6 241346.1
1000 FLOW-B 90 3 4 0 15 8 1.5 3677.5 9707.1 416302.4
FLOW-P1 19 61 13 0 11 16 0.7 3721.4 9104.5 468323.0
FLOW-P2 3 66 14 0 13 24 0.6 4657.2 8209.1 420043.5
FLOW-P3 3 65 15 0 12 25 0.6 4617.8 8146.5 418783.2
FLOW-F 2 64 14 0 13 27 0.6 4834.0 8058.6 420225.3
FLOW-FE 2 59 14 0 18 27 0.7 5277.2 7978.8 417890.5
FLOW-A 3 68 8 0 15 26 0.8 4948.1 8062.7 509628.8
FLOW-MA 15 0 0 0 0 105 0.0 16698.6 1630.9 463200.4
FLOW-S 1 0 0 0 0 119 0.0 21265.4 673.8 422138.7
Other than that, the solving performance of all configurations is similar, between 110 and 115 instances can be solved to optimality, for the rest we get feasible solutions. The average gap is also very small. Notice how FLOW-B seems to produce better results based on the reported objective value than the other configurations. This is because the result for the instance that failed with NS is missing from the average objective value. Again, we can see significant run-time improvements by activating preprocessing and this run-time using improved pruning (FLOW-P2) gives an additional advantage. Notice how preprocessing is able to reduce the number of required variables to a third of the original value. FLOW-FE shows a distinct advantage when compared to FLOW-F in terms of required run-time and is even able to prove optimality for one more instance. For size 30, FLOW-A is the fastest solution method, the ability to add additional resources no longer seems to be a disadvantage. The last two configurations are still very fast and able to find valid solutions to all instances. Note however that the gap to the optimal solution values increases, the found solutions now being 60% more expensive than the optimal solutions.
By increasing the substrate size to 50 nodes, we can observe the first failures of FLOW-B due to the memory limit. None of the other methods have problematic memory consumption, but they start failing due to NS. Also, the number of instances where only a valid solution could be found increases. Still, all methods employing preprocessing can solve more than 100 instances to optimality. FLOW-A is again slower than the other methods and also keeps being slower for larger instance sizes. Also notice how the run-time advantage of methods using preprocessing starts to shrink, but the difference in variables is still huge.
At size 100 we have reached the point where only half of the instances can be solved to opti-mality by the optimization configurations using preprocessing (FLOW-P1 – FLOW-A). For the remaining instances we mostly find feasible solutions, but for some we fail earlier at LP or NS.
The most common result for FLOW-B is to fail at LP, which means gaps and objective values cannot be compared meaningfully with the other algorithms.
By doubling the instance size again to 200, we get a significant number of failures for all op-timization configurations, but still a surprising number of instances can be solved to optimality by the configurations using preprocessing. This is the last size for which we can compare the number of variables in a meaningful way, as FLOW-B starts failing at Mem for the larger sizes.
FLOW-B almost requires five times more variables than FLOW-FE. For this size class we also have the second and last occurrence of a solution found by FLOW-A that requires additional resources.
For size 500, FLOW-B requires too much memory in most cases. All other optimization con-figurations predominately fail at LP. Observe that if a valid solution can be found, in most cases also its optimality can be proven or the remaining gap is very small.
Applying the different FLOW configurations to the last instance size, we can see that also FLOW-P1 starts to fail a significant number of times due to Mem. The configurations using more preprocessing fail predominantly at LP. Again, if we find a valid solution, we either achieve low gaps or can prove optimality, but this happens less often than for the previous size class. Also the configurations concentrating on just finding valid solutions start to fail, especially FLOW-MA, which needs too much memory. FLOW-S just fails once due to Mem and manages to find valid solutions for all other instances. Here we can clearly see the run-time and memory cost of not only finding valid solutions, but also being able to derive cheap recourse actions if the VNMP
Table 11.3: Main characteristics of different configurations of FLOW depending on the instance load.
Load Method Mem LP NS AR Feas Opt Gap[%] Obj t[s] Vars
0.10 FLOW-B 0 3 4 0 23 180 0.2 1866.3 1812.7 97877.5
FLOW-P1 0 0 3 0 16 191 0.1 1906.7 1237.4 24635.7
FLOW-P2 0 0 0 0 7 203 0.0 1937.8 428.8 16877.0
0.50 FLOW-B 44 25 13 0 21 107 0.2 1985.6 5213.3 201956.9
FLOW-P1 0 58 3 0 32 117 0.5 2348.0 4614.2 130523.2
FLOW-P2 0 31 9 0 39 131 0.5 2891.2 4135.5 104773.1
FLOW-P3 0 27 11 0 42 130 0.5 2924.8 4061.7 100501.8
FLOW-F 0 26 11 0 43 130 0.4 2959.5 4065.7 99785.8
FLOW-FE 0 22 6 0 49 133 0.5 3201.5 3955.1 99474.9
FLOW-A 0 27 10 0 42 131 0.4 2979.1 4084.8 117538.6
FLOW-MA 0 0 0 0 0 210 0.0 6697.3 45.6 115672.5
FLOW-S 0 0 0 0 0 210 0.0 6888.1 44.8 98781.1
0.80 FLOW-B 60 45 3 0 23 79 0.7 1766.8 6442.6 201773.8
FLOW-P1 0 75 11 0 38 86 1.0 2306.7 6128.9 209788.7
FLOW-P2 0 66 7 0 44 93 1.1 2586.2 5973.0 168153.4
FLOW-P3 1 64 6 0 49 90 1.1 2623.4 5923.0 161912.1
FLOW-F 0 61 8 0 49 92 1.1 2686.2 5927.3 160086.1
FLOW-FE 0 61 6 0 48 95 0.9 2734.8 5831.7 159641.8
FLOW-A 2 63 8 0 48 89 1.0 2603.3 6004.9 188727.4
FLOW-MA 0 0 0 0 0 210 0.0 8959.1 111.9 185858.2
FLOW-S 0 0 0 0 0 210 0.0 9879.9 113.1 158224.1
1.00 FLOW-B 61 51 9 0 25 64 1.0 1739.9 7207.3 253519.9
FLOW-P1 19 59 24 0 35 73 1.4 2157.8 6782.6 194607.1
FLOW-P2 3 64 21 0 48 74 1.8 2493.8 6654.7 201272.1
FLOW-P3 3 62 19 0 50 76 1.8 2549.5 6674.9 192672.3
FLOW-F 2 61 22 0 50 75 1.5 2542.8 6626.9 194292.1
FLOW-FE 2 57 24 0 50 77 1.5 2634.3 6612.9 193650.0
FLOW-A 3 68 15 2 48 74 1.3 2430.0 6686.0 223339.1
FLOW-MA 15 0 0 0 0 195 0.0 9717.4 871.7 179565.3
FLOW-S 1 0 0 0 0 209 0.0 11956.2 340.3 194892.7
instance does not have a valid solution. For the smaller sizes, there was no clear difference between FLOW-MA and FLOW-S.
We have discussed the characteristics of different FLOW configurations based on the instance size. Table 11.3 shows the same characteristics, but now based on the instance load. For the lowest load of 0.1, finding at least feasible solutions is not a problem, just FLOW-B fails at LP or NS for some instances. All other configurations manage to find optimal solutions for most instances. Note that for low loads the run-time requirements are very different for the compared
configurations, higher loads will cause the run-time requirements to be more similar. The same holds true for the number of variables. Also observe that the solutions found by FLOW-F and FLOW-FE are the same, but FLOW-FE is faster. Both configurations are better than FLOW-P3.
By increasing the load to 0.5, we see that FLOW-B already starts to fail due to Mem, all other configurations start failing due to LP. Most instances can still be solved to optimality. Going to load 0.8 increases the number of failures due to LP further, we also start seeing failures due to Mem for configurations other than FLOW-B. The average gap reaches 1%. The main difference when using VNMP instances with full load is that for some instances the different configurations fail to find valid solutions more often. It can also be seen that the two instances for which FLOW-A produces a solution which requires additional resources are those with highest load. There is not a lot of observable difference between the characteristics of the configurations FLOW-P2 to FLOW-A, only FLOW-P1 and FLOW-B are worse.
Based on these results, one thing is clear: if we want to tackle instances with more than 50 substrate nodes and with more than 50% load with FLOW, then preprocessing in some form is essential. For the largest instance sizes and highest loads, using more advanced preprocessing techniques becomes important. Valid solutions can be found to nearly all instances in surpris-ingly short time. Especially for large instances with high loads, solving the LP relaxation in the root node is very time consuming. The differences between the configurations from FLOW-P1 to FLOW-A are not very visible and require further analysis. Preprocessing has a very pro-nounced effect on the number of variables, which also translates to shorter run-times, albeit to a lesser extent. This is not surprising since in some sense preprocessing just removes unnecessary ballast but does not make the core problem easier to solve.