2.2 Combinatorial Optimization Problems and Solution Methods
2.2.12 Integer Linear Programming
Integer Linear Programming is a class of methods for solving COPs. In contrast to CP, it is far less flexible with respect to the types of COPs that can be solved. Only if the COP can be modeled as shown by Equations (2.1)–(2.3), i.e., as a system of linear inequalities with a linear objective function and using integer variables, Integer Linear Programming can be applied.
z= min c0x (2.1)
subject to Ax≥b (2.2)
x∈Zn+. (2.3)
Using additional real-valued variables (Mixed Integer Linear Programming) is also allowed, but it is not possible to model anN P-complete COP exclusively with real-valued variables (unless the model is exponential in size). In the following, we will consider Integer Linear Programming, but everything is applicable to Mixed Integer Linear Programming as well. To solve Constraint Satisfaction Problems, setc=0.
Branch & Bound can be applied to solve Integer Linear Programs (ILPs). Some definitions are required before we can cover the details.
Definition 2.2.12(Set of Integer-Feasible Solutions). The set of integer-feasible solutionsXof ILP (2.1)–(2.3) is defined asX ={x∈Zn+|Ax≥b}.
Following this definition, an ILP can also be written asz= min{c0x|x∈X}.
Definition 2.2.13(LP Relaxation). Given an ILP (2.1)–(2.3), the LP relaxation is obtained by replacing the integrality constraints (2.3) byx∈Rn+.
Algorithm 2.6:LP-based Branch & Bound (for minimization problems) Input: ILPmin{c0x:x∈X}
Output: Optimal solutionx∗
1 problem listL={X};// initialize problem list
2 x∗ =0;// best known feasible solution
3 z=∞;// objective of best known feasible solution
4 whileL6=∅do
5 choose setXiand remove it fromL;
6 (xiLP, zi) =LP(Xi);// get optimal LP solution and objective
7 ifLP(Xi) =∅thenpruneXi by infeasibility;
8 else ifzi≥zthenpruneXiby bound;
9 else ifxiLP∈Xthen// LP solution is integer
10 ifzi ≤zthen
11 x∗ =xiLP;// update best known solution
12 z=zi;// update best known objective
13 end
14 pruneXi by optimality;
15 end
16 elseL=L∪ {Xi,1, Xi,2};// partition search-space Xi
17 end
18 returnx∗;
The LP relaxation can be solved efficiently (in polynomial time) by ellipsoid [104] or interior point methods [103]. Much better in practice and more widely used is the simplex method [37], even though it can have an exponential run-time in the worst case [13]. Details on the simplex method can be found in [13]. By using this method, we can define the LP based Branch & Bound scheme (for minimization problems), as presented in Algorithm 2.6.
The problem listLis the central data structure of this algorithm. It contains all partitions of the search-space that have not yet been considered. When this list is implemented as a stack, we get the standard depth-first search (as we have outlined for Branch & Bound). In practice, there are a lot of improvements possible by intelligently choosing which partition to consider next.
In addition to L, we also need to store the best known feasible solution (which is the optimal solution at the end of this algorithm), and its objective. During the execution, this objective is an upper bound on the optimal value, hence the overline and initial value of∞.
After the initialization, the algorithm starts in earnest by considering every search space partition (subproblem) in turn, untilLis empty. For the currently considered subproblem, we solve the LP relaxation and get the optimal solution of the LP relaxationxiLPand its objectivezi. Since we do not enforce variablesxto be integer in the LP relaxation, the LP solution will be fractional in the general case. Its objective is a lower bound (hence underlined) on the objective of the optimal integer solution contained inXi. This is the bounding step, and depending on the result, different actions are executed. If Xi is not even feasible when considering its LP-relaxation
(i.e., there is not even a fractional assignment to all variables that satisfies all inequalities), then Xi cannot contain any integer feasible solutions and can be discarded. If the best fractional solution contained in Xi is not better than the globally best known integer feasible solution, Xi can be discarded without further consideration, since no integer feasible solution that will lead to an improvement exists within it. This step has the potential of quickly recognizing large parts of the search space as uninteresting and therefore hugely increasing performance. If the LP solution happens to be integer, then we store it, if it is better than the best known solution.
As a side-effect, this increases the probability of being able to prune by bound in subsequent iterations. Then we discardXi, since we have already identified the optimal solution contained within. If none of those conditions apply, i.e., the LP solution is fractional and better than the currently best known integer solution, thenXimight contain an improving integer solution and further analysis is required.
This leads us to the branching step, where we partitionXiinto two sets. This is done by selecting a variablex of the LP solution that has a fractional value v, and adding for one partition the constraintx ≤ bvcand for the other partition x ≥ dve. Note that ifxis a binary variable (its domain is{0,1}), it is now forced to be integer in both partitions. By partitioningXi in that way,xiLPis no longer a feasible solution for both partitions. The optimal LP solution of both partitions is likely worse thanzi, so in the best case it may be possible to immediately prune both partitions ofXi, while we could not pruneXi itself. Due to this branching, we can guarantee that the LP solution will be integer feasible at some point going down the search tree (or no LP solution exists and we need to track back), and the search tree depth is finite.
When modeling a COP with ILP, one aim is finding a strong model, that means that the optimal objective of its relaxation is close to the objective of the optimal integer solution. It is possible to add inequalities to a model to strengthen it, i.e., the inequalities are redundant and do not change the set of integer feasible solutions, but they remove feasible solutions of the relaxation.
These inequalities might be added to the model at the beginning or added during search as nec-essary. An added inequality is also called cut, because it cuts away previously feasible fractional solutions. The problem of finding such a violated cut for a given LP solution is called the (cut) separation problem. It is theoretically possible to add cuts to the model until the solution of the LP relaxation yields the optimal integer solution by means of the cutting plane algorithm [38].
Usually, the cutting plane algorithm is computationally too expensive.
Far more promising is the combination of the idea of cut generation and Branch & Bound, yielding highly effective Branch & Cut algorithms. At each node in the Branch & Bound tree, cuts are separated to strengthen the LP relaxation of the current subproblem. Procedures to automatically derive certain types of generic cuts, such as Gomory cuts [68] or Lift-and-Project cuts [8], are included in the major ILP solvers. With Branch & Cut, it is also possible so solve models which are exponential in the number of required inequalities, for instance forbidding all cycles of a specific length when formulating a problem on a graph structure. The initial model does not contain these constraints. Only if a constraint is violated by a LP solution, the cut enforcing the constraint is added.
Instead of adding constraints, it is also possible to dynamically add variables to a model. For the LP relaxation, this is called column generation [64], as a new variable adds a column to the coefficient matrixA. When combined with Branch & Bound for solving ILPs, it is called
Branch & Price. With this method, it is possible to use models with an exponential number of variables. Initially, an LP containing only a small fraction of the variables is solved. An important condition here is that this small fraction of variables has to permit a feasible solution.
This is problematic when even finding a feasible solution is N P-complete. After the LP is solved, a variable that is currently not included in the model and may improve the solution of the LP has to be identified. This is called the pricing problem. Only variables with negative reduced costs may improve the solution, see the simplex algorithm [37] for details. By means of Danzig-Wolfe decomposition [39], it is possible transform a compact model into a model with exponentially many variables. The transformed model usually has much tighter LP bounds when compared to the original compact model which reduces the number of Branch & Bound nodes that need to be considered.
As an example for a model with exponentially many variables, consider a variant of the Vehicle Routing Problem (VRP), where packets have to be delivered to customers by a fleet of vehicles.
Each vehicle starts from the central depot, delivers packets to a set of customers and returns to the depot. The total length of the driven distance by all vehicles has to be minimized. All packets that need to be delivered by a vehicle need to fit into the vehicle. One decision variable could represent a specific route of a vehicle that is valid with respect to the capacity constraint of the vehicle. There are exponentially many such variables. In the model for this VRP, we select one route for each vehicle, such that every customer is visited exactly once, and the total cost of the routes is minimized.
One point of interest of the column generation approach (besides potentially better LP bounds) is that it is possible to hide arbitrary constraints within the pricing subproblem and the LP model deals only with variables that respect those constraints. However, there are also some downsides.
First of all, column generation cannot be stopped during execution and still yield a valid LP bound. Only after we have proven that no more improving variables exist is the LP bound valid.
Otherwise it might be too high (for a minimization problem), which means we might then prune subproblems during Branch & Bound that may still contain improving integer feasible solutions.
The cutting plane method can be stopped and we get a valid (if not as strong as possible) lower bound. Proving that no further cuts exist is only necessary when an integer solution is found, otherwise the integer solution could be invalid for the complete (exponential) model. Branching sometimes is also problematic for Branch & Price. Assume we need to chose (i.e., set to 1) one of exponentially many variables. When we branch on one of those variables, we either fix it to 1 (and are finished unless there are also other variables), or we fix it to 0 and have basically no additional restriction since there are still exponentially many other variables that may need to be generated and branched upon. As a result, the search tree is very skewed. The third major hurdle of column generation are convergence issues. Without taking care, a lot of variables will be generated that improve the LP bound only a tiny bit which leads to long convergence times.
Stabilization techniques [45] are required to combat this. Despite these challenges, Branch &
Price has been very successfully applied to Cutting & Packing and Network Design problems.
For an overview on column generation, see [44, 118].
1