2.2 Exact methods
2.2.3 Linear Programming
Linear programming (LP) defines the problem of minimizing or maximizing a linear cost function subject to linear inequalities. LP has been shown to be in the complexity class P which means that for any problem which can be formulated as an LP, there exists a polynomial-time algorithm which solves the problem. First of all, we will give a formal definition of a linear program:
min c0x (2.3)
s.t. ai0x≥bi ∀i∈M1 (2.4)
ai0x≤bi ∀i∈M2 (2.5)
ai0x=bi ∀i∈M3 (2.6)
xj ≥0 ∀j∈N1 (2.7)
xj ≤0 ∀j∈N2 (2.8)
The linear cost function (2.3) is also denoted as objective function. The feasible region of the problem is described by inequalities (2.4)–(2.6) and the domain of the variables is described in inequalities (2.7) and (2.8). Every set of feasible variable assignments, i.e., which lies in the feasible region of the problem is regarded as feasible solution. The solution from the feasible set of solutions, which minimizes the objective function (2.3) is denoted as optimal solution. Note, that there maybe more than one solution that minimizes the objective function in which case all those solutions would be denoted as optimal solution. The value of the objective function for the optimal solution is regarded asoptimal cost.
An LP is said to be instandard form if it looks like:
min c0x (2.9)
s.t. Ax=b (2.10)
x≥0 (2.11)
By transforming all inequalities of an arbitrary LP to the formAx=band having only non-negative variables x≥ 0 every LP can be brought into standard form where the process of bringing an LP into standard form is regarded toreduction to standard form.
To formally define the feasible region of an LP we define apolyhedron [10].
Definition 1. A polyhedron is a set that can be described in the form{x∈Rn|Ax≥b}, where A is an m×n matrix and b is a vector in Rm.
Now, we define a solution of an LP. First of all, we need to describe the term of anactive constraint:
Definition 2. A constraint of the form ax∗ =b is said to be active if vectorx∗ satisfies it.
Then, we can define basic solutionsand basic feasible solutionsas follows:
Definition 3. Consider a polyhedron P defined by linear equality and inequality con-straints, and letx∗ be an element of Rn.
• Vector x∗ is a basic solution if all equality constraints are active and out of the constraints that are active at x∗, there aren of them linearly independent.
• If x∗ is a basic solution that satisfies all of the constraints, it is called a basic feasible solution.
Definition 3 formally defines whatbasic feasible solutionsare, but we are mostly interested in solving the LP to optimality. Thus, we need also information about the optimality of extreme points:
2.2. Exact methods Theorem 1. Consider an LP minimizing cx over a polyhedron P. Suppose that P has
at least one extreme point and that there exists an optimal solution. Then, there exists an optimal solution which is an extreme point of P.
For a proof of Theorem 1 see [10]. As basic feasible solutionsand the term optimality are now introduced we can start thinking of algorithms which are able to solve LPs. This is explained in the next section.
Solution Algorithms for Linear Programs
Well known are the following three different methods for solving LPs:
• simplex algorithm,
• ellipsoid method, and
• interior point method.
Since it was shown by Khachiyan [80] that the ellipsoid methodis able to solve LPs in polynomial time, it is known that LPs are generally in the complexity classP. This was also shown by the work of Karmarkar [79], an Indian mathematician, in 1984, where he proposed the interior point method which is also a polynomial-time algorithm for solving LPs. Even, if the well knownsimplexalgorithm of Dantzig [33] has an exponential runtime in the worst-case scenario, it is effective in practice [88] and implemented in most commercial (mixed integer) linear-programming solvers, such as CPLEX or Gurobi.
Basically, the idea of the simplex method is to walk along the edges of the feasible polyhedron in direction of reduced costs (in case of a minimization problem) until no further improvement in the current solutions neighborhood is possible. The simplex algorithm terminates after a finite number of steps. A full implementation guide including also performance enhancements can be found in [10].
Duality
Duality is an important property in LP and it is also a substantial property for finding dual bounds. Most notably, every feasible solution found through the dual problem provides a dual bound on the objective value of the primal problem.
Definition 4. Two problems
(MIP) z= max{c(x) :x∈X} (2.12)
(D) w= min{w(u) :u∈U} (2.13)
form a weak-dual pair if c(x) ≤ w(u)∀x ∈ X and all u ∈ U. If z = w, they form a strong-dual pair.
Theorem 2. Suppose x is a feasible solution to the primal problem and u a feasible solution to its dual problem, then
p·b≤c·x. (2.14)
if its a minimization problem.
Therefore, any feasible solution of the dual problem gives a dual bound on the objective of the primal problem which is a useful property when solving LPs.
2.2.4 (Mixed) Integer Linear Programming
The basics for (mixed) integer linear programming lies in the theory of LP and with the simplex algorithm of Dantzig there already exists a practically efficient solution method for those programs. However, LPs can only cope with real numbers which means that many practical problems cannot be expressed with an LP. Imagine, a company is planning to build new factories and they have to decide where to place these new facilities. Therefore, a decision is needed whether to build or not to build a factory on a particular place. This cannot be expressed by an LP. When integer variables are needed to model a problem we speak about (mixed) integer linear programs. A model which contains only integer variables is called integer program, a model containing only binary decision variables, i.e., all variables can only take a value of zero or one are called binary programs, and if a model contains integer as well as real-valued variables it is a mixed integer linear program.
The question arising is, how MIPs can be (efficiently) solved. An obvious way in solving those models is to solve the LP instead of the MIP by replacing all integer variable domains with a real-valued domain. Rounding the fractional values to integer values could be a solution to retrieve a valid integer solution for the model. But, of course, rounding is not sufficient as this rounded solution can be far away from the optimal integer solution. For finding an algorithm that solves MIPs, some information is needed at which point a given solutionx∗ can be proved optimal. Let z be the optimal solution of integer programIP, ¯zan upper bound ofz andza lower bound. The optimal solution is reached when
z¯−z≤ (2.15)
where is a small optimality tolerance. The question is how to find upper and lower bounds to the optimal solution of the MIP. First, we define what a relaxation of the original MIP is.
A simple and useful relaxation ofIP is the so called LP relaxation when the domains of the integer variables are changed to real values. Obviously, the optimal solution to the LP relaxation is a lower bound or dual bound to the optimal solution (in case of minimization).
Upper bounds or primal bounds are any feasible solution to the problem/model.
As all other types of problems, also a MIP could be solved to optimality with complete enumeration. However, as this is not tractable a more sophisticated solution approach
2.2. Exact methods has to be found. An approach is described in the next section, namelyLP-based
branch-and-bound.
LP-based branch-and-bound
When considering an integer program z= max{cx:x∈S}, the question is how it can be divided into subproblems such that it can be solved faster and more efficiently.
Proposition 1. Let S1, . . . , Sk be a decomposition of S into smaller problem, and let zk = max{cx:x∈Sk}∀k= 1, . . . , K. Then, z= maxkzk.
A way of reaching such a decomposition is to solve the so called LP relaxation of the MIP.
Usually, this results in some integer variables being fractional. If this is not the case the optimal LP solution is also the optimal solution of the MIP. A way of eliminating such fractional variables is to branch over these variables and make them integer. Again a full enumeration tree would not be tractable, so a smarter method has to be applied. This is done by computing bounds and pruning the tree, similar as described in Section 2.2.1.
In general there are three types of pruning in a branch-and-bound tree:
prune by optimality if lower and upper bound on a node in the tree have the same value there is no need to expand the node further.
prune by bound if the best found primal bound so far is higher (for maximization problems) than the dual bound in the current node it is not possible to find the optimal solution in the subtree of this node.
prune by infeasibility if the solution would become infeasible, the subtree of the current node can be discarded.
Using this information an algorithm can be implemented which uses LP-based branch-and-bound. However, there are further decisions to be made, like, e.g., in which order should the tree be traversed, in which order should the nodes be expanded. These decisions influence the performance of the algorithm.