Weak duality refers to the observation that the dual objective values are lower bounds for the primal objective values. This is true for all convex optimization problems, but because of the special structure of conic programming problems we can derive an explicit form of the dual function and show that the dual problem is also a conic optimization problem.
The Lagrangian for problem (PC) is
L(x, y, s) =cTx+yT(b−Ax)−sTx,
withx∈ Kands∈ K?. For all feasiblex,y∈Rmands∈ K?, the inequality L(x, y, s) =cTx−sTx≤cTx (3.1) holds. Therefore thedual function defined by
f?(y, s) = inf
x {L(x, y, s)}
will satisfy f?(y, s) = inf
x {L(x, y, s)} ≤inf
x {L(x, y, s), x∈ F } ≤inf
x
cTx, x∈ F =p?, wherep? is the primal optimal value. This implies
f?(y, s)≤p?≤cTx, (3.2)
which in words means that any dual objective value will be a lower bound for all primal objective values.
Observe that ifc−ATy−s= 0 thenL(x, y, s) =bTy. However ifc−ATy−s6=
0, then ∆x = −c+ATy +s satisfies L(α∆x, y, s) = bTy−αk∆xk22 and as α→ ∞, L(α∆x, y, s)→ −∞. In this case L(x, y, s) is unbounded below and we can conclude that
f?(y, s) =
(bTy c−ATy−s= 0,
−∞ otherwise.
The dual problem in standard form incorporates constraints for the region wheref?(y, s) is finite and has the form
maximize
x∈Rn
bTy subject toATy+s=c,
s∈ K?. (DC)
A consequence of weak duality is that, when the primal is unbounded it admits no lower bound, and therefore there can be no feasible dual point. On the other hand, unboundedness of the dual implies that the dual objective values admit no upper bound and there can be no feasible primal point. Conversely if there is a feasible primal point the dual must be bounded, and if there is a dual feasible point the primal must be bounded.
It is important to understand not only when problems are solvable but when the solution set is bounded, for this has algorithmic consequences. We can show that whenever there exists astrictlyfeasible primal point, the optimal set of the dual problem is bounded, and conversely when there exists a strictly feasible dual point, the optimal set of the primal problem is bounded. To show this we need the following lemma, which shows that a problem is unbounded if and only if there exists a recession direction along which the objective is reduced.
Lemma 3.2.1. The primal problem is unbounded iff there exists a recession direction ∆xfor the feasible setF such thatcT∆x <0.
Proof. If such a direction exists then the problem is unbounded, for if x is a feasible point, the half-line x+α∆xfor α >0 is feasible and cT(x+α∆x) = cTx+α(cT∆x)→ −∞asα→ ∞.
To show the converse, assume that the primal problem is unbounded. Choose an arbitrary pointx0∈ F and for every k∈N form the setCk =F ∩ {cTx≤
−k} and the set ˆCk ={kx−xx−x0
0k x∈Ck}. The sets ˆCk are nonempty bounded and closed and the sequence ˆCk+1 ⊆ Cˆk is monotonically decreasing. The Cantor intersection theorem states that there exists adsuch that d∈ ∩∞i=0Cˆk. Therefore there exists a sequence ofβk >0 such thatx+βkd∈Ck⊆ F.
The sequenceβk admits no upper bound, for−k > cT(x0+βkd)> cTx0− βkkdkkck implies that βk > k−ckdkkckTx0 and thereforex0+βd ∈ F for allβ >0.
This establishes thatdis a recession direction. Finally ifcTx0=p0andcTd≥0 thencT(x0+βkd)≥p0 for allβk reaching a contradiction. This implies thatd is a recession direction with cTd <0.
With this result it is simple to prove that if primalstrictly feasible points exist then the dual problem is bounded, and that if dualstrictly feasible points exist then the primal is bounded.
Lemma 3.2.2. If the dual problem is strictly feasible, then no recession di-rection ∆x with cT∆x≤ 0 exists and the dual is bounded. If furthermore the primal problem is feasible, then the solution exists and the dual solution set is bounded.
Proof. Assume there exists a strictly feasible dual ATyˆ+ ˆs=c and that there is a recession direction for the primal feasible set ∆x such that cT∆x ≤ 0.
Since A∆x= 0, we have ˆyTA∆x=cT∆x−ˆsT∆x= 0 and therefore cT∆x= ˆ
sT∆x >0, which contradicts the existence of the recession direction. The strict inequality is due to the strict feasibility of ˆs. This result also implies that if the solution set exists, it must be bounded, otherwise there must exist a recession
direction for the feasible set where (x?+α∆x)Tc = p? for all α ≥ 0 with
∆xTc= 0.
3.2.1 Strong duality
Denote byp?=cTx? (the primal objective value at the solution), and byd?= bTy?(the dual objective value at a solution of the dual problem). A primal-dual pair is said to satisfystrong duality if the equalityd?=p?holds. Strong duality has several consequences, one of them being the condition that at the solution s?Tx?= 0.
Observe that
d?=f?(y?, s?)≤ L(x?, y?, s?) =cTx?−y?T(b−Ax?)−s?Tx?=p?−s?Tx?≤p?=d?, so that
p?−s?Tx?=p?,
ands?Tx? = 0. The condition s?Tx? = 0 is calledcomplementarity and as we will now show, is a sufficient condition for a primal dual feasible point to be optimal.
Lemma 3.2.3. If (x, y, s) is primal and dual feasible and if xTs = 0, then (x, y, s)is primal and dual optimal.
Proof. Assumex, y, sare primal and dual feasible, and thatxTs= 0.
xT(ATy+s) =xTc (3.3)
=⇒ bTy=xTc (3.4)
=⇒ f?(y, s) =f(x). (3.5)
Thereforef(x) =p? andf?(y, s) =d?.
3.2.2 Conditions for strong duality
For linear optimization problems strong duality always holds, but for conic programming the picture is more complicated. However, if there exist strictly feasible primal and dual points, then strong duality holds at all solutions.
Theorem 3.2.4. If there exists a strictly feasible primal-dual point, then the primal and dual are solvable and strong duality will hold at all optimal pairs x?, s?.
Proof. Since the dual problem is strictly feasible, Lemma 3.2.2 implies that the primal is bounded. However, the primal problem is feasible by assumption and therefore the primal is solvable. The same argument with the role of primal and dual reversed shows that the dual is also solvable. Furthermore, from Lemma 3.2.2 it follows that the solution sets of the primal and dual problem are bounded.
To show that strong duality holds we construct the somewhat artificial sys-tem
A˜=
A −b
−cT p?−1
,
wherep?is the primal optimal value, and let ˜K be the cone ˜K=K ×R+×R+. By the theorem of the alternative (2.1.9) one of the two following statements must be true:
1. There existsx∈intK,τ >0, κ >0 such that A −b
−cT p? −1
x τ κ
= 0.
2. There exists −y
−η
such that
AT −c
−bT p?
−1
−y
−η
∈ K?×R+×R+.
The first case never holds because if it did, we would haveAxτ =b,p?> cT xτ and
x
τ is a primal feasible point with a lower objective value than p?. Therefore, a solution −yT, −ηT
must exist for the second system. Observe thatη6= 0 for ifη= 0 then ATy+s= 0 for somes∈ K? andbTy≥0, so (y, s) is a recession direction with bTy ≥ 0, which contradicts the strict feasibility of the primal.
Finally observe that the equations AT yη +sη =c, sη ∈ K?, andbT yη ≥p? hold.
Therefore yη,sη is a feasible dual point and weak duality implies thatbT yη =p?. This establishes the existence of a dual feasible point that achieves the primal optimal value. Therefore the dual optimal valued?has to be equal to the primal optimal value p?.
3.2.3 Certificates of infeasibility and unboundedness
Assume that there exists a dual direction with
bT∆y >0, AT∆y+ ∆s= 0, ∆s∈ K?. (CI) If (y, s) is a dual feasible point, we know from (2.1.6) that the point (y+ α∆y, s+α∆s) is feasible. Also along this direction the dual objective
f?(y+α∆y, s+α∆s) =−bT(y+α∆y)
can be increased arbitrarily. This implies that the dual problem is unbounded, and from weak duality we can conclude that the primal is infeasible.
On the other hand, if there exists a direction ∆xthat satisfies
A∆x= 0, cT∆x <0, x∈ K, (CU) thenx+α∆xis feasible for allα >0 and along ∆xthe objective can be decreased arbitrarily. This implies the primal is unbounded and the dual infeasible.
A certificate of unboundedness is a direction ∆xthat satisfies (CU) and a certificate of infeasibility is a direction ∆y,∆sthat satisfies (CI).