Greedy Algorithm And
Matroid Intersection Algorithm
Summer Term 2010
Talk Summary
Paul Wilhelm
1 Greedy Algorithm
Abstract
Many combinatorial optimization problems can be formulated in terms of indepen- dence systems. In the first part I want to introduce the class of greedy algorithms, which find an optimal solution for all weight functions if and only if the inde- pendence system is a matroid. For some problems like finding the matching in a bipartite graph or the travelling salesman problem the system is not a matroid.
But I prove that every independence system is the finite intersection of matroids.
The problem of finding the matching in a bipartite graph, can be described as intersection of two matroids. For this case I present the Edmond’s Intersection Algorithm which solve the problem in polynomial time.
1 Greedy Algorithm
The maximization problem is defined as the task to find a setX ∈ I such that the weight w(X) = P
e∈Xw(e) for w : E → R is maximized. For the maximization problem, we can define the best-in-greedy algorithm for a given independence oracle:
1.1 Best-In-Greedy Algorithm
Sort E ={e1, ..., en} s.t. w(e1)≥ ... ≥ w(en).
Set X :=∅ ∈ I.
For i:=1 to n do: If F ∪ {ei} ∈ I then set X :=X∪ {ei}.
Output: X ∈ I.
The complexity of sorting E is O(n·log(n)), but we have to ask the oracle in each step of the loop. Therefore the complexity of the algorithm mainly depends on the complexity of the independence oracle.
1.2 Feasibility In Matroids
In general the greedy algorithm find only a local optimum and the cardinality of the output has not to be maximum, the output is just maximal (i.e. there exists no independent superset). But for matroids we can prove the following theorem if we assume that the greedy algorithm work correctly in the independence system:
Theorem (Feasability). An independence system is a matroid if and only if the best-in-greedy algorithm for all weight functions finds an optimal solution for the maximization problem.
1.2 Feasibility In Matroids Proof : First we prove that in a matroid the best-in-greedy algorithm finds an optimal solution for all weight functions. Letwbe an abitrary weight function and X = {x1, ..., xr} the output of the greedy algorithm. Without loss of generality we may assume w(x1)≥ ...≥ w(xr) (if not we change the numbering). We show that X has a maximum cardinality by contradiction, we then use that to prove by contradiction that w(X) is maximum1:
Assume there exists a set Y ∈ I with greater cardinality than X. By the augmentation property follows, that there exists ay ∈Y\Xsuch thatX∪{y} ∈ I.
Therefore a t has to exist such that {x1, ..., xt, y, xt+1, ..., xr} ∈ I with w(xt) ≥ w(y) ≥ w(xt+1). Hence {x1, ..., xt} ⊆ {x1, ..., xt, y} ∈ I. But then y should be choosen in step t+ 1 of the greedy algorithm, what contradicts the correctness.
Next we assume that there exists a Y = {y1, ..., yq} ∈ I such that w(Y) >
w(X) and w(yi) ≥ w(yi+1). By the definiton of the weight function follows that P
yi∈Y w(yi) > P
xi∈Xw(xi) and because |Y| ≤ |X| there exists a k such that w(yk) > w(xk). Define X0 := {x1, ..., xk−1} (= ∅ if k = 1) and Y0 := {y1, ..., yk}.
Because of the augmentation property there exists a yt∈Y0\X0 with t ≤k such that {x1, ..., xk−1, yt} =X0∪ {yt} ∈ I. But then yt should be choosen before step k of the greedy algorithm, because w(yt) ≥ w(yk) > w(xk). This contradicts the assumption that X is the correct output of the greedy algorithm.
Secondly we want to prove by contradiction that the feasibility of the greedy al- gorithm for all weight functions implies that the independence system is a matroid.
We assume that the independence system is not a matroid, i.e. there exists sets I, J ∈ I with|I|<|J| such that for alle∈J\I the set I∪ {e}is dependent. Un- der this assumption the greedy algorithm would not work for the following weight function2 with a variableε >0 which we specify later:
w(e) :=
1 +ε , if e∈ I 1 , if e∈ J \I 0 , if e∈ E\ {I∪J}
For this weight function the greedy algorithm would first choose all elements of the set I, because they got the heighest weight. After this it can not choose any element of J, because of the assumption. Only elements of E\ {I ∪J} could be choosen afterwards. Therefore the weight of the outputX of the greedy algorithm is
w(X) =|I| ·(1 +ε) + 0.
1Due to (Oxley, 2006, pp. 63-64).
2Due to (Lee, 2004, p. 60).
2 Matroid Intersection
But for the weight ofJ holds the inequality
w(X) =|I|(1 +ε)< w(J) = |J\I|+|I∩J|(1 +ε)
forε < |I|−|I∩J||J\I| −1. This implies that the greedy algorithm would not have found the optimal solution for that ε, and that is a contradiction to the correctness of the algorithm for all weight functions.
2 Matroid Intersection
Proposition. Any independendence system is a finite intersection of matroids.
Proof : For each circuitC ∈ C in (E,I) we define IC :={F ⊆E|C\F 6=∅}. We will prove that every (E,IC) is an independence system, even a matroid3 and that the intersection of all IC is indeed I.
Obviously is ∅ ∈ IC because C\ ∅ 6=∅.
Also forI ⊆J with J ∈ IC is I ∈ IC because C\J 6=∅ impliesC\I 6=∅.
The independence system (E,IC) is a matroid because for all subsets of E the rankr is equal to the lower rank %:
If X ∈ IC then X has an unique maximal independent subset in IC namely X itself. Hence r(X) = %(X) := min{|Y| : Y ⊆ X, Y ∈ I and Y ∪ {x} 6∈ I ∀x ∈ X\Y}=|X|
If X 6∈ IC then every maximal independent subset of X is of the form X\ {c}
for any c∈C, because C\(X\ {c})6=∅. Hence r(X) = %(X) =|X| −1
As the rank is equal to the lower rank for all subsets of E the rank quotient of (E,IC) is one and therefore the independence system is a matroid.4
Now we show that (E,I) = (E,∩C∈CIC):
Every F ⊆ ∩C∈CIC is independent. If not there would exist a circuit C ∈ C with C ⊆F such that C\F =∅, what would contradict the assumption thatF is out of the intersection.
If F ∈ I then for all C ∈ C would be C \F 6=∅ and therefore F would be an element in the intersection.
To find a maximum cardinality set X out of the intersection of two matroids (E,I1) and (E,I2) we can use Edmonds’ matroid intersection algorithm. The idea of the algorithm is to start with X = ∅ ∈ I1∩ I2 and augment by one element in each step. When you find no more e ∈ E such that X = X∪ {e} ∈ I1∩ I2,
3Due to (Du, 2008, p. 24).
4See Propositon 13.7. (Korte and Vygen, 2007, p. 308).
2.1 Edmonds Matroid Intersection Algorithm then construct a special directed bipartite graph GX over the disjoint vertice sets X andE\X. Search in this graph for a certain shortest alternating pathP. With X = X4V(P) = X \V(P)∪V(P)\X augment X by one element and repeat until you can not augment X anymore.
Fori = 1,2 let Ci(X, e) be the unique circuit in the matroid (E,Ii) which is a subset ofX∪ {e}. For all X ∈ I1∩ I2, we can define the directed auxiliary graph GX by:
A(1)X :={(x, y)|y∈E\X, x∈C1(X, y)\ {y}}
A(2)X :={(y, x)|y∈E\X, x∈C2(X, y)\ {y}}
GX := (X∪E\X, A(1)X ∪A(2)X ).
Then we are searching for the shortest path from SX :={y∈E\X|X∪ {y} ∈ I1} to TX :={y∈E\X|X∪ {y} ∈ I2} to augment X.
Fig 13.2. from (Korte and Vygen, 2007, p. 324)
If SX ∩TX 6= ∅ and therefore X∪ {e} ∈ I1 ∩ I2 for all e ∈ SX ∩TX, we can augment X by any element in SX ∩TX.
If SX ∩TX =∅ and the length of the shortest SXTX-path is greater then zero, we augment the set by using the symmetric difference ofX and the vertices of the path.
If there exists noSXTX-path at all we are done andX ∈ I1∩I2got its maximum size.
This leads to the following algorithm with polynomial complexity on the maxi- mum of the two independence oracles5:
2.1 Edmonds Matroid Intersection Algorithm
Set X :=∅.
While we can augment X do:
Add all elements e such that X =X∪ {e} ∈ I1∩ I2. For all y∈E\X compute Ci(X, y).
Compute SX, TX, and GX.
Find shortest SXTX-path P in GX.
If no P exists stop, else set X=X4V(P) and repeat.
Output: X.
To prove the correctness of this algorithm we need three lemma from (Korte and Vygen, 2007, pp. 323-325), the first two show that indeed X4V(P)∈ I1∩ I2
5A short proof of the complexity you can find in (Korte and Vygen, 2007, Theorem 13.32 p.
326), a more detailed proof you can find in (Papadimitriou and Steiglitz, 1982, pp. 297-298).
2 Matroid Intersection
and the third shows that the algorithm finds the set of maximum cardinality in the intersection:
2.2 Correctness Of The Algorithm
Lemma (13.27). Let (E,I) be a matroid and X ∈ I. Let x1, ..., xs ∈ X and y1, ..., ys 6∈X with
(a) xk ∈C(X, yk) for k = 1, ..., sand (b) xj 6∈C(X, yk) for 1≤j < k ≤s.
Then is (X\ {x1, ..., xs})∪ {y1, ..., ys} ∈ I.
Proof : Let be Xr := (X\ {x1, ..., xr})∪ {y1, ..., yr}, then we want to prove that Xr ∈ I for all r ∈ {1, ..., s} by induction:
For r= 0 we have nothing to prove because X0 =X ∈ I.
In the inductive step we are assuming that Xr−1 ∈ I and want to prove that Xr ∈ I by case destinction:
If Xr−1∪ {yk} ∈ I it follows by I2 that (Xr−1\ {xr})∪ {yr}=Xr ∈ I.
If Xr−1∪ {yk} 6∈ I, we prove that Xr ∈ I by contradiction. If we assume that Xr−1 \ {xr} ∪ {yr} 6∈ I, there has to exists a circuit C ⊆ Xr−1 \ {xr} ∪ {yr}.
Additionally there exists a circuitC(X, yr)⊆X∪ {yr}for which even C(X, yr)⊆ X\ {x1, ..., xr−1} ∪ {yr} because of b). This circuits are distinct, because xr 6∈ C by assumption and xr ∈ C(X, yr) by a). Hence yr ∈ C ∪C(X, yr), otherwise C ⊆ Xr−1 or C(X, yr) ⊆ X would be and therefore independent, what would contradict the fact that both are circuits. Now by C36 follows that there exists a circuitC3 ⊆(C∪C(X, yr))\ {yr}, for which holds:
C3 ⊆([Xr−1\ {xr} ∪ {yr}]∪[X\ {x1, ..., xr−1} ∪ {yr}])\ {yr}
⊆[X\ {x1, ..., xr} ∪ {y1, ..., yr−1}]∪X\ {x1, .., xr−1}
⊆X\ {x1, ..., xr} ∪ {y1, ..., yr−1}
⊆Xr−1 ∈ I
This would imply that there exists an independent circuit, what is a contradiction and therefore isXr ∈ I in contrary to the assumption.
6See (Korte and Vygen, 2007, Theorem 13.12 (C3) pp. 312-313).
2.2 Correctness Of The Algorithm Lemma (13.28). Let X ∈ I1 ∩ I2. Let y0, x1, y1, ..., xs, ys be in this order the vertices of a shortest y0-ys-path in GX, with y0 ∈SX and ys ∈TX. Then is
X0 := (X∪ {y0, .., ys})\ {x1, ..., xs} ∈ I1∩ I2. Proof : Because y0, ..., ys is a shortesty0-ys-path in GX we know that
1. (xj, yj)∈A(1)X :={(x, y)|y ∈E\X, x∈C1(X, y)\ {y}} and 2. (yj−1, xj)∈A(2)X :={(y, x)|y∈E\X, x∈C2(X, y)\ {y}}
for all j ∈ 1, ..., s. From this we can show that the requirements of lemma 13.27 and the properties a) and b) are fulfilled in both matroids.
First we want to show thatX0 ∈ I1:
We define Xe :=X ∪ {y0} which is independent because y0 ∈ SX. We know that xj ∈Xe because xj ∈C1(X, yj)\ {yj} ⊆X ⊆Xe and we knowyj ∈E\Xe because yj 6=y0 for allj = 1, ..., sand therefore the requirements of the lemma are fulfilled for X.e
Property a) of the lemma is satisfied, because from 1. for all j ∈ {1, ..., s}
follows that xj ∈ C1(X, yj) ⊆ C1(X, ye j). Property b) is satisfied, because if for any k < sa j < k exists such that xj ∈C1(X, ye k) =C1(X, yk)7 there would be a shortcutxjyk, what contradicts the fact thaty0x1y1...xjyj...xkyk...ysis the shortest x0-ys-path.
Altogether lemma 13.27 implies thatXe\ {x1, ..., xs} ∪ {y1, ..., ys}=X0 ∈ I1. Analog we want to show thatX0 ∈ I2:
We define Xe := X∪ {ys} which is independent because ys ∈ TX. We know that xj ∈ Xe because xj ∈ C2(X, yj−1)\ {yj−1} ⊆ X ⊆ Xe and we know yj−1 ∈ E \Xe because yj−1 6=y0 for allj = 1, ..., sand therefore the requirements of the lemma are fulfilled for X.e
Property a) of the lemma is satisfied, because from 1. for allj ∈ {1, ..., s}follows that xj ∈ C2(X, yj−1) ⊆ C2(X, ye j−1). Property b) is satisfied, because if for any j < s a i < j exists such that xj ∈ C2(X, ye i−1) = C2(X, yi−1)8 there would be a shortcut yi−1xj, what contradicts the fact that y0x1...yi−1xi...yj−1xj...ys is the shortest x0-ys-path.
Now lemma 13.27 implies that Xe \ {x1, ..., xs} ∪ {y0, ..., ys−1}=X0 ∈ I2.
7See (Korte and Vygen, 2007, Lemma 13.12 b)): X∪ {yk} ⊆X∪ {y0} ∪ {yk}contains at most one circuit and therefore both cycles are equal.
8See (Korte and Vygen, 2007, Lemma 13.12 b)): X∪ {yi−1} ⊆X∪ {ys} ∪ {yi−1} contains at most one circuit and therefore both cycles are equal.
References
Lemma (13.30). X ∈ I1 ∩ I2 is maximum if and only if there is no SX-TX-path in GX.
Proof : We prove by contradiction that if X ∈ I1∩ I2 is maximum there exists no SX-TX-path in GX. If we assume there exists a SX-TX path, there also exists a shortest one. We apply lemma 13.28 and obtain a setX0 ∈ I1∩ I2 with|X|<|X0| what contradicts the requirement that X is maximum.
Then we prove that the set X is maximum, if there is no SX-TX-path. Let R be the set of vertices reachable fromSX inGX. Hence R∩TX =∅. Let r1 and r2 be the rank function ofI1 and I2, respectively.
Fig 13.3. from (Korte and Vygen, 2007, p. 326)
First we prove r2(R) =|X∩R| by contradiction:
Ifr2(R)>|X∩R|, there would exist a y ∈R\X such that (X∩R)∪ {y} ∈ I2. Because y∈R and R∩TX =∅we know that y6∈TX={y ∈E\X|X∪ {y} ∈ I2} and therefore a circuitC2(X, y)6∈ I2 exists. Because {y} ∈ I2 and (X∩R)∈ I2, there has to exist a x∈X\R withx∈C2(X, y), even x∈C2(X, y)\ {y} because x 6= y. But then (y, x) ∈ A(2)X = {(y, x)|y ∈ E \X, x ∈ C2(X, y)\ {y}} means there exists an edge in GX which leaves R because x ∈ X\R. This contradicts the definition of R.
Now we prove r1(E\R) = (X\R) by contradiction:
Ifr1(E\R)>|X\R|, there would exist ay∈(E\R)\(X\R) such that (X\R)∪ {y} ∈ I1. Becausey6∈RandSX ⊆Rwe knowy6∈SX={y ∈E\X|X∪{y} ∈ I1} and therefore a circuit C1(X, y) 6∈ I1 exists. Because {y} ∈ I1 and (X\R) ∈ I1, there has to exist ax∈X∩R withx∈C1(X, y), evenx∈C1(X, y)\ {y} because x 6= y. But then (x, y) ∈ A(1)X := {(x, y)|y ∈ E\X, x ∈ C1(X, y)\ {y}} means there exists an edge in GX which leaves R because y 6∈ R. This contradicts the definition of R.
Altogether we have|X|=|X\R|+|X∩R|=r2(R) +r1(E\R). We know that for all X0 ∈ I1 ∩ I2 holds the inequality |X0| ≤ r2(R) +r1(E \R) = |X|9. And thereforeX is maximum.
References
[Du 2008] Du, Ding-zhu: Design and Analysis of Computer Algorithms (Lec- ture 11, Lecture Notes 2008). (2008). – URL http://www.utdallas.edu/
~dxd056000/cs6363/lect11.ppt
9See (Korte and Vygen, 2007, Proposition 13.29).
References [Korte and Vygen 2007] Korte, Bernhard ;Vygen, Jens: Combinatorial Opti- mization: Theory and Algorithms. Fourth Edition. Springer Berlin Heidelberg, 12 2007 (Algorithms and Combinatorics). – ISBN 9783642090929
[Lee 2004] Lee, Jon: A First Course in Combinatorial Optimization. Cambridge University Press, 2004. – ISBN 0521010128
[Oxley 2006] Oxley, James G.: Matroid Theory (Oxford Graduate Texts in Mathematics). Oxford University Press, USA, 8 2006. – ISBN 9780199202508 [Papadimitriou and Steiglitz 1982] Papadimitriou, Christos H. ; Steiglitz,
Kenneth: Combinatorial Optimization: Algorithms and Complexity. Prentice Hall, Inc, 1982. – ISBN 0131524623
