Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Vertex Cover Linear Progamming and Approximation Algorithms Joshua Wetzel Department of Computer Science Rutgers University–Camden
[email protected]
March 24, 2009
Joshua Wetzel
Vertex Cover
1 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
What is Linear Programming?
Optimize a linear objective fn subject to linear constraints. Examples: min
7x1 + x2 + 3x3
s.t.
max
6x1 + 2x2 − 4x3
s.t. x1 − x2 + 3x3 ≥ 10
5x1 + x2 − 2x3 ≤ 14
5x1 + 2x2 − x3 ≥ 6
2x1 − 2x2 + 4x3 ≤ 20
x1 , x2 , x3 ≥ 0
x1 , x2 , x3 ≥ 0
Joshua Wetzel
Vertex Cover
2 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Feasible Solutions: Upper Bounding OPT
min
Linear Program:
Feasible Solutions: x1 = 2, x2 = 1, x3 = 4
7x1 + x2 + 3x3
7(2) + 1(1) + 5(4) = 35
s.t. x1 − x2 + 3x3 ≥ 10 5x1 + 2x2 − x3 ≥ 6 x1 , x2 , x3 ≥ 0
x1 = 58 , x2 = 12 , x3 = 3 7( 58 ) + 1( 12 ) + 5(3) = 26.7
All constraints are satisfied. OPT ≤ 35, OPT ≤ 26.7 Joshua Wetzel
Vertex Cover
3 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Finding a Lower Bound on OPT Linear Program: min
7x1 + x2 + 3x3
s.t. x1 − x2 + 3x3 ≥ 10 5x1 + 2x2 − x3 ≥ 6 x1 , x2 , x3 ≥ 0
OPT ≥ ? Joshua Wetzel
Vertex Cover
4 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Lower Bounding OPT: Adding the Constraints
min
7x1 + x2 + 3x3
s.t. x1 − x2 + 3x3 ≥ 10 +
5x1 + 2x2 − x3 ≥ 6
6x1 + x2 + 2x3 ≥ 16 OPT ≥ 16
Joshua Wetzel
Vertex Cover
5 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Lower Bounding OPT min 7x1 + x2 + 3x3 s.t. +
x1 − x2 + 3x3 ≥ 10
(y1 )
5x1 + 2x2 − x3 ≥ 6
(y2 )
x1 ( y1 + 5y2 ) + x2 ( −y1 + 2y2 ) + x3 ( 3y1 − y2 ) ≥ 10y1 + 6y2 10y1 + 6y2 is a lower bound on OPT if: y1 + 5y2 ≤ 7 −y1 + 2y2 ≤ 1 3y1 − y2 ≤ 3 Joshua Wetzel
Vertex Cover
6 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
LP to Lower Bound OPT
max 10y1 + 6y2 s.t. y1 + 5y2 ≤ 7 −y1 + 2y2 ≤ 1 3y1 − y2 ≤ 3 y1 , y2 ≥ 0
Joshua Wetzel
Vertex Cover
7 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Primal LP and Dual LP Dual LP: Primal LP:
max 10y1 + 6y2
min 7x1 + x2 + 3x3 s.t.
s.t.
y1 + 5y2 ≤ 7
x1 − x2 + 3x3 ≥ 10
−y1 + 2y2 ≤ 1
5x1 + 2x2 − x3 ≥ 6
3y1 − y2 ≤ 3
x1 , x2 , x3 ≥ 0
y1 , y2 ≥ 0
Every primal LP has a corresponding dual LP. If the primal is a min problem, the dual is a max problem. There is a dual constraint corresponding to each primal variable. Joshua Wetzel
Vertex Cover
8 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
LP Duality Theorems DualFeasible ≤ DualOPT = PrimalOPT ≤ PrimalFeasible
The Dual LP: The Primal LP: min 7x1 + x2 + 3x3 s.t.
max 10y1 + 6y2 s.t.
y1 + 5y2 ≤ 7
x1 − x2 + 3x3 ≥ 10
−y1 + 2y2 ≤ 1
5x1 + 2x2 − x3 ≥ 6
3y1 − y2 ≤ 3
x1 , x2 , x3 ≥ 0
y1 , y2 ≥ 0
Joshua Wetzel
Vertex Cover
9 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Vertex Cover
Input: Given G = (V , E) Non-negative weights on vertices
Objective: Find a least-weight collection of vertices such that each edge in G in incident on at least one vertex in the collection.
Joshua Wetzel
Vertex Cover
10 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Vertex Cover: Example
24
50
24
50 10
15
10
15
6
30
6
30
18
12
18
12
COST = 97
COST = 108
Joshua Wetzel
Vertex Cover
11 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Approximation Algorithms
NP-hard problems. No optimal poly-time algorithms are known
β-approximation alg., A, for a minimization problem P poly-time algorithm. for every instance I of P, A produces solution of cost at most β · OPT (I) OPT (I)?
Joshua Wetzel
Vertex Cover
12 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Approximation Algorithms compute a lower bound on OPT . compare cost of our solution with the lower bound.
lower bound
OPT
upper bound
β
Joshua Wetzel
Vertex Cover
13 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Vertex Cover
Input: Given G = (V , E) Non-negative weights on vertices
Objective: Find a least-weight collection of vertices such that each edge in G in incident on at least one vertex in the collection
Joshua Wetzel
Vertex Cover
14 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Unweighted Vertex Cover: Algorithm
Find a maximal matching in G Include in our cover both vertices incident on each edge of the matching
Joshua Wetzel
Vertex Cover
15 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Unweighted Vertex Cover: Example
1
1
1
1
1
1 1
1
Joshua Wetzel
Vertex Cover
16 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Unweighted Vertex Cover: Example
1
1
1
1
1
1 1
1
Joshua Wetzel
Vertex Cover
17 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Unweighted Vertex Cover: Example
1
1
1
1
1
1 1
1
Joshua Wetzel
Vertex Cover
18 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Unweighted Vertex Cover: Example 1
1
1
1
1
1 1
1
Cost = 6 Joshua Wetzel
Vertex Cover
19 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Analysis: Feasibility 1
1
1
1
1
1 1
1
Every black edge shares a vertex with a green edge Joshua Wetzel
Vertex Cover
20 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Analysis: Approximation Guarantee
1
1
1
1
OPT has to choose at least one endpoint from each green edge. We choose both endpoints for each green edge.
1
1 1
Hence:
1
Our Cost ≤ 2OPT
Joshua Wetzel
Vertex Cover
21 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Unweighted Vertex Cover: Tight Example
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
COSTAlg = 6
OPT = 3
Joshua Wetzel
Vertex Cover
22 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Bad Example
200
1
200
200
1
200
CostAlg = 201
OPT = 1
Joshua Wetzel
Vertex Cover
23 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Vertex Cover: IP Formulation
xv ← 1 if v is in our cover, 0 otherwise X min wv xv v ∈V
s.t. xa + xb ≥ 1, ∀ e = (a, b) xv ∈ {0, 1}, ∀ v ∈ V
Joshua Wetzel
Vertex Cover
24 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
LP Relaxation Integer programs have been shown to be NP-hard Relax the integrality constraints xv ∈ {0, 1}, ∀ v ∈ V min
X
wv xv
v ∈V
s.t. xa + xb ≥ 1, ∀ e = (a, b) xv ≥ 0, ∀ v ∈ V
Joshua Wetzel
Vertex Cover
25 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Why Do This?
LP can be solved in polynomial time Every solution to the IP is also a solution to the LP Hence: OPTLP ≤ OPTIP We can use OPTLP as a lower bound on OPTIP
Joshua Wetzel
Vertex Cover
26 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
LP Solution: Example xa + xb ≥ 1, ∀ e = (a, b)
xa + xb ≥ 1, ∀ e = (a, b)
xv ≥ 0, ∀ v ∈ V
xv ∈ {0, 1}, ∀ v ∈ V
1/2
1
1/2
1/2
OPTLP = 1.5 Joshua Wetzel
0
1
OPTIP = 2 Vertex Cover
27 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
LP-Rounding Algorithm
x ∗ ← optimal LP soln. xˆv ← 1 if xv∗ ≥ 12 , otherwise xˆv ← 0 Include v in our cover iff xˆv = 1
Joshua Wetzel
Vertex Cover
28 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Analysis: Feasibility
min
X
wv xv
v ∈V
s.t. xa + xb ≥ 1, ∀ e = (a, b) xv ≥ 0, ∀ v ∈ V 1 1 or xb∗ ≥ 2 2 xˆa = 1 or xˆb = 1
xa∗ ≥
Joshua Wetzel
Vertex Cover
29 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Analysis: Approximation Guarantee
Our Cost =
X
wv
=
X
wv xˆv
v ∈V
v ∈C
≤
X
wv (2xv∗ )
v ∈V
= 2
X
wv xv∗
v ∈V
= 2OPTLP ≤ 2OPTIP
Joshua Wetzel
Vertex Cover
30 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
LP-Rounding: Tight Example
1
1/2
1/2
1/2
OPTLP = 1.5
Joshua Wetzel
1
1
Our Cost = 3
Vertex Cover
31 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Primal-Dual Method
PrimalOPT ≤ OPTIP Solving the LP is expensive. DualFeasible ≤ DualOPT = PrimalOPT ≤ PrimalFeasible Better Alternative: Construct the dual LP Construct an algorithm that manually tightens dual constraints to obtain a ’maximal’ dual solution
Joshua Wetzel
Vertex Cover
32 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Constructing the Dual LP
Primal LP: min
X
wv xv
v ∈V
s.t. xa + xb ≥ 1, ∀ e = (a, b)
(ye )
xv ≥ 0, ∀ v ∈ V
Joshua Wetzel
Vertex Cover
33 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Primal LP and Dual LP
Dual LP:
Primal LP: min
X
max
wv xv
X
ye
e∈E
v ∈V
s.t.
s.t.
xa + xb ≥ 1, ∀ e = (a, b) xv ≥ 0, ∀ v ∈ V
Joshua Wetzel
X
ye ≤ wv , ∀ v ∈ V
e : e hits v
ye ≥ 0, ∀ e ∈ E
Vertex Cover
34 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Bar-Yehuda and Even Algorithm
Inititally all edges are uncovered. While ∃ an uncovered edge in G: Choose an arbitrary edge, e Raise the value of ye for that edge P until one of its incident vertices , v , becomes full (i.e y e = wv ) e:e hits v
C ← C ∪ {v } Any edge that touches v is considered to be covered
Return C as our vertex cover
Joshua Wetzel
Vertex Cover
35 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Bar-Yehuda and Even Algorithm: Example
0 24
50
10 0
0
0
0 15
6 0 0
30
18
0
0
0 12
Joshua Wetzel
Vertex Cover
36 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Bar-Yehuda and Even Algorithm: Example X
ye ≤ wv
e:e hits v 0 24
50
10 0
0
0
0 15
6 0 0
30
18
0
0
0 12
Arbitrarily choose e and raise ye until a vertex is full Joshua Wetzel
Vertex Cover
37 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Bar-Yehuda and Even Algorithm: Example X
ye ≤ wv
e:e hits v 0 24
50
10 0
6
0
0 15
6 0 0
30
18
0
0
0 12
Joshua Wetzel
Vertex Cover
38 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Bar-Yehuda and Even Algorithm: Example X
ye ≤ wv
e:e hits v 0 24
50
10 0
6
0
0 15
6 0 0
30
18
0
0
0 12
Joshua Wetzel
Vertex Cover
39 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Bar-Yehuda and Even Algorithm: Example X
ye ≤ wv
e:e hits v 0 24
50
10 15
6
0
0 15
6 0 0
30
18
0
0
0 12
Joshua Wetzel
Vertex Cover
40 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Bar-Yehuda and Even Algorithm: Example X
ye ≤ wv
e:e hits v 0 24
50
10 15
6
0
0 15
6 0 0
30
18
0
0
0 12
Joshua Wetzel
Vertex Cover
41 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Bar-Yehuda and Even Algorithm: Example X
ye ≤ wv
e:e hits v 18 24
50
10 15
6
0
0 15
6 0 0
30
18
0
0
0 12
Joshua Wetzel
Vertex Cover
42 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Bar-Yehuda and Even Algorithm: Example X
ye ≤ wv
e:e hits v 18 24
50
10 15
6
0
0 15
6 0 0
30
18
0
0
0 12
Joshua Wetzel
Vertex Cover
43 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Bar-Yehuda and Even Algorithm: Example X
ye ≤ wv
e:e hits v 18 24
50
10 15
6
0
0 15
6 0 0
30
18
12
0
0 12
Joshua Wetzel
Vertex Cover
44 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Bar-Yehuda and Even Algorithm: Example X
ye ≤ wv
e:e hits v 18 24
50
10 15
6
0
0 15
6 0 0
30
18
12
0
0 12
Joshua Wetzel
Vertex Cover
45 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Bar-Yehuda and Even Algorithm: Example X
ye ≤ wv
e:e hits v 18 24
50
10 15
6
0
17 15
6 0 0
30
18
12
0
0 12
Joshua Wetzel
Vertex Cover
46 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Bar-Yehuda and Even Algorithm: Example 18 24
50
10 15
6
0
17 15
6 0 0
30
18
12
0
0 12
Cost = 107 Joshua Wetzel
Vertex Cover
47 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Bar-Yehuda and Even Algorithm: Analysis 18 24
50
10 15
6
Dual Obj. Fn: X max ye
0
e
17 15
6 0
0
30
18
12
0
0
Our Cost = wt(red vertices) X ≤ 2 ye
12
X
e hits red
≤ 2
X
ye
e
ye ≤ wv
= 2DFS
e:e hits v
≤ 2OPT Joshua Wetzel
Vertex Cover
48 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Bar-Yehuda and Even Algorithm: Tight Example 1
1
1
1
1
1 1
1
1
6
1
1
6 1
1
1
1
COSTOPT = 6
Joshua Wetzel
1 1
1
COSTBar-Yehuda = 12
Vertex Cover
49 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Integrality Gap 1
1/2
1/2
1/2
0
1
OPTLP = 1.5
OPTIP = 2
For a complete graph of n vertices OPTLP = n/2 OPTIP = n − 1 lim OPTIP n→∞ OPTLP
n−1 n→∞ (n/2)
= lim
=2
Joshua Wetzel
Vertex Cover
50 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Reference
R. Bar-Yehuda and S. Even. A linear time approximation algorithm for the weighted vertex cover problem. J. of Algorithms 2:198-203, 1981.
Joshua Wetzel
Vertex Cover
51 / 52
Linear Programming Theory Vertex Cover LP-Rounding Primal-Dual
Thank You.
Joshua Wetzel
Vertex Cover
52 / 52