18.434 Seminar in Theoretical Computer Science

1 of 5

Tamara Stern 2.9.06

Set Cover Problem (Chapter 2.1, 12)

What is the set cover problem? Idea: “You must select a minimum number [of any size set] of these sets so that the sets you have picked contain all the elements that are contained in any of the sets in the input (wikipedia).” Additionally, you want to minimize the cost of the sets. Input: Ground elements, or Universe U = {u1 , u 2 ,..., u n } Subsets S1 , S 2 ,..., S k ⊆ U Costs c1 , c 2 ,..., c k Goal: Find a set I ⊆ {1,2,..., m} that minimizes

∑c i∈I

i

, such that U S i = U . i∈I

(note: in the un-weighted Set Cover Problem, c j = 1 for all j)

Why is it useful? It was one of Karp’s NP-complete problems, shown to be so in 1972. Other applications: edge covering, vertex cover Interesting example: IBM finds computer viruses (wikipedia) elements- 5000 known viruses sets- 9000 substrings of 20 or more consecutive bytes from viruses, not found in ‘good’ code A set cover of 180 was found. It suffices to search for these 180 substrings to verify the existence of known computer viruses. Another example: Consider General Motors needs to buy a certain amount of varied supplies and there are suppliers that offer various deals for different combinations of materials (Supplier A: 2 tons of steel + 500 tiles for $x; Supplier B: 1 ton of steel + 2000 tiles for $y; etc.). You could use set covering to find the best way to get all the materials while minimizing cost.

18.434 Seminar in Theoretical Computer Science

2 of 5

Tamara Stern 2.9.06

How can we solve it? 1. Greedy Method – or “brute force” method Let C represent the set of elements covered so far Let cost effectiveness, or α , be the average cost per newly covered node

Algorithm 1. C Å 0 2. While C ≠ U do Find the set whose cost effectiveness is smallest, say S Let α =

c( S ) S −C

For each e ∈ S-C, set price(e) = α CÅC ∪S 3. Output picked sets

Example

U X

.

. .. . . . Y . . . . .

Z

c(X) = 6 c(Y) = 15 c(Z) = 7

c( Z ) 7 = =1 Choose Z: α Z = | S −C | 7 c( X ) 6 Choose X: α X = = =2 | S −C | 3 c(Y ) 15 Choose Y: α Y = = = 7.5 | S −C | 2

U X

.

. .2 .2 .2 . .1 1 Y .1 1. 7.5. 1. .7.5

Z

1 1

Total cost = 6 + 15 + 7 = 28 (note: The greedy algorithm is not optimal. An optimal solution would have chosen Y and Z for a cost of 22)

18.434 Seminar in Theoretical Computer Science

3 of 5

Tamara Stern 2.9.06

Theorem:

The greedy algorithm is an H n factor approximation algorithm for the minimum set cover problem, where H n = 1 +

1 1 + ... + ≈ log n . 2 n

Proof: (i) We know

∑ price(e) = cost of the greedy algorithm = c(S ) +c(S 1

e∈U

2

) + ... + c( S m )

because of the nature in which we distribute costs of elements. (ii) We will show price(ek ) ≤

OPT , where ek is the kth element covered. n − k +1

Say the optimal sets are O1 , O2 ,..., O p . a

= c(O1 ) +c(O2 ) + ... + c(O p ) .

So, OPT

Now, assume the greedy algorithm has covered the elements in C so far. Then we know the uncovered elements, or U − C , are at most the intersection of all of the optimal sets intersected with the uncovered elements:

U −C

b

≤

O1 ∩ (U − C ) + O2 ∩ (U − C ) + ... + O p ∩ (U − C )

In the greedy algorithm, we select a set with cost effectiveness α , where c

α ≤

c(Oi ) Oi ∩ (U − C )

, i = 1... p . We know this because the greedy algorithm

will always choose the set with the smallest cost effectiveness, which will either be smaller than or equal to a set that the optimal algorithm chooses. c

Algebra:

c(Oi ) ≥ α ⋅ Oi ∩ (U − C ) a

=

OPT

c

∑ c(Oi ) ≥ α ⋅ ∑ Oi ∩ (U − C ) i

α ≤

b

≥ α ⋅U −C

i

OPT , |U − C |

Therefore, the price of the k-th element is:

α ≤

OPT n − (k − 1)

=

OPT n − k +1

Putting (i) and (ii) together, we get the total cost of the set cover: n

∑ price(ek ) ≤ k =1

n

OPT

∑ n − k +1 k =1

1⎞ ⎛ 1 = OPT ⋅ ⎜1 + + ... + ⎟ = OPT ⋅ H n n⎠ ⎝ 2

18.434 Seminar in Theoretical Computer Science

4 of 5

Tamara Stern 2.9.06

2. Linear Programming (Chapter 12) “Linear programming is the problem of optimizing (minimizing or maximizing) a linear function subject to linear inequality constraints (Vazirani 94).” For example, n

min imize

∑c j =1

j

n

subject to

∑a j =1

ij

xj x j ≥ bi

i = 1,..., m

xj ≥ 0

j = 1,..., n

There exist P-time algorithms for linear programming problems.

Example Problem

U = {1,2,3,4,5,6}, S1 = {1,2}, S 2 = {1,3}, S 3 = {2,3}, S 4 = {2,4,6}, S 5 = {3,5,6}, S 6 = {4,5,6}, S 7 = {4,5}

⎧1 if select S j

- variable x j for set S j = ⎨

⎩0 if not

k

- minimize cost: Min

∑c j =1

j

x j = OPT

- subject to:

x1 x1

+

x2 x2

+ +

x3 x3

+

x4 +

x5

x4 x4 possible solution: x1 = x 2 = x3 =

1 , 2

+

x5 x5

+ + +

x6 x6 x6

+ +

x7 x7

x 4 = x5 = x 7 = 0, x 6 = 1 Æ

≥ ≥ ≥ ≥ ≥ ≥

∑x

1 1 1 1 1 1 i

= 2.5

This would be a feasible solution – however, we need x j ∈ {0,1}, which is not a valid constraint for linear programming. Instead, we will use x j ≥ 0 , which will give us all correct solutions, plus more (solutions where x j is a fraction or greater than 1).

18.434 Seminar in Theoretical Computer Science

5 of 5

Tamara Stern 2.9.06

Given a primal linear program, we can create a dual linear program. Let’s revisit linear program: n

min imize

∑c j =1

j

n

subject to

∑a j =1

ij

xj x j ≥ bi

i = 1,..., m

xj ≥ 0

j = 1,..., n

Introducing variables yi for the ith inequality, we get the dual program: m

max imize

∑b y i =1

i

m

subject to

∑a i =1

ij

i

y j ≤ ci

j = 1,..., n

yi ≥ 0

i = 1,..., m

LP-duality Theorem (Vazirani 95): The primal program has finite optimum iff its dual has finite

(

)

(

)

optimum. Moreover, if x * = x1* ,..., x n* and y * = y1* ,..., y m* are optimal solutions for the primal and dual programs, respectively, then

n

m

j =1

i =1

∑ c j x *j = ∑ bi yi* .

By solving the dual problem, for any feasible solution for y1 → y 6 , you can guarantee that the optimal result is that solution or greater.

References: http://en.wikipedia.org/wiki/Set_cover_problem. Vazirani, Vijay. Approximation Algorithms. Chapters 2.1 & 12.

http://www.orie.cornell.edu/~dpw/cornell.ps. p7-18.