CSE 594: Combinatorial and Graph Algorithms SUNY at Buffalo, Fall 2006

Lecturer: Hung Q. Ngo Last update: November 14, 2006

Analyzing approximation algorithms with the dual-tting method

A greedy algorithm for WEIGHTED SET COVER

One of the best examples of combinatorial approximation algorithms is a greedy algorithm approximating the WEIGHTED SET COVER problem. An instance of the S ET C OVER problem consists of a universe set U of m elements, a family S of n subsets of U , where set S S is weighted with wS . We want to nd a sub-family of S with minimum total weight such that the union of the sub-family is U (i.e. covers U ). Consider the following greedy algorithm. Algorithm 1.1. G REEDY-S ET-C OVER(U, S , w) 1: C , A U 2: while A = do 3: Pick S S with the least cost per un-covered element, i.e. pick S such that 4: AAS 5: C C {S } 6: end while 7: return C

wS |S A|

is minimized.

In this section, we analyze this algorithm combinatorially. Then, a linear programming based analysis will be derived in the next section. Without loss of generality, suppose the algorithm returns a collection {S1 , . . . , Sk } of k sets. Let Xi be the set of newly covered elements of U after the ith step. Let xi = |Xi |, and wi = wSi which is the weight of the ith set picked by the algorithm. Assign a cost c(u) = wi /xi to each element u Xi , for all i k . For any set S S , we rst estimate uS c(u). Let ai = |S Xi |. Then, it is easy to see the following: wS w1 a1 + + ak x1 wS w2 a2 + + ak x2 . . . . . . . . . wk wS . ak xk Hence,
k

c(u) =
u S i=1

wi ai xi

ai
i=1

wS wS H|S | , ai + + ak

where H|S | = 1 + 1/2 + + 1/|S | is the |S |th harmonic number. Since |S | m for all S , we conclude that c(u) Hm wS , S S . (1)
u S

One may ask, what if ai + + ak = 0 for some i. This is not a problem. Since S = , a1 + + ak = wk i 0. If ai + + ak = 0 for some i, then all the terms ai w xi , . . . , ak xk can be ignored. Let T be any optimal solution, then cost(C ) =
u U

c(u)
T T uT

c(u)
T T

H|T | wT Hm cost(T ).

We thus have proved the following theorem. Theorem 1.2. G REEDY-S ET-C OVER has approximation ratio Hm . Exercise 1. In the S ET M ULTICOVER problem, each element u is required to be covered mu times, where mu is a positive integer. Each set can be picked multiple times. The cost of picking S k times is kwS . Devise a greedy algorithm for S ET M ULTICOVER with approximation ratio Hm (and prove that!). Exercise 2. In the M AXIMUM C OVERAGE problem, we are given a universe U , a collection S of subsets of U , and a positive integer k . Each element u in the universe has a non-negative integer weight wu . The problem is to nd k members of S whose union has the maximum total weight. Suppose we solve this problem by greedily pick the best set in each iteration until k sets are picked. (Best set is the set maximizing total weight of uncovered elements.) Prove that this strategy has 1 k approximation ratio 1 1 k . Exercise 3. Consider the WEIGHTED VERTEX wv > 0. Consider the following algorithm
COVER

problem in which each vertex v is weighted with

Algorithm 1.3. LR V ERTEX C OVER(G, w) 1: C = 2: For each v V (G), let c(v ) wv 3: while C is not a vertex cover do 4: Pick an uncovered edge (u, v ), let min{c(u), c(v )} 5: c(u) c(u) ; c(v ) c(v ) 6: Add into C all vertices v having c(v ) = 0. 7: end while 8: return C Prove that this is a 2-approximation algorithm.

Analyzing GREEDY SET COVER with dual-tting

It is natural to nd out how Algorithm 1.1 relates to the integer programming formulation of S ET C OVER. The IP for S ET C OVER is wS xS min
S S

subject to
S u

xS 1,

u U,

(2)

xS {0, 1}, S S . The LP-relaxation is min

S S

wS xS xS 1, u U,
S u

subject to

(3)

xS 0, S S . 2

And, the dual LP is max

u U

yu yu wS , S S ,
u S

subject to

(4)

yu 0, u U. The dual constraints look very much like relation (1), except that we need to divide both sides of (1) by Hm . Thus, for each u U , if we set yu = c(u)/Hm , then y is a dual feasible solution. It follows that cost(C ) =
u U

c(u) = Hm cost(y) Hm OPT.

More general covering problems

The C ONSTRAINED S ET M ULTICOVER problem is a generalization of the S ET C OVER problem in which each elements u U needs to be covered mu times, where mu is a positive integer. The corresponding integer program can be written as min
S S

wS xS xS mu , u U,
S u

subject to

(5)

xS {0, 1}, S S . When relaxing this program, it is no longer possible to remove the upper bounds xS 1 (otherwise an integral optimal solution to the LP may not be an optimal solution to the IP). The LP-relaxation is min
S S

wS xS xS mu , u U,
S u

subject to

(6)

xS 1, S S , xS 0, S S . The dual linear program is now max

u U

mu y u
S S

zS S S , (7)

subject to
u S

yu zS wS ,

yu , zS 0, u U, S S . We will try to devise a greedy algorithm to solve this problem and analyze it using the dual-tting method. Algorithm 3.1. G REEDY-S ET-M ULTICOVER(U, S , w, m) 1: C = ; A U 2: // We call an element u U alive if mu > 0. Initially all of A are alive 3: while A = do S 4: Pick S such that |Sw A| is minimized. 5: C = C {S } 3

mu mu 1 for each u S A 7: Remove from A all elements u with mu = 0 8: end while 9: return C
6:

The next step is to write the cost of C as a multiple of the cost of a feasible solution to (7), namely cost(C ) = ( u mu yu S zS ) for some feasible solution (y, z) of (7). For each element u U and each j [mu ], let c(u, j ) be the cost of covering u for the j th time. If S covers u for the j th time, and wS AS is the set of alive elements before S was picked, then c(u, j ) = |S AS | . If S was chosen before T , then AT AS . Thus, wS wT wT . |S AS | |T AS | |T AT | Consequently, for any u we have c(u, 1) c(u, mu ). The nal cost is
mu

cost(C ) =
uU j =1

c(u, j ).
S S

In order to write this sum as ( sense to try cost(C ) =

u U

mu yu

zS ) (keeping in mind that yu , zS 0), it makes

mu 1

mu c(u, mu )
u U uU j =1 mu

[c(u, mu ) c(u, j )] [c(u, mu ) c(u, j )]

uU j =1

=
u U

mu c(u, mu )

The double sum (after the minus sign) is non-negative, which is good. We need to write it in the form S S zS somehow. Note that, each time u is covered, a term c(u, mu ) c(u, j ) is added into the double sum. For each S C , suppose S covers u S AS the ju,S th time. Then,
mu

[c(u, mu ) c(u, j )] =
uU j =1 S C uS AS

[c(u, mu ) c(u, ju,S )] .

Now, let be a number to be determined. Dene yu = zS = 1 c(u, mu ), u U 1 [c(u, mu ) c(u, ju,S )] S C

uS AS

S /C
u S

For (y, z) to be dual-feasible, we would like to nd so that, for each S S , Consider any S / C . In this case, y u zS =
u S

yu zS wS .

c(u, mu ).
u S

Let u1 , . . . , uk be the elements of S . Without loss of generality, assume that u1 was completely covered before u2 , and so on. Then, right before ui is completely covered, |AS | k (i 1). Hence, c(ui , mui ) wS /(k i + 1). Consequently, y u zS
u S

k i=1

wS Hm wS . ki+1

Now, consider any S C . In this case we have y u zS =

u S

c(u, mu )
u S

[c(u, mu ) c(u, ju,S )]

uS AS

= 1 c(u, mu ) +
uS \AS uS AS

c(u, ju,S )

Let u1 , . . . , uk be elements in S \ AS which were completely covered in that order. Note that 0 k < k . Note also that uS AS c(u, ju,S ) = wS . Similar to the previous reasoning, we get y u zS =
u S

k i=1

wS + wS ki+1

Hm wS .

Hence, (y, z) would be a dual feasible solution if we pick = Hm , which is also an approximation ratio for Algorithm 3.1. Exercise 4. Devise a greedy algorithm for S ET M ULTICOVER with approximation ratio Hm . Analyze your algorithm using the dual-tting method. Exercise 5. In the M ULTISET M ULTICOVER problem, we are given a collection S of multisets of a universe U . For each S S , let M (S, u) be the multiplicity of u in S . Each element u needs to be covered mu times. We can assume M (S, u) mu for all S, u. Devise a greedy algorithm for M ULTISET M ULTICOVER with approximation ratio Hd , where d is the largest multiset size. The size of a multiset is the total multiplicity of its elements. Analyze your algorithm using the dual-tting method. Exercise 6. Consider the integer program min{cT x | Ax b}, where A, b have non-negative integral entries, and x is required to be non-negative and integral also. This is called a covering integer program. Use scaling and rounding to reduce covering integer programs to M ULTISET M ULTICOVER, so that we can use the greedy algorithm for the M ULTISET M ULTICOVER instance to get a greedy algorithm for the C OVERING I NTEGER P ROGRAM instance with approximation ratio O(lg n), where n is the input size of the covering integer program. (Thus, the instance of M ULTISET M ULTICOVER must have size polynomial in n.) Exercise 7. Vaziranis book. Problem 24.12, page 241.

Historical Notes
The greedy approximation algorithm for S ET C OVER is due to Johnson [5], Lov asz [6], and Chv atal [2]. Feige [4] showed that approximating S ET C OVER to an asymptotically better ratio than ln m is NP-hard. The dual-tting analysis for G REEDY S ET C OVER was given by Lov asz [6]. Dobson [3] and Rajagopalan and Vazirani [8] studied approximation algorithms for covering integer programs. The dualtting method has found applications in other places [1, 7].

References
[1] P. C ARMI , T. E RLEBACH , AND Y. O KAMOTO, Greedy edge-disjoint paths in complete graphs, in Graph-theoretic concepts in computer science, vol. 2880 of Lecture Notes in Comput. Sci., Springer, Berlin, 2003, pp. 143155.

 [2] V. C HV ATAL , A greedy heuristic for the set-covering problem, Math. Oper. Res., 4 (1979), pp. 233235. [3] G. D OBSON, Worst-case analysis of greedy heuristics for integer programming with nonnegative data, Math. Oper. Res., 7 (1982), pp. 515531. [4] U. F EIGE, A threshold of ln n for approximating set cover (preliminary version), in Proceedings of the Twenty-eighth Annual ACM Symposium on the Theory of Computing (Philadelphia, PA, 1996), New York, 1996, ACM, pp. 314318. [5] D. S. J OHNSON, Approximation algorithms for combinatorial problems, J. Comput. System Sci., 9 (1974), pp. 256278. Fifth Annual ACM Symposium on the Theory of Computing (Austin, Tex., 1973). , On the ratio of optimal integral and fractional covers, Discrete Math., 13 (1975), pp. 383390. [6] L. L OV ASZ [7] M. M AHDIAN , E. M ARKAKIS , A. S ABERI , AND V. VAZIRANI, A greedy facility location algorithm analyzed using dual tting, in Approximation, randomization, and combinatorial optimization (Berkeley, CA, 2001), vol. 2129 of Lecture Notes in Comput. Sci., Springer, Berlin, 2001, pp. 127137. [8] S. R AJAGOPALAN AND V. V. VAZIRANI, Primal-dual RNC approximation algorithms for set cover and covering integer programs, SIAM J. Comput., 28 (1999), pp. 525540 (electronic). A preliminary version appeared in FOCS93.