Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community for help and clear up your study doubts
Discover the best universities in your country according to Docsity users
Free resources
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
That is, decision prob- lems for which “yes” instances have a proof that can be verified in polynomial time. • A: usually denotes the algorithm ...
Typology: Lecture notes
1 / 214
This page cannot be seen from the preview
Don't miss anything!
Computer Science, ETH Z¨urich
These notes will be updated regularly. Please read critically; there are typos throughout, but there might also be mistakes. Feedback and comments would be greatly appreciated and should be emailed to ghaffari@inf.ethz.ch.
Last update: December 25, 2020
Pr[X] ≥ 1 −
poly(n)
say, Pr[X] ≥ 1 − (^) n^1 c for some constant c ≥ 2.
Useful distributions
Bernoulli Coin flip w.p. p. Useful for indicators
Pr[X = 1] = p E[X] = p Var(X) = p(1 − p)
Binomial Number of successes out of n trials, each succeeding w.p. p; Sample with replacement out of n items, p of which are successes
Pr[X = k] =
n k
pk(1 − p)n−k
E[X] = np Var(X) = np(1 − p) ≤ np
Geometric Number of Bernoulli trials until one success
Pr[X = k] = (1 − p)n−^1 p
E[X] =
p
Var(X) =
1 − p p^2
Hypergeometric Number of successes in n draws without replacement, from a population of N items in which K are successful:
Pr[X = k] =
k
n−k
n
E[X] = n ·
Var(X) = n ·
N − n N − 1
Exponential Parameter: λ; Written as X ∼ Exp(λ)
Pr[X = x] =
λe−λx^ if x ≥ 0 0 if x < 0
E[X] =
λ Var(X) =
λ^2
Theorem (Markov’s inequality). If X is a nonnegative random variable and a > 0 , then
Pr[X ≥ a] ≤
a
Theorem (Chebyshev’s inequality). If X is a random variable (with finite expected value μ and non-zero variance σ^2 ), then for any k > 0 ,
Pr[|X − μ| ≥ kσ] ≤
k^2
Theorem (Bernoulli’s inequality). For every integer r ≥ 0 and every real number x ≥ − 1 ,
(1 + x)r^ ≥ 1 + rx
Theorem (Chernoff bound). For independent Bernoulli variables X 1 ,... , Xn, let X =
∑n i=1 Xi. Then,
Pr[X ≥ (1 + ) · E(X)] ≤ exp(
) for 0 <
Pr[X ≤ (1 − ) · E(X)] ≤ exp(
) for 0 < < 1
By union bound, for 0 < < 1 , we have
Pr[|X − E(X)| ≥ · E(X)] ≤ 2 exp(
Remark 1 There is actually a tighter form of Chernoff bounds:
∀ > 0 , Pr[X ≥ (1 + )E(X)] ≤ (
e (1 + )1+^
Remark 2 We usually apply Chernoff bound to show that the probability of bad approximation is low by picking parameters such that 2 exp(−
(^2) E(X) 3 )^ ≤ δ, then negate to get Pr[|X − E(X)| ≤ · E(X)] ≥ 1 − δ.
Theorem (Probabilistic Method). Let (Ω, A, P ) be a probability space,
Pr[ω] > 0 ⇐⇒ ∃ω ∈ Ω
Combinatorics taking k elements out of n:
(n k
(n+k− 1 k
Example
S 1
e 1
e 2
e 3
e 4
e 5
Suppose there are 5 elements e 1 , e 2 , e 3 , e 4 , e 5 , 4 subsets S 1 , S 2 , S 3 , S 4 , and the cost function is defined as c(Si) = i^2. Even though S 3 ∪ S 4 covers all vertices, this costs c({S 3 , S 4 }) = c(S 3 ) + c(S 4 ) = 9 + 16 = 25. One can verify that the minimum set cover is S∗^ = {S 1 , S 2 , S 3 } with a cost of c(S∗) = 14. Notice that we want a minimum cover with respect to c and not the number of subsets chosen from S (unless c is uniform cost).
Since finding the minimum set cover is N P-complete, we are interested in al- gorithms that give a good approximation for the optimum. [Joh74] describes a greedy algorithm GreedySetCover and proved that it gives an Hn- approximation^2. The intuition is as follows: Spread the cost c(Si) amongst the vertices that are newly covered by Si. Denoting the price-per-item by ppi(Si), we greedily select the set that has the lowest ppi at each step unitl we have found a set cover.
Algorithm 1 GreedySetCover(U, S, c)
T ← ∅. Selected subset of S C ← ∅. Covered vertices while C 6 = U do Si ← arg minSi∈S\T (^) |cS(iS\iC)|. Pick set with lowest price-per-item T ← T ∪ {Si}. Add Si to selection C ← C ∪ Si. Update covered vertices end while return T
(^2) Hn = ∑n i= 1 i = ln(n) +^ γ^ ≤^ ln(n) + 0.^6 ∈ O(log(n)), where^ γ^ is the Euler-Mascheroni constant. See https://en.wikipedia.org/wiki/Euler-Mascheroni_constant.
Consider a run of GreedySetCover on the earlier example. In the first iteration, ppi(S 1 ) = 1/3, ppi(S 2 ) = 4, ppi(S 3 ) = 9/2, ppi(S 4 ) = 16/3. So, S 1 is chosen. In the second iteration, ppi(S 2 ) = 4, ppi(S 3 ) = 9, ppi(S 4 ) = 16. So, S 2 was chosen. In the third iteration, ppi(S 3 ) = 9, ppi(S 4 ) = ∞. So, S 3 was chosen. Since all vertices are now covered, the algorithm terminates (coincidentally to the minimum set cover). Notice that ppi for the unchosen sets change according to which vertices remain uncovered. Furthermore, once one can simply ignore S 4 when it no longer covers any uncovered vertices.
Theorem 1.3. GreedySetCover is an Hn-approximation algorithm.
Proof. By construction, GreedySetCover terminates with a valid set cover T. It remains to show that c(T ) ≤ Hn · c(OP T ) for any minimum set cover OP T. Upon relabelling, let e 1 ,... , en be the elements in the order they are covered by GreedySetCover. Define price(ei) as the price- per-item associated with ei at the time ei was purchased during the run of the algorithm. Consider the moment in the algorithm where elements Ck− 1 = {e 1 ,... , ek− 1 } are already covered by some sets Tk ⊂ T. Tk covers no elements in {ek,... , en}. Since there is a cover^3 of cost at most c(OP T ) for the remaining n − k + 1 elements, there must be an element e∗^ ∈ {ek,... , en}
whose price price(e∗) is at most c n(−OP Tk+1^ ).
S
not in OP T
OP T
OP Tk
e 1
ek− 1
ek ek+
en
U
We formalize this intuition with the argument below. Since OP T is a set cover, there exists a subset OP Tk ⊆ OP T that covers ek... en. Sup-
(^3) OP T is a valid cover (though probably not minimum) for the remaining elements.
Tight bound example for GreedySetCover Consider n = 2 · (2k^ − 1) elements, for some k ∈ N \ { 0 }. Partition the elements into groups of size 2 · 20 , 2 · 21 , 2 · 22 ,... , 2 · 2 k−^1. Let S = {S 1 ,... , Sk, Sk+1, Sk+2}. For 1 ≤ i ≤ k, let Si cover the group of size 2 · 2 i−^1 = 2i. Let Sk+1 and Sk+2 cover half of each group (i.e. 2k^ − 1 elements each) such that Sk+1 ∩ Sk+2 = ∅.
S 1 S 2 S 3 Sk
Sk+
Sk+
elts
elts
elements
2 · 2 k−^1 elements
Suppose c(Si) = 1, ∀i ∈ { 1 ,... , k + 2}. The greedy algorithm will pick Sk, then Sk− 1 ,... , and finally S 1. This is because 2 · 2 k−^1 > n/2 and 2 · 2 i^ > (n −
∑k− 1 j=i+1 2 ·^2
j (^) )/2, for 1 ≤ i ≤ k − 1. This greedy set cover costs
k = O(log(n)). Meanwhile, the minimum set cover is S∗^ = {Sk+1, Sk+2} with a cost of 2. A series of works by Lund and Yannakakis [LY94], Feige [Fei98], and Dinur [DS14, Corollary 1.5] showed that it is N P-hard to always approximate set cover to within (1 − ) · ln |U|, for any constant > 0.
Theorem 1.4 ([DS14, Corollary 1.5]). It is N P-hard to always approximate set cover to within (1 − ) · ln |U|, for any constant > 0.
Proof. See [DS14, Corollary 1.5]
In this section, we show that one may improve the approximation factor from Hn if we have further assumptions on the set cover instance. View- ing a set cover instance as a bipartite graph between sets and elements, let ∆ = maxi∈{ 1 ,...,m} degree(Si) and f = maxi∈{ 1 ,...,n} degree(ei) represent the maximum degree of the sets and elements respectively. Consider the follow- ing two special cases of set cover instances:
Special case: Small ∆
Theorem 1.5. GreedySetCover is a H∆-approximation algorithm.
Proof. Suppose OP T = {O 1 ,... , Op}, where Oi ∈ S ∀i ∈ [p]. Consider a set Oi = {ei, 1 ,... , ei,d} with degree(Oi) = d ≤ ∆. Without loss of generality, suppose that the greedy algorithm covers ei, 1 , then ei, 2 , and so on. For
1 ≤ k ≤ d, when ei,k is covered, price(ei,k) ≤ (^) dc−(Ok+1i) (It is an equality if the greedy algorithm also chose Oi to first cover ei,k,... , ei,d). Hence, the greedy cost of covering elements in Oi (i.e. ei, 1 ,... , ei,d) is at most
∑^ d
k=
c(Oi) d − k + 1
= c(Oi) ·
∑^ d
k=
k
= c(Oi) · Hd ≤ c(Oi) · H∆
Summing over all p sets to cover all n elements, we have c(T ) ≤ H∆ ·c(OP T ).
Remark We apply the same greedy algorithm for small ∆ but analyzed in a more localized manner. Crucially, in this analysis, we always work with the exact degree d and only use the fact d ≤ ∆ after summation. Observe that ∆ ≤ n and the approximation factor equals that of Theorem 1.3 when ∆ = n.
Special case: Small f
We first look at the case when f = 2, show that it is related to another graph problem, then generalize the approach for general f.
Vertex cover as a special case of set cover
Definition 1.6 (Minimum vertex cover problem). Given a graph G = (V, E), find a subset S ⊆ V such that:
(i) S is a vertex cover: ∀e = {u, v} ∈ E, u ∈ S or v ∈ S
(ii) |S|, the size of S, is minimized We next argue that each instance of minimum vertex cover can be seen as an instance of minimum set cover problem with f = 2, and (more importantly for our approximation algorithm) any instance of minimum set cover problem with f = 2 can be reduced to an instance of minimum vertex cover. When f = 2 and c(Si) = 1, ∀Si ∈ S, the minimum vertex cover can be seen as an instance of minimum set cover. Given an instance I of minimum vertex cover I = 〈G = (V, E)〉 we build an instance I∗^ = 〈U∗, S∗〉 of minimum set cover as follows:
