Approximation

1
Approximation

• We have now seen that, regardless of the
  herculean efforts made up to now, a VERY large
  number of problems belong to the NPC class we
  have no efficient ( polynomial-time) algorithms
  for producing a (n optimal) solution and,
  furthermore, no realistic hope of finding any
  such algorithms.
• What do we do?
• Try to find a solution that is near enough
  being optimal to be usable, at a computational
  cost that is no worse than low polynomial, at
  least on average.
• How? --- Be clever

2
Approximation

• Def. we say that an algorithm for a problem has
  an approximation ratio of r(n) if, for any input
  of size n, the cost C of the solution produced by
  the algorithm is with a factor of r(n) of the
  cost C of an optimal solution max(C/C, C/C)
  r(n).
• For maximization problems we will have 0 C
  C, while a minimization algorithm will have 0
  C C.
• What you would, ideally, look for, is classes of
  algorithms where you can trade the quality of the
  approximation for computation time some problems
  admit such classes, while we dont have anything
  comparable for some other problems

3
Approximation

• Def. an algorithm that achieves (max(C/C, C/C)
  r(n)) an approximation ratio of r(n) is called
  a r(n)-approximation algorithm.
• Note a 1-approximation algorithm produces an
  optimal solution.
• Def. an approximation scheme for an optimization
  problem is an approximation algorithm that takes
  two inputs an instance of the problem and a
  positive number e, such that the scheme is a (1
  e)-approximation algorithm.

4
Approximation

• Def. an approximation scheme is a
  polynomial-time approximation scheme if, for any
  fixed e gt 0, the scheme runs in time polynomial
  in the size n of the input instance.
• Def. an approximation scheme is a fully
  polynomial-time approximation scheme if it is an
  approximation scheme and its running time is
  polynomial in both 1/e and the size n of the
  input instance (e.g., O((1/e) j nk) with j, k,
  positive integers).

5
Approximation

• The Vertex-Cover Problem.
• Def. a vertex cover of an undirected graph G
  (V, E) is a subset V Í V such that if (u, v) Î
  E, then either u Î V or v Î V or both. The
  size of a vertex cover is the number of vertices
  in it.
• Def. the vertex-cover problem is to find a
  vertex cover of minimum size - an optimal vertex
  cover.

6
Approximation

• The Algorithm. We will show that this fairly
  simple-minded algorithm will produce, in
  polynomial time, a vertex cover which is a
  2-approximation (so no worse than twice as large
  as an optimal one).

7
Approximation

• This shows the execution of the algorithm on an
  example graph. Start, arbitrarily, with (b, c).
  Removing all edges incident to b and c leaves
  graph (b). Next, pick (e, f), and remove edges
  incident to e and f. We end up with graph (c).
  (d, g) is the last remaining edge, add d and g to
  the cover
• V b, c, d, e, f, g.
• Optimal cover b, d, e.

8
Approximation

• Theorem 35.1. APPROX-VERTEX-COVER is a
  polynomial-time 2-approximation algorithm.
• Proof.
• The algorithm is polynomial-time. This is not
  hard the loop picks an edge (u, v) repeatedly
  from E, adding its endpoints to C, and deletes
  all the edges covered by either u or v. Time
  O(V E) if we use adjacency list
  representation.
• The approximation ratio. Let A denote the set of
  edges that were picked in line 4 of the
  algorithm. In order to cover all edges in E, an
  optimal cover C must include at least one
  endpoint of each edge in E. By construction of A,
  no two edges in A share an endpoint.

9
Approximation

• Thus no two edges in A are covered by the same
  vertex in C, and we must have the lower bound
  C A. Each execution of line 4 picks an
  edge neither of whose two vertices is as yet in
  C C 2A. Combining the two expressions
• C 2A 2C.
• QED.

10
Approximation

• The Traveling Salesman Problem. We are given a
  complete undirected graph G (V, E), and a
  non-negative integer-valued cost function c V
  V Z. We must find a minimum cost tour. Let c
  (A) denote the total cost of the edges in a
  subset A Í E.
• Def. a function c satisfies the triangle
  inequality if, for each triple of vertices u, v,
  w Î V,
• c(u, w) c(u, v) c(v, w).
• Meaning a direct path is always no longer than
  an indirect one
• We will now assume that the cost function
  satisfies the triangle inequality.

11
Approximation

• Before going on, we should prove that TSP remains
  in NPC even with the restriction that the cost
  function satisfies the triangle inequality
  (problem 35.2-2).
• TSP with triangle inequality.
• Approx-TSP-Tour(G, c)
• select r Î VG as a root
• compute an MST T for G starting from r - via
  MST-Prim(G, c, r)
• let L be the list of vertices visited in a
  preorder walk of T
• return the hamiltonian cycle H that visits the
  vertices in order L.

12
Approximation

• The TSP using Approx-TSP-Tour(G, c). Let a be
  the root. Let c be the ordinary euclidean
  distance. (d) gives the approximate tour, (e)
  gives an optimal one.

13
Approximation

• Theorem 35.2. Approx-TSP-Tour is a
  polynomial-time 2-approximation algorithm for the
  TSP with triangle inequality.
• Proof.
• The algorithm is polynomial-time. The dominant
  operation is the computation of the MST.
  MST-Prim can be shown to be Q(V2).
• The approximation ratio. Let H be an optimal
  tour, T a MST. Since removing any edge from a
  tour gives a spanning tree, we must have c(T)
  c(H). The listing through the walk will
  actually give a list of vertices with
  repetitions a vertex is added to the list when
  first encountered and also whenever
  re-encountered after

14
Approximation

• finishing a visit to a non-empty sub-tree note
  that each edge of T is traversed exactly twice.
  If we denote the full walk by W, we have c(W) 2
  c(T). Combining, we have
• c(W) 2c(T) 2c(H),
• and this gives that the cost of W is within the
  correct factor of the cost of the optimal tour.
  The only problem is that W is not a tour, since
  it may visit some vertices more than once. The
  triangle inequality assumption allows us to
  remove vertices from W without increasing the
  cost of the path say that vertex w1 is followed
  by a vertex w2 that has been already visited. Say
  that the next vertex in W is w3. We can remove
  the edges (w1, w2) and (w2, w3), replacing them
  by (w1, w3), without

15
Approximation

• increasing the length of the path. If w3 has
  already been visited, continue along the list W
  until you find an unvisited vertex, or the start
  vertex as the last vertex in W. Add the
  appropriate edge. After we are done, we have a
  hamiltonian cycle H. By construction, and the
  triangle inequality condition, c(H) c(W)
  2c(T) 2c(H).
• QED.
• What happens if we drop the triangle inequality
  assumption? Do we need this extra constraint on
  the problem, reasonable as it may seem?

16
Approximation

• Theorem 35.3. If P ? NP, then for any constant r
  1, there is no polynomial-time approximation
  algorithm with approximation ratio r for the
  general traveling-salesman problem.
• Proof. By contradiction assume that for some r
  1 there is a polynomial-time approximation
  algorithm A with approximation ratio r. w.l.g.,
  assume r is an integer. We show how to use A to
  solve the hamiltonian-cycle problem in polynomial
  time - which would then imply that P NP.
• Let G (V, E) be an instance of the
  hamiltonian-cycle problem. We turn G into an
  instance of TSP as follows let G (V, E) be
  the complete graph on V

17
Approximation

• E (u, v) u, v Î V and u ? v.
• Assign an integer cost to each edge in E
• We can clearly go from G to G in polynomial time
  (in V and E).
• Now consider the instance ltG, cgt of TSP.
• Assume G had a hamiltonian cycle H since c
  assigns a cost of 1 to each edge in E, H is a
  tour with c(H) V.
• Assume G does not contain a hamiltonian cycle
  any tour in G must contain an edge not in E.
  Cost (rV 1) (V - 1) rV V gt
  rV.

18
Approximation

• Because of the difference in cost between edges
  in G and those not in G, the difference in cost
  between a tour in G and one with some edge not in
  G is at least V. Now apply the approximation
  algorithm A to the instance ltG, cgt of TSP.
  Since A is guaranteed to return a tour of cost no
  more than r times optimal, if G contains a
  hamiltonian cycle, then A must return it if G
  does not contain one, then A returns a tour - in
  G - of cost more than rV. In either case, we
  can decide the hamiltonian cycle problem in
  polynomial time, which, because HAM-CYCLE Î NPC,
  would imply P NP.

19
Approximation

• The Set-Covering Problem. This is a
  generalization of the vertex-cover problem let X
  be a finite set and F be a family of subsets of
  X, such that every element of X belongs to at
  least one subset of F . We say that S Î F covers
  its elements. The problem is to find a
  minimum-size subset C Í F whose members cover
  all of X We say that anyC
  satisfying the previous equation covers X.

20
Approximation

• The decision version asks about the existence of
  a cover of size at most k. One can show (35.3.-2)
  that the decision version of the set-covering
  problem is in NPC. We propose the following as a
  polynomial-time approximation algorithm

21
Approximation

• Theorem 35.4. Greedy-Set-Cover is a
  polynomial-time r(n)-approximation algorithm,
  where r(n) H(maxS S Î F ),
• with H(d) Si1d(1/i), the dth Harmonic Number
  (H(0) º 0).
• Proof.
• a) The algorithm is polynomial-time in X and F
  . We can observe that the number of
  iterations of the while loop in lines 3-6 is
  bounded above by min(X, F ) the loop body
  can be implemented to run in time O(XF ).
  Thus the whole algorithm can be implemented in
  time O(XF min(X, F )). A linear
  implementation is possible (53.3-3).

22
Approximation

• b) The approximation ratio. We will assign a cost
  of 1 to each set selected by the algorithm,
  distributing this cost uniformly over all the
  elements of this set that are being covered for
  the first time. If we let Si denote the ith
  subset added by the algorithm, we have a cost of
  1 added when Si is selected if cx denotes the
  cost of an element x Î X (in particular x Î Si)
  covered for the first time by Si, we have the
  formula
• Since we assign one unit of cost at each step of
  the algorithm, C Sx Î X cx . Furthermore,
  the cost assigned to an optimal cover is given by
  SC

23
Approximation

• b) The approximation ratio. We will assign a cost
  of 1 to each set selected by the algorithm,
  distributing this cost uniformly over all the
  elements of this set that are being covered for
  the first time. If we let Si denote the ith
  subset added by the algorithm, we have a cost of
  1 added when Si is selected if cx denotes the
  cost of an element x Î X (in particular x Î Si)
  covered for the first time by Si, we have the
  formula
• Since we assign one unit of cost at each step of
  the algorithm, C Sx Î X cx . Furthermore,
  the cost assigned to an optimal cover is given by
  SS Î CSxÎScx. Since each x must be in at least
  one set of any optimal cover,

24
Approximation

• SS Î CSxÎScx Sx Î X cx , where we are counting
  each x only once in the second sum. Combining the
  inequalities, we have C Sx Î X cx SS Î
  CSxÎScx .
• Claim For any set S Î F , SxÎScx H(S).
• Pf. See textbook.
• Combining these results
• C Sx Î X cx SS Î CSxÎScx SS Î C
  H(S)
• C H(maxS S Î F ),
• giving the desired conclusion.

25
Approximation

• Randomized Algorithms.
• Def. we say that randomized algorithm for a
  problem has an approximation ratio of r(n) if,
  for any input of size n, the expected cost C of a
  solution produced by the randomized algorithm is
  with a factor of r(n) of the cost C of an
  optimal solution max(C/C, C/C) r(n).
• The basic difference between deterministic and
  randomized algorithms is that the randomized
  algorithms deal with expected costs, and expected
  bounds, rather than deterministic ones any
  particular instance could be much worse or much
  better than the expected value.

26
Approximation

• MAX-3-CNF-SAT. Given any instance of 3-CNF-SAT,
  this instance may or may not be satisfiable. For
  it to be satisfiable, every clause must evaluate
  to 1. We would like to know how close to
  satisfiable an instance is how many clauses can
  be satisfied?
• A randomized algorithm would set each variable to
  true with probability p, and false with
  probability q 1 - p.
• Assumption no clause contains both a variable
  and its negation. Can be removed w.l.g. - Ex.
  35.4-1.

27
Approximation

• Theorem 35.6. Given an instance of MAX-3-CNF-SAT
  with n variables x1, x2, , xn and m clauses, the
  randomized algorithm that independently sets each
  variable to 1 with probability 1/2 and to 0 with
  probability 1/2 is a randomized 8/7-approximation
  algorithm.
• Proof. For i 1, , m let Yi denote the
  indicator random variable which takes the value 1
  if the ith clause is satisfied, 0 if not Yi
  I(clause i is satisfied). By assumption, no
  literal appears more than once in a clause, and a
  variable cannot appear along with its negation
  this simply means that the settings of the three
  literals are independent.

28
Approximation

• Pr(Yi is not satisfied) Pr(l1i 0)Pr(l2i
  0)Pr(l3i 0) 1/21/21/2 1/8.
• Pr(Yi is satisfied) 1 - Pr(Yi is not satisfied)
  7/8.
• By definition of expected value
• EYi 1 Pr(Yi is satisfied) 0 Pr(Yi is not
  satisfied) 7/8.
• If we let Y denote the number of satisfied
  clauses, we can represent Y Y1 Y2 Ym.
  Applyibg the expectation operator to both sides,
  and making use of its linearity (over sums of
  independent random variables) EY Si1m EYi
  m7/8. Since m is an upper bound on the number
  of satisfied clauses, the approximation ratio is
  m/(m7/8) 8/7. QED.

29
Approximation

• Linear programming the Algorithm. Skip.

30
Approximation

• The Subset-Sum Problem.
• The language ( decision problem) SUBSET-SUM is
  defined as
• SUBSET-SUM lt S, t gt S is a finite subset of
  N, t Î N, and there exists S Í S such that t
  SsÎS s .
• The optimization problem asks to find the largest
  sum t.
• We will first find an exponential-time exact
  solution, and then modify it so that it becomes a
  fully-polynomial approximation scheme (
  polynomial in the size of the input and in 1/e,
  where e gt 0 is a parameter.

31
Approximation

• An exponential-time exact algorithm. Assume S
  x1, x2, , xn. If L l1, l2, , lk is a list
  of integers, L xi l1 xi, l2 xi, , lk
  xi. The procedure Merge-Lists(L, L) returns
  the sorted list that is the merge of the two
  lists with duplicates removed. It can be
  implemented, obviously, in O(L L).

32
Approximation

• Since the lists could double in length through
  each pass through the loop, the total time could
  be exponential. It could be better (e.g.,
  polynomial) if t is a polynomial in S or all
  the numbers in S are bounded by a polynomial in
  S. (why? - the first should be obvious since
  the number of elements under consideration can
  never be more than polynomial in S the
  second?)
• To show the algorithm correct, let Pi denote the
  set of all values that can be obtained by
  selecting a (possibly empty) subset of x1, x2,
  , xi and summing its members. We can easily
  prove by induction on i that Li is a sorted list
  containing every element of Pi, t.

33
Approximation

• We will now modify this exact algorithm so that
  it loses exactness but gains polynomiality - a
  tradeoff. The only way to guarantee
  polynomiality it to ensure that the size of the
  lists Li do not grow exponentially we need to
  trim the lists after each merge operation.
• How? We give a trimming parameter d Î (0, 1).
  Trimming a list L by d means removing as many
  elements as possible in such a way that for every
  y removed form L, the resulting list L contains
  an element z such that y/(1 d) z y. z
  represents y in the new list.

34
Approximation

• We introduce the algorithm (L y1, , ym)

35
Approximation

• It is clear that the running time of Trim is
  Q(m) it should also be clear that the gaps
  between successive remaining members increase
  exponentially you can think of this algorithm as
  providing a near geometric series with ratio (1
  d) that will act as a sieve, leaving only those
  elements that jump through the next element of
  the series, while dropping all the intermediate
  ones.
• The next step involves introducing the
  approximation parameter e Î (0, 1). The
  procedure must return a number z whose value is
  within a 1 e factor of the optimal solution.

36
Approximation

• The algorithm. What we must show is that the use
  of the Trim procedure forces the lists Li to grow
  in ways that are no worse than polynomial in n.
  We must also show that the remaining elements
  allow us to approximate the optimal solution
  within the claimed multiplicative factor.

37
Approximation

• Theorem 35.8. Approx-Subset-Sum is a fully
  polynomial-time approximation scheme for the
  Subset-Sum problem.
• Proof. Trimming Li and removing from Li every
  element greater than t maintain invariant the
  property that every element of Li is also an
  element of Pi. Thus z is the sum of some subset
  of S. If y Î Pn is an optimal solution of the
  subset-sum problem, we must have that z y.
  We need to show two things
• y/z 1 e.
• The running time of the algorithm is polynomial
  in both 1/e and the size of the input.

38
Approximation

• Proof of 1.
• Claim for every element y Î Pi, y t, there is
  a z Î Li s.t. y /(1 e/(2n))i z y.
• Pf. By induction on i.
• If y Î Pn is an optimal solution, the inequality
  above must hold for y, and thus there is a z Î
  Ln s.t. y/(1 e /(2n))n z   y.
• The left-hand inequality becomes y/z (1 e
  /(2n))n. Since z is, by definition, the largest
  element of Ln, we must have y/z (1 e
  /(2n))n.
• We still need to prove (1 e /(2n))n (1 e).

39
Approximation

• The first observation comes from Calculus
  limn (1 e/(2n))n ee/2.
• If we think of n as a continuous variable,
  consider the function f (n) ln((1 e/(2n))n )
  n ln(1 e/(2n)). Differentiating w.r.t. n
• f(n) ln(1 e/(2n)) n/(1 e/(2n))(-e/(2n2))
• ln(1 e/(2n)) - e/(2n e) // expand
  ln is a power series
• -e/(2n e) e/(2n) - (e/(2n))2/2
  (e/(2n))3/3 - (e/(2n))4/4
• -e/(2n e) e/(2n) - (e/(2n))2/2
  g(n) // g(n) gt 0
• e2(2n - e)/(8n2(2n e)) g(n) gt 0
• Since (1 e/(2n))n exp(f(n)), differentiation,
  and the inequality above, give that (1 e/(2n))n
  is monotonically increasing in n. (Ex. 35.5-3)

40
Approximation

• For n large
• (1 e/(2n))n ee/2 1 e/2 (e/2)2 1 e.
• The second inequality follows from an examination
  of the power series expansion ex 1 x
  x2/2! x3/3! 1 x x2 (- x2/2!
  x3/3! ) and we use 0 lt x lt 1 to prove that
  -xn/n! xn1/(n1)! lt - n xn1/(n1)! for
  n gt 1.
• The third follows from the fact that 0 lt e lt 1.
  This completes the proof of 1.

41
Approximation

• Proof of 2. We derive a bound on the length of
  Li. the trimming construction implies that, after
  trimming, successive elements z and z of Li must
  differ by at least a factor of 1 e/(2n) z/z gt
  1 e/(2n). Each list must contain the value 0,
  possibly the value 1, and up to floor(log1e/(2n)
  t) additional values. The number of elements in
  each list Li is at most
• log1e/(2n) t 2 ln(t)/ln(1 e/(2n))
  2 2n (1 e/(2n)) ln(t)/e 2 //
  x/(1 x) ln(1 x) x, for x gt -1 //
  proof compare derivatives over the range 4n
  ln(t)/e 2 // 0 lt e lt 1
• This bound is polynomial in the size of the input

42
Approximation

• which is the number of bits lg(t) needed to
  represent t, plus the number of bits needed to
  represent S (polynomial in n) and in 1/e. Since
  the running time of Approx-Subset-Sum is
  polynomial in the lengths of the Li,
  Approx-Subset-Sum is a fully polynomial
  approximation scheme.