Approximation Algorithms

1
Approximation Algorithms

• Analysis of Algorithms CS 515
• Ashish Kapoor

2
Motivation Getting around NP-completeness

• Exponential time may be acceptable for small
  inputs Brute Force
• Isolate special cases that can run in polynomial
  time Divide-and-Conquer
• Near-optimal solutions may be acceptable
  Approximation Algorithms

3
Performance Ratios

• C is the cost of the optimal solution
• C is the cost of the solution produced by an
  approximation algorithm
• An algorithm has an approximation ratio of ?(n)
  if for an input of size n, C is within a factor
  of C

4
Performance Ratios cont.

• Maximization problem
• 0 lt C lt C
• C / C factor by which the cost of the optimal
  solution is larger than the cost of the
  approximate solution
• Minimization problem
• 0 lt C lt C
• C / C factor by which the cost of the
  approximate solution is larger than the cost of
  the optimal solution

5
How do approximations algorithms work?

• Exploit the nature of the problem
• Use greedy techniques
• Use linear programming
• Use dynamic programming
• Use random assignments

6
Speaking of random stuff

• random_sort (int array )
• boolean done false
• while (!done)
• work_array copy_array(array)
• randomize_array(work_array)
• done is_sorted(work_array)
• return work_array
• I really tried to resist but hey, you had to know
  this was coming.

7
The vertex-cover problem

• A vertex cover of an undirected graph G(V,E) is
  a subset V of V such that if (u,v) is an edge of
  G, then either u belongs in V or v belongs in V
  (or both)
• The size of a vertex cover is the number of
  vertices in it

8
The vertex-cover problem cont.

• The vertex-cover problem is to find a vertex
  cover of minimum size
• Such a vertex-cover is called an optimal vertex
  cover
• This problem is NP-complete

9
Consider the graph
b
c
d
a
e
f
g
By inspection, optimal vertex cover is
b,d,e C size of optimal solution 3
10
Approx-Vertex-Cover (G)

• C ? NULL
• E ? EG
• while E ! NULL
• do let (u,v) be an edge of E
• C ? C U u,v
• remove from E all edges incident on u
  or v
• return C

11
Back to our graph
b
c
d
a
e
f
g
C
a b
c d
e f
Approximate vertex cover. C6
E
(a,b)
(b,c)
(c,d)
(c,e)
(d,e)
(d,f)
(d,g)
(e,f)
12
Improved version
b
c
d
a
e
f
g
C
d e b c
Approximate vertex cover. C4
E
(a,b)
(b,c)
(c,d)
(c,e)
(d,e)
(d,f)
(d,g)
(e,f)
13
Is it possible to approximate the optimal
solution?
b
c
d
a
e
f
g
In this case, and with our algorithm, NO!
14
Analysis of Algorithm

• Running time O(VE)
• 2-Approximation Algorithm
• Minimization problem C 3 C 6
• Factor C/C 2

15
2-Approximation Algorithm

• A set of edges picked by approximation
  algorithm
• No 2 edges in A share an endpoint. Thus no 2
  edges in A are covered by the same vertex in C
• Lower bound C gt A
• We pick an edge for which neither of its
  endpoints are already in C
• Upper bound C 2A
• Therefore C 2A lt C

16

• Any questions Connie?

17
The traveling-salesperson problem.

• Given a complete undirected graph G(V,E)
• Each edge has a nonnegative integer cost c(u,v)
• Find a Hamiltonian cycle of G with minimum cost.
• This is a NP complete problem

18
TSP side note

• Performance of exact methods to solve the problem
• Graph Year versus the size of problem solved to
  optimality

19
TSP side note cont.

• Largest problem solved to optimality till circa
  2002 has 16,862 cities
• Using heuristics, a problem with 1,904,711 cities
  was solved to 1 of optimality

20
TSP side note cont. Optimal solution to
21
TSP side note cont. Heuristic solution to
22
Metric TSP approximation

• The cost function satisfies the triangle
  inequality for all vertices u, v, w in V
• c(u,w) lt c(u,v) c(v,w)
• Eulerian graphs and Eulerian circuits
• Eulerian circuit cycle that uses each edge
  exactly once
• Eulerian graph graph with a Eulerian circuit
• An undirected graph is Eulerian iff each vertex
  has even degree.

23
Approximate-TSP-tour (G)

• Find MST T of G
• Double every edge of the MST to get a Eulerian
  graph G
• Find an Eulerian circuit E on G
• Output the vertices of G in order of appearance
  in E

24
Analysis

• This is a 2 approximation algorithm
• Proof
• cost(T) lt C (if we remove an edge from the
  optimal solution, we get a spanning tree)
• cost(E) 2 cost(T)
• C lt cost(E) (due to triangular inequality)
• C lt cost(E) 2cost(T) lt 2 C

25
Can we improve this approximation

• The previous algorithm was a factor 2 algorithm.
• Recall line 2
• Double every edge of the MST to get a Eulerian
  graph G
• If we can work avoid this, we could possibly have
  a better solution !

26
Perfect Matching

• Matching a matching is a subset M of E such that
  for all vertices v in V, at most one edge is
  incident on v.
• Perfect Matching is a matching M such that for
  all vertices v in V exactly one edge of M is
  incident on v.

27
1.5 approximation algorithm

• Find a MST T of G
• Find a minimum cost perfect matching M on the set
  of odd degree vertices in T.
• Add M to T to get the Eulerian graph G
• Find an Eulerian circuit E on G
• Output the vertices of G in order of appearance
  in E

28
Example
29
Analysis

• cost(E) cost(T) cost(M) lt C C/2
• C lt cost(E) lt 1.5 C

30
Randomized approximation algorithm for
MAX-3-CNF-SAT

• Each clause has 3 distinct literals. Also assume
  that no clause contains a variable and its
  negation.
• Problem Return an assignment of variables which
  maximizes the number of clauses which evaluate to
  true.

31
The random approximation algorithm

• Now pay close attention because this algorithm is
  really complex! No really it is.
• Are you ready to be amazed? Here it comes
• Randomly assign a 0 or a 1 value to all variables.

32
Analysis

• We have set each variable to 1 with a ½
  probability.
• We have set each variable to 0 with a ½
  probability.
• We define an indicator random variable
• Yi I clause i is satisfied
• Yi 1 as long as at least one of the literals in
  the ith clause has been set to 1.
• A clause is not satisfied iff all its literals
  are 0.
• Pr clause i is not satisfied (½)3 1/8

33
Analysis cont.

• So Pr clause i is satisfied 1 1/8 7/8
• Lemma 5.1 (CLRS)
• Given a sample space S and an event A in the
  sample space, let XA IA. Then EXA PrA
• EYi 7/8

34
Analysis cont.

• Let Y number of satisfied clauses.
• Y Y1Y2.... Ym
• So EY
• Esum i to m of Yi
• sum i to m of EYi by linearity of expectation
• 7m/8
• Since m is the upper bound on the number of
  satisfied clauses, the approximation ratio is
  m/(7m/8) 8/7
• Therefore this is a 8/7 approximation algorithm
• gotta love these proofs by randomificationality
  

35
Approximate weighted vertex cover using linear
programming

• CLRS says
• We shall, however, compute a lower bound on the
  weight of the minimum-weight vertex cover, by
  using a linear program. We will then round this
  solution and use it to obtain a vertex cover. (my
  italics and underline)

36
The equations

• Associate a variable x(v) with each vertex v in
  V. x(v) is in 0,1
• Linear programming system
• Minimize sum of w(v).x(v) for all v in V
• x(u) x(v) gt 1 for each u,v in E
• x(v) belongs in 0,1 for each v in V

NP-hard
37
So we change the equations

• Minimize sum of w(v).x(v) for all v in V
• x(u) x(v) gt 1 for each u,v in E
• x(v) lt 1 for each v in V
• x(v) gt 0 for each v in V

38
Analysis

• The algorithm used to compute the minimal weight
  vertex cover using this system of linear
  equations is a 2-approximate algorithm.
• Proof is left as an exercise.

39
References

• Chaper 35, Introduction to Algorithms, Cormen et.
  al.
• University of Copenhagen, Stefan Røpke, lecture
  notes
• http//www.diku.dk/undervisning/2003e/404/