CS 450 Design

1
CS 450 Design Analysis of AlgorithmsNotes
14

• Fall 2009Sarah Mocas

2
Chapter 35 - Approximation Algorithms

• Reading
• Introduction to Algorithms, by Cormen,
  Leiserson, Rivest Stein
• Chapter 35
• Approximate Algorithms
• (only sections covered in the notes
• Introduction and Section 35.1)

3
Chapter 35 - Approximation Algorithms

• Motivation Getting around NP-completeness
• Exponential time may be okay for small inputs -
  Brute Force
• Isolate special cases that can run in polynomial
  time - Divide-and-Conquer
• Near-optimal solutions may be good enough -
  Approximation Algorithms

4
Chapter 35 - Approximation Algorithms

• Approximation Algorithm
• produces near-optimal solution
• An algorithm has approximation ratio p(n) if
• C cost of the solution produced by the
  approximation algorithm
• C cost of the optimal solution
• n size of input instance

An algorithm has an approximation ratio of ?(n)
if for an input of size n, C is within a factor
of C
5
Chapter 35 - Approximation Algorithms

• Performance Ratios
• Maximization problem
• 0 lt C C
• C/C factor by which the cost of the optimal
  solution is larger than the cost of the
  approximate solution
• Minimization problem
• 0 lt C C
• C/C factor by which the cost of the approximate
  solution is larger than the cost of the optimal
  solution

C cost of the solution produced by the
approximation algorithm C cost of the optimal
solution
6
Chapter 35 - Approximation Algorithms

• Examples
• MAX C 4, C 3 max(4/3, 3/4) 4/3
• 1.3 is the factor by which the cost of the
  optimal solution is larger than the cost of the
  approximate solution
• MIN C 3, C 4 max(3/4, 4/3) 4/3
• 1.3 is the factor by which the cost of the
  approximate solution is larger than the cost of
  the optimal solution
• OPT. C 4, C 4 max(4/4, 4/4) 1
• This means the approximation solution is the
  same as an optimal solution

C cost of the solution produced by the
approximation algorithm C cost of the optimal
solution
7
Chapter 35 - Approximation Algorithms

• Approximation Scheme
• (1e)-approximation algorithm for fixed e
• also gets e gt 0 as an input
• Polynomial-Time Approximation Scheme
• running time is polynomial in n
• Fully Polynomial-Time Approximation Scheme
• running time is also polynomial in (1/e)
• constant-factor decrease in e causes only
  constant-factor running-time increase
• Example running time O((1/e)2n3)

8
Chapter 35 - Approximation Algorithms

• How do approximations algorithms work?
• Exploit the nature of the problem
• Use greedy techniques
• Use linear programming
• Use dynamic programming
• Use random assignments (algorithms)

Its a bag of tricks.
9
Chapter 35 - Approximation Algorithms

• VERTEX-COVER
• Polynomial-time 2-approximation algorithm
• TRAVELING-SALESPERSON PROBLEM
• TSP with triangle inequality
• Polynomial-time 2-approximation algorithm
• TSP without triangle inequality
• Negative result no polynomial-time
  p(n)-approximation
• SET-COVER
• polynomial-time p(n)-approximation algorithm
• p(n) is a logarithmic function of set size
• SUBSET-SUM
• Fully polynomial-time approximation scheme
• MAX-3-CNF Satisfiability
• Randomized p(n)-approximation algorithm

10
Chapter 35 - Approximation Algorithms

• Vertex-cover
• A vertex cover of an undirected graph G(V,E) is
  a subset V of V so that if (u,v) is an edge of
  G, then either u belongs in V or v belongs in V
  (or both)
• The size of a vertex cover is the number of
  vertices in it

C size of optimal solution 3
b
c
d
a
e
f
g
11
Chapter 35 - Approximation Algorithms

• Vertex-cover problem
• The vertex-cover problem is to find a vertex
  cover of minimum size
• Such a vertex-cover is called an optimal vertex
  cover
• This problem is NP-complete

12
Chapter 35 - Approximation Algorithms

• Approx-Vertex-Cover(G)
• C ? NULL
• E ? EG
• while E ! NULL
• do let (u,v) be an edge of E
• C ? C U u,v
• remove from E all edges incident on u
  or v
• return C

13

• Approx-Vertex-Cover (G)
• C ? NULL
• E ? EG
• while E ! NULL
• do let (u,v) be an edge of E
• C ? C U u,v
• remove from E all edges incident on u
  or v
• return C

C 3 max(6/3, 3/6) 2
b
c
d
a
e
f
g
C
a b
c d
e f
Approximate vertex cover. C6
E
(a,b)
(b,c)
(c,d)
(c,e)
(d,e)
(d,f)
(d,g)
(e,f)
14

• Approx-Vertex-Cover (G)
• C ? NULL
• E ? EG
• while E ! NULL
• do let (u,v) be an arbitrary edge of E
• C ? C U u,v
• remove from E all edges incident on u
  or v
• return C

• C 3
• max(4/3, 3/4) 1.3

b
c
d
a
e
f
g
C
d e b c
Approximate vertex cover. C4
E
(a,b)
(b,c)
(c,d)
(c,e)
(d,e)
(d,f)
(d,g)
(e,f)
15
Chapter 35 - Approximation Algorithms

• Analysis of Approx-Vertex-Cover Algorithm
• Running time O(VE) so ploy. time in G
• With an adjacency list to represent E
• Claim 2-approximation algorithm for VC

Returns a vertex cover at most twice as large as
the optimal

• Approx-Vertex-Cover (G)
• C ? NULL
• E ? EG
• while E ! NULL
• do let (u,v) be an arbitrary edge of E
• C ? C U u,v
• remove all edges incident on u or v
• return C

16
Chapter 35 - Approximation Algorithms
The set C is a vertex cover since the loop on
line 3 runs until every edge is has been covered
by some vertex in C

• Approx-Vertex-Cover(G)
• C ? NULL
• E ? EG
• while E ! NULL
• do let (u,v) be an arbitrary edge of E
• C ? C U u,v
• remove from E all edges incident on u
  or v
• return C

17
Chapter 35 - Approximation Algorithms

• Proof that its a 2-approximation algorithm
• A set of edges picked by the approximation
  algorithm in line 4
• No 2 edges in A share an endpoint because they
  were deleted in line 6. So no 2 edges in A are
  covered by the same vertex in C the same would
  be true of an opt. cover C that was a cover for
  A
• Lower bound C A
• Note here C is a opt. cover for A but maybe not
  be opt. for G ltV,Egt a little more work here

18
Chapter 35 - Approximation Algorithms

• Proof that its a 2-approximation algorithm
• In line 4 we pick an edge for which neither of
  its endpoints are already in C. Every new edge
  added to A adds exactly 2 new vertices to C.
• Upper bound C 2A
• Therefore C 2A 2C
• Conclusion
• Approx-Vertex-Cover is a polynomial time
  2-approximation algorithm

19
Chapter 35 - Approximation Algorithms
Hamiltonian Cycle of an undirected graph G(V,E)
is a simple cycle that contains each vertex in V.

• Traveling-Salesperson Problem
• Given a complete undirected graph G(V,E)
• Each edge has a nonnegative integer cost c(u,v)
• Find a Hamiltonian cycle of G with minimum cost.
  
• Hamiltonian cycle is a NP complete problem

• Largest problem solved to optimality till circa
  2002 has 16,862 cities
• Using heuristics, a problem with 1,904,711 cities
  was solved to 1 of optimality

20
Chapter 35 - Approximation Algorithms

• Bottom line
• If you need to compute the answer to a hard
  problem, look to see if there is a

• Approximation algorithm
• Randomized algorithm
• Way to reformulate the problem to be tractable
• Also an AI solution using a heuristics may work

http//www.flickr.com/photos/vickispix/1751855975/
in/set-72157594298325372/
21
Chapter 35 - Approximation Algorithms

• Steven Skiena Dept. of Computer Science Stony
  Brook University
• Approximation Algorithms
• Lecture Topics Approximating Vertex Cover, The
  Euclidean Traveling Salesman, Non-deterministic
  Turing Machines
• http//www.cs.sunysb.edu/algorith/video-lectures/
  1997-25.html