Approximation Algorithm: Iterative Rounding

1
Approximation Algorithm Iterative Rounding

• Lecture 15 March 9

2
Lower bound and Approximation Algorithm
For NP-complete problem, we cant compute an
optimal solution in polytime.
The key of designing a polytime approximation
algorithm is to obtain a good (lower or upper)
bound on the optimal solution.
The general strategy (for a minimization problem)
is
lowerbound
SOL
OPT
SOL c lowerbound ? SOL c OPT
3
Lower bound and Approximation Algorithm
lowerbound
SOL
OPT
To design good approximation algorithm, we need a
good lowerbound.
For example, if 100 lowerbound OPT for some
instance, then if we compare SOL to lowerbound
to analyze the performance, we could not achieve
anything better than an 100-approximation
algorithm.
4
Lower bound and Approximation Algorithm
lowerbound
SOL
OPT
Goal to find a lowerbound as close to OPT as
possible.
In metric TSP and the minimum Steiner tree
problem, we use minimum spanning tree as a
lowerbound.
In general, it is often difficult to come up with
a good lowerbound.
5
Linear Programming and Approximation Algorithm
LP
lowerbound
SOL
OPT
Linear programming a general method to compute a
lowerbound in polytime.
To computer an approximate solution, we need to
return an (integral) solution close to an
optimal LP (fractional) solution.
6
An Example Vertex Cover
Vertex cover find an minimum subset of vertices
which cover all the edges.
A linear programming relaxation of the vertex
cover problem.
Clearly, LP is a lowerbound on the optimal vertex
cover, because every vertex cover corresponds to
a feasible solution of this LP.
7
An Example Vertex Cover
How bad is this LP?
LP n/2
LP 2.5
OPT 4
OPT n-1
1
0.5
1
0
0.5
0.5
0.5
0.5
1
1
8
An Example Vertex Cover
Optimal integer solution.
Integrality gap

Optimal fractional solution.
Over all instances.
In vertex cover, there are instances where this
gap is almost 2.
9
Half-integrality
Theorem For the vertex cover problem,
every vertex (or basic) solution of the LP
is half-integral, i.e. x(v) 0, ½, 1
There is a 2-approximation algorithm for the
vertex cover problem.
The integrality gap of the vertex cover LP is at
most 2.
10
Survivable Network Design

• Input
• An undirected graph G (V,E),
• A cost c(e) on each edge,
• A connectivity requirement r(u,v) for each pair
  u,v.

• Output
• A minimum cost subgraph H of G which has
• r(u,v) edge-disjoint paths between each pair
  u,v.
• (That is, H satisfies all the connectivity
  requirements.)

11
Survivable Network Design

• Minimum spanning tree r(u,v) 1 for all pairs.
• Minimum Steiner tree r(u,v) 1 for all pair of
  required vertices.
• Hamiltonian path r(u,v) 2 for all pairs and
  every edge has cost 1.
• k-edge-connected subgraph r(u,v) k for all
  pairs.
• Minimum cost k-flow r(s,t) k for the source s
  and the sink t.

Survivable Network Design is NP-complete.
12
Linear Programming Relaxation
At least r(u,v) edges crossing S
S
For each set S separating u and v, there should
be at least r(u,v) edges crossing S. Let f(S)
max r(u,v) S separates u and v.
u
v
for each subset S of V
13
Special Case Minimum Spanning Tree
How bad is this LP?
LP n/2
LP 2.5
OPT 4
OPT n-1
for each subset S of V
Cannot even solve minimum spanning tree!
0.5
0.5
1
1
0.5
0.5
1
1
0.5
14
Half-integrality
Does the LP has half-integral optimal solution?
Consider the minimum spanning tree problem, i.e.
f(S)1 for all S.
Peterson Graph
All 1/3 is a feasible solution and has cost 5.
Any half-integral solution having cost 5 must be
a Hamitonian cycle.
But Peterson graph does not have an Hamitonian
cycle!
So, no half-integral optimal solution!
15
Separation Oracle
for each subset S of V
There are exponentially many constraints, but
this LP can still be solved in polynomial time by
the ellipsoid method. The reason is that we can
design a polynomial time separation oracle to
determine if x is a feasible solution of the LP.
16
Separation Oracle
for each subset S of V
Remember f(S) max r(u,v) S separates u and
v.
At least r(u,v) edges crossing S
S
Max-Flow Min-Cut Every (u,v)-cut has at least
r(u,v) edges if and only if there are r(u,v)
flows from u to v.
u
v
Separation oracle check if each pair u,v has a
flow of r(u,v)!
17
Linear Programming Relaxation
for each subset S of V
What is the integrality gap of this LP?
18
Moment of Inspiration
All 1/3 is a feasible solution.
But this is not a vertex solution!
This is a vertex solution.
Thick edges have value 1/2 Thin edges have value
1/4.
19
Structural Result of the LP
Kamal Jain
Theorem. Every vertex solution has an edge with
value at least 1/2
Corollary. There is a 2-approximation algorithm
for survivable network design.
20
Iterative Rounding

• Initialization H , f f.
• While f ? 0 do
• Find a vertex solution, x, of the LP with
  function f.
• Add every edge with x(e) 1/2 into H.
• Update f for every set S, set
• Output H.

A new vertex solution is computed in each
iteration
Update the connectivity requirements.
Guaranteed to exist
21
Analysis
Corollary. There is a 2-approximation algorithm
for survivable network design.
Intuitive reason we only pick an edge when the
LP picks at least half.
Proof
Let say we pick an edge e.
Key LP-c(e)x(e) is a feasible solution for the
next iteration.
cost(H) c(e) cost(H) 2c(e)x(e) cost(H)
2c(e)x(e) 2(LP-c(e)x(e)) 2LP
2OPT.
22
Some Remarks

1. The iterative rounding algorithm performs very
   well in practice.
2. No combinatorial algorithm has an performance
   ratio better than O(log n).

23
Spanning Tree with Degree Constraints

• Input
• An undirected graph G (V,E),
• A degree upper bound k.

• Output
• A spanning tree with degree at most k.

NP-complete (Hamiltonian path when k2).
24
Spanning Tree with Degree Constraints
Motivation to find a spanning tree in which
there is no overloaded vertices.
Furer and Raghavachari 92

• Given k, there is a polynomial time algorithm
  which does the following
• Either the algorithm
• Show that there is no spanning tree with maximum
  degree at most k.
• Find a spanning tree with maximum degree at most
  k1.

In other words, there is an 1 algorithm for this
problem!
25
Minimum Spanning Tree with Degree Constraints

• Input
• An undirected graph G (V,E),
• A cost c(e) on each edge e,
• A degree upper bound k.

• Output
• A minimum spanning tree with degree at most k.

Question Is there a 1 algorithm for this
problem as well? That is, a
polytime algorithm which returns a
minimum spanning tree with maximum degree at
most k1.
26
Conjecture
Let OPT be the minimum cost of a spanning tree
with maximum degree k.
Goemans
Conjecture Given k, there is a polynomial time
algorithm which returns a spanning tree with cost
at most OPT and maximum degree at most k1.
Note that we do not restrict ourselves to MST.
27
Previous Work
Reference Cost Guarantee Degree
Furer and Raghavachari 92 Unweighted Case k1
8 k
Konemann, Ravi 01 02 O(1) O(klog n)
CRRT 05 06 O(1) O(k)
Ravi, S. 06 MST kp (pdistinct costs)
Goemans 06 1 k2
28
Our Result
Singh Lau 07
Theorem Given k, there is a polynomial time
algorithm which returns a spanning tree with cost
at most OPT and maximum degree at most k1.
Technique Adaptation of iterative rounding, but
we do not round.
Mohit Singh
29
Spanning Tree Polytope

• Formulate a linear programming relaxation.
• min ?e2 E ce xe
• s.t. ?e2 E(V) xe V-1
• ?e2 E(S) xe S-1
• xe 0
• E(S) set of edges with both endpoints in S.
• Separation oracle Cunningham 84
• ) Optimization in poly time

Any tree has n-1 edges
for each subset S of V
Cycle elimination constraints
30
Spanning Tree Polytope
min ?e2 E ce xe s.t. ?e2 E(V) xe V-1
?e2 E(S) xe S-1 xe 0
Recall A vertex solution is the unique solution
of m linearly independent tight
inequalities, where m denotes the
number of variables.
31
Spanning Tree Polytope
min ?e2 E ce xe s.t. ?e2 E(V) xe V-1
?e2 E(S) xe S-1 xe 0
If there is an edge of 0, delete it.
If there exists a leaf vertex v, then include the
edge incident at v in and remove v from G.
32
Spanning Tree Polytope
min ?e2 E ce xe s.t. ?e2 E(V) xe V-1
?e2 E(S) xe S-1 xe 0
Theorem There are at most n-1 linearly
independent tight inequalities of this
type, where n denotes the number of
vertices.
Claim A vertex solution of the LP must have a
leaf vertex.
If there is no leaf vertex, then every vertex has
degree 2, and hence there are at least 2n/2n
edges, a contradiction to the above theorem.
33
Spanning Tree Polytope

• No 1-edge ) many fractional edges
• Vertex Solution with few constraints ) few
  fractional edges
• Derive contradiction

So a vertex solution must have an edge of 0 or a
leaf vertex, in either case we can finish by
induction. This proves that the linear program
has integer optimal solution.
34
A Simple 2 Algorithm

• Extend spanning tree polyhedron
• OPT min ?e2 E ce xe
• s.t. ?e2 E(V) xe V-1
• ?e2 E(S) xe S-1 ?e2
  ?(v) xe Bv 8 v 2 W
• xe 0

for each subset S of V
The degree constraint of each vertex could be
different
Goal Find a spanning tree with cost at most OPT
and degree is violated by at most 2.
35
First Try

• Initialize F?.
• While F is not a spanning tree
• Solve LP to obtain vertex solution x.
• Remove all edges e s.t. xe0.
• If there is a leaf vertex v with edge u,v, then
  include u,v in F. Decrease Bu by 1. Delete v
  from G.

• If algorithm works then we solve the problem
  optimally!
• Cannot pick an edge with 1 gt xe ½ lose
  optimality of the cost.

36
A Simple 2 Algorithm

• Initialize F?.
• While F is not a spanning tree
• Solve LP to obtain extreme point x.
• Remove all edges e s.t. xe0.
• If there is a leaf vertex v with edge u,v, then
  include u,v in F. Decrease Bu by 1. Delete v
  from G.
• If there is a vertex v2 W such that degE(v)
  Bv2, then remove the degree constraint of v.

37
Removing Degree Constraints
If there is a vertex v2 W such that degE(v)
Bv2, then remove the degree constraint of v.
This is only done one, and the degree constraint
is violated by at most 2!
1
1/3
Bv1
Bv1
1
1/3
1
1/3
38
A Simple 2 Algorithm

• Initialize F?.
• While F is not a spanning tree
• Solve LP to obtain extreme point x.
• Remove all edges e s.t. xe0.
• If there is a leaf vertex v with edge u,v, then
  include u,v in F. Decrease Bu by 1. Delete v
  from G.
• If there is a vertex v2 W such that degE(v)
  Bv2, then remove the degree constraint of v.

• Lemma For any vertex solution x, one of the
  following is true
• Either there is a leaf vertex v.
• Or there is a vertex with degree constraint such
  that degE(v)Bv2

39
Analysis
OPT min ?e2 E ce xe s.t. ?e2 E(V) xe
V-1 ?e2 E(S) xe S-1 ?e2 ?(v) xe
Bv 8 v 2 W xe 0
Theorem There are at most n-1 linearly
independent tight inequalities of this
type, where n denotes the number of
vertices.
Proof of the Lemma Suppose not. Every vertex
has degree at least 2. Every vertex in W has
degree at least 4. E ½(2(n-W)4W)
nW The set of tight constraints E
n-1W A contradiction.
40
A Simple 2 Algorithm

• A quick summary
• Find a leaf vertex v, add the only edge at v and
  remove v.
• - Dont lose the cost optimality and the degree
  bound.
• Find a vertex with at most Bv2 neighbours and
  has a degree
• constraint, remove the degree constraint.
• - Violate the degree bound by at most 2.

By the Lemma, one of the two possibilities must
hold.
41
Obtaining 1 algorithm
v leaf ) xu,v1 Why not pick any e s.t. xe1 ?

• Initialize F?.
• While F is not a spanning tree
• Solve LP to obtain extreme point x.
• Remove all edges e s.t. xe0.
• If there exists a leaf vertex v with edge u,v,
  then include u,v in F. Decrease Bu by 1. Delete
  v from G.
• If there exists a vertex v2 W such that degE(v)
  Bv2, then remove the degree constraint of v.

Replace by Bv1
42
Concluding Remarks

• No combinatorial algorithm has a good performance
  ratio.
• Similar techniques can be used for more general
  problems, e.g. minimum maximum degree k-ec
  subgraph
• Can deal with lower and upper bounds.
• A unifying framework for network design problems.
• Proofs are based on uncrossing techniques in
  combinatorial optimiation.