Semidefinite Programming

1
Semidefinite Programming

• Lecture 22 Apr 2

2
Semidefinite Programming

• What is semidefinite programming?
• A relaxation of quadratic programming.
• A special case of convex programing.
• A generalization of linear programming.
• Can be optimized in polynomial time.

• What is it good for?
• Shannon capacity
• Perfect graphs
• Approximation algorithms
• Image segmentation and clustering
• Constraint satisfaction problems
• Number theory, quantum computation, etc

3
Maximum Cut
(Maximum Cut) Given an undirected graph, with an
edge weight w(e) on each edge e, find a partition
(S,V-S) of V so as to maximum the total weight of
edges in this cut, i.e. edges that have one
endpoint in S and one endpoint in V-S.
4
Maximum Cut
When is computing maximum cut easy?
When we are given a bipartite graph.
The maximum cut problem can also be interpreted
as the problem of finding a maximum bipartite
subgraph.
There is a simple greedy algorithm with
approximation ratio ½.
Similar to vertex cover.
5
Quadratic Program for MaxCut
The two sides of the partition.
if they are on opposite sides.
if they are on the same side.
6
Quadratic Program for MaxCut
This quadratic program is called strict quadratic
program, because every term is of degree 0 or
degree 2.
This is unlikely to be solved in polynomial time,
otherwise PNP.
7
Vector Program for MaxCut
8
Vector Program for MaxCut
This is a relaxation of the strict quadratic
program (why?)
Vector program linear inequalities over inner
products.
Vector program semidefinite program.
Can be solved in polynomial time (ellipsoid,
interior point).
9
Geometric Interpretation
Think of as an n-dimensional vector.
Contribute more to the objective if the angle is
bigger.
10
Demonstration
Rubber band method.
László Lovász
11
Algorithm

• (Max-Cut Algorithm)
• Solve the vector program. Let
  be an optimal solution.
• Pick r to be a uniformly distributed vector on
  the unit sphere .
• Let

12
Analysis
Claim
13
Analysis
Suppose and has an edge.
Contribution to semidefinite program
Contribution to the solution
Approximation Ratio
14
Analysis
Proof Linearity of expection.
Let W be the random variable denoting the weight
of edges in the cut.
Claim
15
Algorithm

• (Max-Cut Algorithm)
• Solve the vector program. Let
  be an optimal solution.
• Pick r to be a uniformly distributed vector on
  the unit sphere .
• Let

Repeat a few times to get a good approximation
with high probability.
This algorithm performs extremely well in
practice.
Try to find a tight example.
16
Remarks
Hard to imagine a combinatorial algorithm with
the same performance.
Assuming the unique games conjecture, this
algorithm is the best possible! That is, it is
NP-hard to find a better approximation algorithm!
17
Constraint Satisfaction Problems
(Max-2-SAT) Given a formula in which each clause
contains two literals, find a truth assignment
that satisfies the maximum number of clauses.
e.g.
An easy algorithm with approximation ratio ½.
An LP-based algorithm with approximation ratio ¾.
An SDP-based algorithm with approximation ratio
0.87856.
18
Vector Program for MAX-2-SAT
(Max-2-SAT) Given a formula in which each clause
contains two literals, find a truth assignment
that satisfies the maximum number of clauses.
Additional variable (trick)
A variable is set to be true if
A variable is set to be false if
19
Vector Program for MAX-2-SAT
Denote v(C) to be the value of a clause C, which
is defined as follows.
Consider a clause containing 2 literals, e.g.
. Its value is
20
Vector Program for MAX-2-SAT
Objective
where a(ij) and b(ij) is the sum of coefficients.
21
Algorithm

• (MAX-2-SAT Algorithm)
• Solve the vector program. Let
  be an optimal solution.
• Pick r to be a uniformly distributed vector on
  the unit sphere .
• Let be
  the true variables.

22
Analysis
Term-by-term analysis.
First consider the second term.
Contribution to semidefinite program
Contribution to the solution
Approximation Ratio
23
Analysis
Term-by-term analysis.
Consider the first term.
Contribution to semidefinite program
Contribution to the solution
Approximation Ratio
24
Algorithm

• (MAX-2-SAT Algorithm)
• Solve the vector program. Let
  be an optimal solution.
• Pick r to be a uniformly distributed vector on
  the unit sphere .
• Let be
  the true variables.

This is a 0.87856-approximation algorithm for
MAX-2-SAT.
Can be improved to 0.931!
25
More SDP in approximation algorithms

• Sparsest cut O(vlog n)
• Constraint satisfaction problems
• Correlation clustering
• Graph colouring

A very powerful tool in the design of
approximation algorithms.
Useful to know geometry and algebra.
Next two lectures SDP and perfect graphs.
26
Summary Topics

• Classical problem TSP, Steiner trees
• Covering problem vertex cover, set cover
• Packing problem knapsack, bin packing
• Graph partitioning problem multiway cut,
  multi-cut
• Job scheduling makespan, general assignment
• Network design Steiner network, degree
  constrained spanning trees
• Constraint satisfaction maximum cut, max 2-SAT

27
Summary Techniques

• Combinatorial arguments TSP, Steiner trees,
  multiway cut
• Greedy algorithm and Randomized rounding set
  cover
• Dynamic programming knapsack, bin packing
• Region Growing multi-cut
• Iterative relaxation scheduling, network design
  problems
• Primal-dual method vertex cover, multi-cut in
  tree
• Semidefinite programming maximum cut, constraint
  satisfaction

28
Concluding Remarks
Learn to design better heuristics. e.g. iterative
rounding, SDP performs extremely well in practice.
Use LP or SDP as a good way to estimate the
optimal value. e.g. help to test the performance
of a heuristic.
Relax the search space to a convex set.
29
Remaining Schedule
Apr 10-11 NO CLASSES Apr 17-18 Last week of
lecture (perfect graphs, SDP) Apr 23-24 Project
presentation (15 minutes) May 1 Project report
deadline May 8 Homework 3 deadline