Semidefinite Programming and Approximation

1
Semidefinite Programming and Approximation
Algorithms for NP-hard Problems A Survey
Sanjeev AroraPrinceton University
2
NP-completeness
Thousands of problems are NP-complete (TSP,
Scheduling, Circuit layout, Machine Learning,..)
Pragmatic Researcher
Why the fuss? I am perfectly content with
approximatelyoptimal solutions. (e.g., cost
within 10 of optimum)
Good news Possible for a few problems.
(Approximation Algorithms)
Bad News NP-hard for many problems. (PCPs)
3
Talk Outline

• Defn of approximation and example
• SDP and its use in approximation
• Understanding SDPs lt-gt high dimensional
  geometry
• Faster algorithms (multiplicative update rule)
• Limitations of SDPs local vs global issues
• Connections (a) metric spaces (b) avg case
  complexity (c) unique games conjecture
• Open problems

4
Approximation Algorithms
MAX-3SAT Given 3-CNF formula ?, find assignment
maximizing the number of satisfied clauses.
An ?-approximation algorithm is one that for
every formula, produces in polynomial time an
assignment that satisfies at least OPT/?
clauses. (? gt 1).
Good News KZ97 An 8/7-approximation
algorithm exists.
Bad News Hastad97 If P ? NP then for every ?
gt 0, an(8/7 -?)-approximation algorithm does not
exist.
(Similar results for many other problems)
5
Good news (for me)

• Status of many basic problems is still
  unresolved
• Vertex Cover
• Sparsest Cut and most graph partitioning
  problems
• Graph coloring
• Random instances of 3SAT

My feeling Interesting algorithms remain
undiscovered semidefinite programming
(SDP) may be helpful.
SDP Generalization of linear programming
Graph
Vector Representation
6
Example 2-approximation for Min Vertex Cover
G (V, E)
Vertex Cover Set of vertices that touches
every edge
LP Relaxation
most
Claim Value at least OPT/2
Proof On Complete Graph Kn, OPT n-1 but
setting all xi 1/2 gives feasible LP soln
Proof Rounding
7
General Philosophy
Interested in NP-hard Minimization
Problem Value OPT
Write tractable relaxationvalue
Round to get a solution of cost
Approximation ratio
Integrality gap
8
Main Idea in SDP Simulate nonlinear programming
Nonlinear program for Vertex Cover
Homogenized
SDP relaxation New variable intended to stand
for
9
How do you understand thesevector programs?
Ans. Interesting geometric analysis
10
Understanding SDPs lt--gt Understanding phenomena
in high-dimensional geometry
Vertex Cover SDP
computes c-approximation for c lt 2 iff following
is true
Vertices n unit vectorsEdges almost-antipodal
pairs
Every graph in this family has an independent
set of size
Rn
Thm Frankl-Rodl87 False.
11
SDP rounding The two generations
First generation Uses random hyperplane as in
GW Edge-by-edge analysis
Max-2SAT and Max-CUT GW94 Graph coloring
KMS95 MAX-3SAT KZ97 Algorithms for Unique
Games..
Second generation Global rounding and
analysis Graph partitioning problems
ARV04, Graph deletion and directed
partitioning problems ACMM05, New analysis of
graph coloring ACC06 Disproof of UGC for
expanding constraints AKKSTV08
(Similarly, two generations of results showing
limitson performance of SDPs)
12
1st Generation Rounding Ratio 1.13.. for MAX-CUT
Goemans-Williamson93
Semidefinite Relaxation DP 91, GW 93
13
Randomized Rounding
GW 93
Rn
v2
v1
v6
Form a cut by partitioning v1,v2,...,vn around a
random hyperplane.
v3
v5
SDPOPT
Old math rides to the rescue...
14
Fact 1 No rounding algorithm can produce a
better solution out of this SDP
Feige-Schectman
Edges between all pairs of vectorsmaking an
angle 138 degrees.
Fact 2 If P NP then impossible to get
1.09-approximation by any efficient algorithm
Hastad97
Fact 3 If unique games conjecture is true, it
is impossible to get a better than
1.13-approximation.KKMO05 (i.e., algorithm on
prev. slide is optimal)
15
2nd Generation for c-balanced
separator
G (V, E) constant c gt0
1
-1
Goal Find cut s.t. each side contains
at least c fraction of nodes and
minimized
SDP
Triangle inequality
Angle subtended by the line joiningtwo of them
on the third is non-obtuse condition.
16
Rounding algorithm for
-approximation
ARV04
1. Pick random hyperplane
S
T
5. Output level of BFS tree with least of edges.
17
Geometric fact underlying the analysis
(restatement of ARV04 Structure Theorem by
AL06)
Vertices unit vectorssatisfying triangle
inequality
(expander ?(S) ?(S) )
Edges
Proof is delicate and difficult
18
Issue of Running Time
Solving SDPs with m constraints takes time.
m n3 in some of these SDPs!
Next few slides Often, can reduce running time
O(n2) or O(n3).
AHK05, AK07
Main idea Primal-dual schema.Solve to
approximate optimality using insights from the
rounding algorithms.
Multiplicative Weight-Update Rule for psd
matrices
19
Classical MW update rule (Example predicting
the market)
1 for correct prediction
0 for incorrect

• N experts on TV
• Can we perform as good as the best expert ?

ThmGoing back to Hannan, 1950s Yes.
20
Weighted Majority Algorithm LW94
Predict according to the weighted majority

• Maintain a weight for each expert. Initially
• At step t, if expert is prediction was
  incorrect,

Claim Expected Payoff of our algorithm
Similar algorithms discovered in a variety of
areas decision theory, learning theory
(boosting), cryptography (hardcore sets),
approx soln of LPs,.. (see survey A, Hazan,
Kale)
21
Primal-dual approach for SDP relaxations A.,
Kale07
At step t
Primal player PSD matrix Pt candidate primal
Let me run the rounding algorithm on Pt, get a
primal integer candidate and point out how
pitiful it is.
Dual player
Feedback matrix Mt
Primal player Pt1 exp(-? ?t Mt)
(Analysis uses formal analogy between real s and
symmetric matrics
Other ingredients flow computations,
eigenvalues, dimension reduction tricks, etc.
22
Implications for geometric embeddings of metric
spaces
(X, d) metric space
x
y
f(y)
d(x, y)
f(x)
C distortion
Thm (Bourgain85) For every X, there is f s.t.
C O(log n).
Open qs since then is it possible to achieve
smaller C forconcrete X, say X ?
CGR05,ALN05 Yes, C possible
for X
KV06 Cannot reduce C below
23
Unique Games
Given Number p, and m equations in n vars of the
form
Promise Either there is a solution that
satisfies fractionof constraints or no
solution satisfies even fraction.
UGC Khot02 Deciding which case holds is
intractable.
Seems to capture our current limitations of
thinking aboutSDPs basis of many recent
hardness results.
24
Anatomy of a UGC-based hardness result
Variables
Interpret as a graph
Prove using harmonicanalysis that near--optimum
solnscorrespond to good Solution to theunique
game
Equations
Variables
Replace edges/verticeswith hypercube-like gadgets
25
Limitations of SDPs
For many problems, we know neither an NP-hardness
result (via PCPs) nor a good SDP-based approach.
Can we show that known SDPs dont work??
1st generation results Specific SDPs dont work
2nd Generation results Large families of LPs or
SDPs dont work
ABL02, ABLT06 Proving integrality gaps
without knowing the LP. Much subsequent work,
especially on families obtained from lift and
project ideas)
26
Lifted SDP relaxations
Recall SDP tries to simulate nonlinear
programming Variable for
Why not take it to the next level? Variablesfor
products of up to k variables.
This is the main idea of Lovasz-Schrijver91,
Sherali-Adams, Lasserre etc.
SDP as a proof system Integrality gaps proved
in 2nd generation results.
27
Main issue Local versus Global
Example Erdos There are graphs on n vertices
that cannot be colored with 100 colors
yet every subgraph on 0.01 n vertices is
3-colorable.
LP relaxations or SDP relaxations concern
local conditions. How well do such local
conditions capture global property in question?
Results for MAX-k-SAT , AAT05, Vertex
CoverABLT06, STT07ab MAX-CUT, Vertex
Cover etc. CMM08
Lifted SDPs. Connections to Proof Complexity.
28
Connections to Avg. Case Complexity(SDP used in
reductions)
Problems like 3SAT seem difficult not only in the
worstcase but also on average. (Needs careful
definition!)
Theory of Avg Case complexity exists, but doesnt
usuallyapply to problems of practical interest.
Recent development Interreducibility among
someaverage case problems of interest.
Feige01Alekh.03
SDP is used in the reduction!
29
Open problems

• Techniques for proving lowerbounds on lifted
  SDPs.(difficult local-global results)
• New rounding algorithms
• Clarify nature of connection to average case
  complexity.
• Resolve UGC (recently, disproof of UGC when the
  constraint graph is an expander.AKKSTV08
• SDP as a proof technique---apply to open problems
  of circuit complexity, communication complexity
  etc.

Looking forward to many developments THANK YOU!
30
SDPs and MW Updates Primal-dual algorithm
Known MW Update rule --gt Approx. solutions to
LPs PST91, Y95, GK97,..etc.
experts lt-gt constraintspayoffs lt-gt slack in
constraint
AK07 Matrix MW update rule that uses formal
analogybetween psd matrices and nonnegative real
s.
Golden-Thompson
(Spl. Case LPs SDPs with 0s on
offdiagonals)
Other ingredients flow computations,
eigenvalues, dimension reduction tricks, etc.
31
Embeddings and Cuts
ThmLLR94, AR94 Integrality gap for SDP for
Nonuniform Sparsest Cut Min distortion of any
embedding of into
Rounding algorithm of ARV04 gives insight into
structureof basis of new embeddings
Hardness results for sparsest cut yielded
insights at the heart of the embedding
impossibility results.