Approximate Inference Using Planar Graph Decomposition

1
Approximate Inference Using Planar Graph
Decomposition

• Amir Globerson and Tommi Jaakkola
• In Advances in Neural Information Processing
  Systems 19, 2006

Presented by Jonathan Huang at SELECT Lab,
12/12/06
2
The Ising Model

• Let G(V,E) be a graph
• The Ising Model of statistical physics is a
  distribution on binary variables x1,x2,,xN which
  takes the form
• xi ? -1,1
• Z(?) is the partition function which normalizes
  the distribution.

Field Strengths
Coupling Strengths
3
The Ising Model

• For example, if G(V,E) is the following graph
• Then

x1
x2
x3
x4
4
Zero External Field

• If there are only interaction potentials, we will
  say that there is no external field
• (?i0)

5
The Ising Model

• In general, the partition function Z(?) is
  intractable to compute because it involves
  summing over all joint settings of x1,x2,,xN
• This paper presents a way to approximately
  compute the partition function

6
Outline

• Some Graph Theory Definitions
• Exact Solution for Planar Ising models with zero
  external field
• Partition Function bounds via the Planar
  Decomposition
• Bound Optimization
• Compare to Tree Reweighting approximations

7
Planar Graphs

• A planar graph is a graph which can be drawn on a
  plane such that no edges intersect

Planar
Non-Planar
8
The Dual Graph

• Given a planar graph G, there exists a dual graph
  G which has a vertex for each face of G and an
  edge for each edge in G joining two neighboring
  faces

G
(G shown in red)
9
Plane Triangulations

• A plane triangulation of a planar graph G is
  obtained by adding edges until all the faces of
  the resulting graph are bounded by triangles

10
Plane Triangulation Dual

• The vertices of the dual of a plane triangulated
  graph are all of degree 3

11
Perfect Matchings

• A perfect matching is a subset of edges F? E such
  that every vertex in G has exactly one edge in F
  incident on it
• If the edges are weighted, then the weight of the
  matching is defined to be the product of the
  weights of the edges in the matching

12
Outline

• Some Graph Theory Definitions
• Exact Solution for Planar Ising models with zero
  external field
• Partition Function bounds via the Planar
  Decomposition
• Bound Optimization
• Compare to Tree Reweighting approximations

13
Exact Calculation of the Partition Function

• Let G be the planar graph associated with the
  given Ising model
• We assume zero external field
• Strategy
• Convert G to a new weighted graph GPM for which
  Z(?) is equal to the sum over all perfect
  matchings
• Sum over all perfect matchings

14
Agreement Edge Sets

• A set of edges E in a triangulated graph G is an
  agreement edge set (AES) if for every triangle
  face F in G, either
• The edges in F are all in E
• Or, exactly one of the edges in F is in E

or
15
Agreement Edge Sets (cont.)

• There is a correspondence between pairs of
  assignments (x,-x) and Agreement Edge Sets
• Map an assignment x to the set of edges such that
  xixj

16
Agreement Edge Sets (cont.)

• The contribution of a given assignment x is
• If x corresponds to the AES E, then its
  contribution can be written as
• (the point of all this is that a sum over all
  assignments is equivalent to a sum over all AESs)

17
Converting G to GPm

• Step 0 Start with a Planar Graph G

18
Converting G to GPm

• Step 1 Triangulate G to get GTri
• Set new edge weights to be 1
• This defines a new Ising model with the same
  normalization
• Wed like to sum over the Agreement Edge Sets of
  GTri because

19
Converting G to GPm

• Step 2 Dualize GTri and replace each vertex with
  three vertices that are connected to each other
  and each of the three is connected to one of the
  neighbors of the original vertex
• Call this new graph GPM

20
The Correspondence

• Claim Every Perfect Matching in GPM corresponds
  to an agreement edge set in GTri
• This shows that a sum over agreement edge sets is
  equivalent to a sum over perfect matchings

21
The Correspondence (cont.)

• (An illustration)

A triangle from GTri
Its dual vertex and edges in GPM
22
Exact Calculation of the Partition Function

• Let G be the planar graph associated with the
  given Ising model
• We assume zero external field
• Strategy
• Convert G to a new weighted graph GPM for which
  Z(?) is equal to the sum over all perfect
  matchings
• Sum over all perfect matchings

23
Summing over all Perfect Matchings

• This is P-complete for general graphs
• If all the weights are 1, this is the matrix
  permanent problem
• But for planar graphs (like GPM), it can be
  computed in poly-time using Pfaffians

24
What are Pfaffians??

• Let A be a skew-symmetric matrix
• Theorem The determinant of A can be written as
  the square of a polynomial in the matrix entries
  of A
• Due to Sir Arthur Cayley (not Pfaff)
• Define the Pfaffian of A to be

25
Pfaffians and Perfect Matchings

• Let H be a planar graph with weights wij
• Direct the edges of H such that for each face,
  the number of clockwise-oriented edges on its
  perimeter is odd (this is called a Pfaffian
  Orientation)
• Define a skew-symmetric matrix P(H)

1
1
1
1
1
1
1
1
1
1
1
1
26
Pfaffians and Perfect Matchings

• Theorem Pf(P(H)) is the sum over all weighted
  matchings on H
• The partition function of a planar binary graph
  with pure interaction potentials can be computed
  in polynomial time!

27
Outline

• Some Graph Theory Definitions
• Exact Solution for Planar Ising models with zero
  external field
• Partition Function bounds via the Planar
  Decomposition
• Bound Optimization
• Compare to Tree Reweighting approximations

28
Partition Function bounds via the Planar
Decomposition

• Now consider a non-planar G over binary variables
  with zero external field
• Let G(r) be a set of spanning planar subgraphs of
  G
• Let ?(r) be a set of potentials on the edges of
  G(r)
• Extend ?(r) to be a set of potentials on the
  edges of G by setting potentials to be zero on
  edges not in G(r)
• Z(?(r)) is the partition function on G(r) with
  respect to the parameters ?(r)

29
Partition Function bounds via the Planar
Decomposition

• Now given any distribution ?(r) on the Gr,
• If
• Then by convexity of the log-partition function

30
Outline

• Some Graph Theory Definitions
• Exact Solution for Planar Ising models with zero
  external field
• Partition Function bounds via the Planar
  Decomposition
• Bound Optimization
• Compare to Tree Reweighting approximations

31
Bound Optimization

• The goal is to make this bound as tight as
  possible

32
Bound Optimization

• Since the objective function is convex, the
  optimization is guaranteed to converge to
  globally optimal mixture coefficients and
  parameters for each spanning planar subgraph

33
Marginal Optimality Criterion

• Let G(r), G(s) be two planar subgraphs that both
  contain the edge (i,j).
• At the optimal parameter vector, they must agree
  on the marginal of (i,j)
• So getting at marginal edge probabilities is
  easy
• Just use marginals from one of the planar
  subgraphs

34
Outline

• Some Graph Theory Definitions
• Exact Solution for Planar Ising models with zero
  external field
• Partition Function bounds via the Planar
  Decomposition
• Bound Optimization
• Compare to Tree Reweighting approximations

35
External Fields

• Suppose G is planar, but now consider a nonzero
  external field
• Can rewrite the model in terms of pure
  interaction potentials by adding an additional
  node connected to all original variables
• This break planarity /

(extra edges not drawn here)
36
Experimental Evaluation

• Compared performance of planar decomposition to
  tree reweighting on a 7x7 square lattice

Red Tree Reweighting results, Blue Planar
Decomposition Results
37
Conclusions

• Pros
• Better partition functions bounds than Tree
  Reweighted BP in many cases (except for singleton
  marginals)
• Cons
• Only defined for binary MRFs
• The optimization is carried out explicitly over a
  set of planar subgraphs here, but in TRW, the
  optimization is implicitly over an exponential
  number of spanning trees
• Also, the paper doesnt compare to other
  approximate inference algorithms

38
Thanks