Gibbs measures on trees

1
Gibbs measures on trees
Elchanan Mossel, U.C. Berkeley
mossel_at_stat.berkeley.edu, http//www.cs.berkel
ey.edu/mossel/
2
Lecture Plan

• Gibbs Measures on Trees
• Uniqueness
• Reconstruction
• Mixing times on trees
• Building Trees (Phylogeny)
• Some analytical problems.
• Gibbs Measures on Trees and Other Graphs
• Uniqueness
• Mixing Times.
• Belief Propagation.
• The Replica Method.

3
Gibbs Measures

• A Gibbs Measure on a (finite) graph G(V,E) is
  given by
• Node potentials (?v v 2 V) and
• Edge Potentials (?e e 2 E)
• The probability of ? (?(v) v 2 V) 2 AV is
  given by

• P? Z-1
• ?v 2 V ?v?(v)
• ?e(v,u) 2 E?e?(v),?(u)

G

• Gibbs measures introduced in Statistical Physics.
• Essential in Machine Learning.
• Also known as MRFs, Graphical Models etc.

4
Uniqueness and Reconstruction

• Let ?(v,L) (?(w) d(v,w) L).
• Let ?(v,L)(a) P ?(v) a ?(v,L) P?(v)
  a
• Let Gn be a family of Gibbs measures
• Uniqueness limL ! 1 sup ?(v,L)1 v
  2 Gn 0
• Reconstruction limL ! 1 sup ?(v,L)2 v
  2 Gn ? 0
• Informally
• Uniqueness 8 values of ?(v,L gtgt 1), ?(v) has
  same dist.
• Reocn. ?(v) is typically independent
  of ?(v,L gtgt 1)

?(v,L)
?(v)
L
G
5
Gibbs measures on trees

• On a finite tree, a Gibbs
• measure P can be written as

• Using recursions easy to calculate P?(v) .
  ?(v,L)
• ) Easy to determine uniqueness when extreme
  ?(v,L) are known (Ising, Potts, Independent sets
  )
• Open Problem 1 Given the d-ary tree and a
  general M, determine uniqueness.
• Open Problem 2 Convex asymptotic geometry of
• P?(v) . ?(v,L) as L ! 1

P? ??(0) ?e u ! v Me?(u),?(v)

0


-



-




-
-

• Assume Me are identical.
• Tree is d-ary tree.

6
Gibbs measures on trees a story

• Let Mi,j Phair(daughter) j hair(mother)
  i
• Suppose we know the tree T of all mothers going
  back to Eve.
• Uniqueness Is there any assignment of hair
  color to current population that will yield
  information on Eves?
• Reconstruction Do we expect to have
  information on Eves hair color from current
  population?

7
Reconstruction Recursive Reconstruction
Binary symmetric channel (BSC) Ising model
(no external field)

• T 3-ary regular tree with Me M for all edges.
• Consider the recursive majority function.

• Let pn P n-fold rec-maj(?(0,n)) ?(0) .
• Let ?(p) (1-?) p ? (1-p) and g(p)
  ?3(p)3?2(p)(1-?(p))
• p0 1 and pn1 g(pn) ) pn ! ½ if and only if
  (1-2?) gt 2/3.
• ) Reconstruction if ? lt 1/6.
• Von-Neumann (56) for reliable noisy-computation.
• Later Evans-Schulmann93, Steel94, Mossel98.

8
Spectral Reconstruction

• Let M be the Ising (BSC) model on a b-ary tree T.
• Let f(?n) Maj(?n) sign(??(v) v 2 Ln).
• Theorem (Higuchi 77)
• limn P?0 f(?n) gt ½ if b(1-2?)2 gt 1.
• ) Reconstruction for ternary tree if ? lt ½ -
  (1/3)1/2.
• Let M be any chain and T the b-ary tree
• Let ? be the 2nd eigenvalue of M in absolute
  value.
• ClaimKesten-Stigum66 b ? 2 gt 1 )
  Reconstruction.
• b ? 2 1 is also threshold for census
  MosselPeres04 and robust Janson-Mossel04
  reconstruction.

9
Non Reconstruction - Coupling

• Copying rule. For i ,-
• Pi ! i ? 1 2 ?
• Pi ! Uniform 1? 2 ?
• Continuing down the tree, non-coupled elements
  form a branching process with parameter ?.

/ -
/ -

/ -

• If 2 ? 1, branching process dies ) coupling.
• ) for ? ¼ no reconstruction (this is not
  tight!)
• The threshold for reconstruction is only for
• Ising (BSC) model is given by 2?2 1.

10
Ising Model on Binary Trees
low
interm.
high
bias
bias
no bias
boundary
boundary
no bias
bias
typical boundary
typical boundary
2 ? gt 1 2?2 lt 1
2 ?2 gt 1
2 ? lt 1
Unique Gibbs measure
The transition at 2 ?2 1 was proved
by Bleher-Ruiz-Zagrebnov95,Evans-Kenyon-Peres-Sch
ulman2000,Ioffe99, Kenyon-Mossel-Peres-2001,Martin
elli-Sinclair-Weitz2004.
11
Reconstruction for Markov models

• So the threshold b ? 2 1 is important.
• But M-2000 it is not the threshold for
  extemality
• Not even for 2 2 markov chains.
• Open What is the threshold for
  q3 Potts on binary tree?
• Very RecentBorgs-Chayes-M-Roch b ? 2 1 for
  slightly asymmetric channels.

12
Lecture Plan

• Gibbs Measures on Trees
• Uniqueness
• Reconstruction
• Mixing times on trees
• Building Trees (Phylogeny)
• Some analytical problems.
• Gibbs Measures on Trees and Other Graphs
• Uniqueness
• Mixing Times.
• Belief Propagation.
• The Replica Method.

13
Glauber dynamics sampling Gibbs measures

• Consider the following dynamics on configuration
  ? of Gibbs measure G.
• At rate 1
• Pick a vertex v uniformly at random, and update
  s(v) according to the conditional probability
  given s(w) w v.
• Easy Converges to
• Gibbs distribution.
• Hard How quickly?
• Measure convergence in terms of Markov Operator.

G
14
Ising Model on Binary Trees
low
interm.
high
bias
bias
no bias
no bias
bias
typical boundary
typical boundary
2 ? gt 1 2?2 lt 1
2 ?2 gt 1
2 ? lt 1
Unique Gibbs measure
?2 ?(n1 2 log2 ? ) Reconstruction
No-Reconstruction, ?2 O(1)
In Berger-Kenyon-M-Peres05
15
Relaxation time for the binary tree

• On Trees Fast mixing ? No-Reconstruction.
• Vs. Common wisdom Fast mixing ? Uniqueness.
• Martinelli-Sinclair-Weitz05
• Log-Sob behaves in the same way as Spectral-Gap.
• Study external-fields and boundary conditions

16
Lecture Plan

• Gibbs Measures on Trees
• Uniqueness
• Reconstruction
• Mixing times on trees
• Building Trees (Phylogeny)
• Some analytical problems.
• Gibbs Measures on Trees and Other Graphs
• Uniqueness
• Mixing Times.
• Belief Propagation.
• The Replica Method.

17
Phylogeny

• Phylogeny is the true evolutionary relationships
  between groups of living things

Noah
Shem
Ham
Japheth
Cush
Mizraim
Kannan
18
Pyhlogenetic Inference

• In phylogenetic inference
• The tree is unknown.
• Given a sequence of collections of random
  variables at the leaves (species).
• Collections are i.i.d.
• Want to reconstruct the tree (un-rooted).

19
Pyhlogenetic Reconstruction
20
Markov Model of Evolution
001100011101000011000100

• Simplest evolution model binary symmetric
  channel
• CFN Model
• Tree T (V,E)
• Node states
• Mutation probabilities

0
s(r)
pra
prc
0
s(a)
pab
pa3
1
0
s(c)
s(b)
pc4
pc5
pb1
pb2
1
0
0
0
1
0 Purines (A,G) 1 Pyrimidines (C,T)
s(1)
s(4)
s(5)
s(3)
s(2)
21
Phylogenetic Inference Problem

• Inference
• Given i.i.d. samples at the leaves
• Task Reconstruct the model, i.e. find tree and
  do so efficiently
• Efficiency
• 1) Computational Running time of reconstruction
  algorithm
• 2) Information-theoretic Sequence length
  required for successful reconstruction
• Let n leaves (species)
• k length of sequences needed.

s(1)
s(4)
s(5)
s(3)
s(2)
1
1
1
0
0
0
1
1
0
0
1
0
0
1
1
0
1
1
1
0
1
1
1
0
0
pb2
prc

pra
pa3
pc5
22
Phylogeny Conjectures and Results
Reconstruction
Phylogeny
Reconstruction
k O(log n)
conj
No Reconstruction
k poly(n)
conj
Percolation
Random Cluster
critical ? 1/2
MS03
Ising model
CFN
critical ? 2?2 1
Mo04 DMR05
23
Polynomial Lower Bound at High Mutations

• Proof

24
Logarithmic Reconstruction

• Th2 M 2004 If T is an tree on n leaves s.t.
• For all e, ?min lt ?(e)lt ?max and 2?2min gt 1, ?max
  lt 1.
• Then k O(log n log ?) characters suffice to
  infer the topology with probability 1-? .
• Caveat Need a balanced tree all leaves at the
  same distance from a root.
• Th3 Daskalakis-M-Roch 2005 Above result holds
  for general trees.
• Cameron,Hill,Rao 2006 Experimental
  performence.

25
Balanced Trees

• Two-Step Algorithm M 2004
• 1) Reconstruct one (or a few) level(s) using
  distance estimation.
• 2) Infer sequences at roots using recursive
  majority.
• 3) Start over

26
General Trees Daskalakis, M, Roch, 2005
27
Lecture Plan

• Gibbs Measures on Trees
• Uniqueness
• Reconstruction
• Mixing times on trees
• Building Trees (Phylogeny)
• Some analytical problems.
• Gibbs Measures on Trees and Other Graphs
• Uniqueness
• Mixing Times.
• Belief Propagation.
• The Replica Method.

28
Main analytical problems

• How to analyze recursions of the random measures
  ?(?,L)?
• No general techniques are known (some easy
  methods follow).
• Needed for
• General boundary conditions
• Worst case (uniqueness)
• Average case (Reconstruction)
• Other.
• non-regular trees (strong spatial mixing) and for
  
• families of random trees (optimal error
  correcting codes).

29
Lecture Plan

• Gibbs Measures on Trees
• Uniqueness
• Reconstrution
• Mixing times on trees
• Building Trees (Phylogeny)
• Some analytical problems.
• Gibbs Measures on Trees and Other Graphs
• Uniqueness
• Mixing Times.
• Belief Propagation.
• The Replica Method.

30
Conjecture Uniqueness on tree / graphs

• Consider Gibbs measures where
• All edge potentials are identical ?e ? for all
  e
• All node potentials are trivial ?v 1 for
  all v.
• Graph is regular of degree d.
• Conjecture
• Gibbs measure unique on d-regular tree )
• Gibbs measure unique on any family of d regular
  graphs.
• Recently proved by Weitz for anti-ferromagnet
  Ising models.

• Trivial for random graphs.

G
T
31
Conjecture Uniqueness on tree / graphs

• Very Recently M-Weitz-Wormald-06
• For the hard-core model
• Non-uniqueness of Gibbs measure on 3-regular tree
  
• )
• Exp. Slow mixing on random 3-regular graphs.
• Reconstruction on random 3-regular graphs.
• Moral Slow/Rapid mixing on typical graphs is
  determined by uniqueness on trees.
• Still dont really know how to prove for
• 4-regular graphs
• Other models.

32
Lecture Plan

• Gibbs Measures on Trees
• Uniqueness
• Reconstruction
• Mixing times on trees
• Building Trees (Phylogeny)
• Some analytical problems.
• Gibbs Measures on Trees and Other Graphs
• Uniqueness
• Mixing Times.
• Belief Propagation.
• The Replica Method.

33
Belief Propagation in AI

• Belief Propagation (BP) is a popular method in
  AI/Coding for estimating marginal probabilities
  P?(0) a for a Gibbs measure G.
• It is equivalent TatikondaJordan02 to
  calculating marginal probabilities P?(0)
  a on the computation tree,T(G).
• In particular, uniqueness on infinite computation
  tree ) convergence of BP.
• Uniqueness High girth )
• Convergence to correct marginals
• Open problem Is uniqueness needed?
• Why BP works also when girth is small?

G
T
34
Belief Propagation in Coding

• In coding
• BP is used to decode Low Density Parity Check
  Codes Gallager62
• Proved to be efficient without uniquenessLMSS,R
  SU
• Recursive Analysis up to girth of graph.
• Open Problem Is BP efficient beyond girth?
• Open Problem Can LDPC codes achieve Channel
  Capacity?

35
Replica Symmetry Breaking in Physics

• In Physics
• Replicas are recursive distributional equations
  used to calculate probabilities for spinglasses
  (random codes, random SAT problems).
• Symmetric Replicas ? Belief Propogation.
• Symmetry Breaking Replicas ? Survey
  Propogation.
• MezardMontanari06 Claim Symmetry Breaks
  exactly when reconstruction emerges.
• Open problem/Conjecture Is the reconstruction
  threshold on d-ary tree the right threshold for
  spin-glasses on random d-regular graphs?

36
Lecture Plan

• Gibbs Measures on Trees
• Uniqueness
• Reconstruction
• Mixing times on trees
• Building Trees (Phylogeny)
• Some analytical problems.
• Gibbs Measures on Trees and Other Graphs
• Uniqueness
• Mixing Times.
• Belief Propagation.
• The Replica Method.

37
(No Transcript)
38
A reminder Markov Chains

• A Markov Chain on a (finite) set S is given by an
  initial distribution ? and transition
  probabilities ? ti,j.
• The probability of (?(t))t0T 2 AT1 is given by
• ??(0) ?t0T-1 ? t?(t),?(t1)

?
?1
?2
?0
time
3
0
1
2