Survey Propagation

1
Survey Propagation
2
Outline

• Survey Propagation an algorithm for
  satisfiability1
• Warning Propagation
• Belief Propagation
• Survey Propagation
• Survey Propagation Revisited2
• SPs relation to BP
• Covers

1 Braunstein, A., Mézard, M., and Zecchina, R.
2005. Survey propagation An algorithm for
satisfiability. Random Struct. Algorithms 27, 2
(Sep. 2005), 201-226.
2 Kroc, L. Sabharwal, A. and Selman, B. 2007.
Survey propagation revisited. In UAl 2007.
3
Survey Propagation and Message Passing

• New threshold defining two regions of solutions,
  easy SAT and hard SAT
• Easy SAT region has many, highly-clustered
  solutions, and any two solutions are connected by
  a few number of steps
• Hard SAT region has many clusters of solutions,
  with fewer overall solutions than easy SAT
• SP is an efficient way of computing solutions in
  the Hard SAT region

4
New Model - Factor Graph
5
Message Passing

• Treat the Factor Graph as a network
• Depending on the current state of the network,
  messages will be passed between the nodes
• These messages will later be used to determine
  the final value of a variable

6
Cavity Fields

• Used to determine the value of a given message,
  and varies depending on the actual algorithm in
  use
• Calculated by removing a function node from
  consideration

7
Warning Propagation

• Messages are from the set 0,1
• Passed from Function nodes to Variable Nodes
• Product of cavity fields
• Initially randomly generate messages
• Calculate new warnings for edges until we finish
  the time, or we hit a fixed point for all edges
• New warnings are calculated from the cavity fields

(?b ? V(j)\a ub?j) (?b ? V-(j)\a ub?j)
8
(No Transcript)
9
WP Example

• Local Field
• Contradiction Numbers

The whole algorithm is an iterative process
involving multiple WP executions, simplifying the
graph using local fields
10
Belief Propagation

• Finds the total number of solutions, as well as
  the fraction of assignments with variable Xi is
  true
• Two types of messages
• From Function node to Variable node
• Given a value for Xi, the probability that it
  satisfies the function clause
• From Variable node to Function node
• Probability that variable i takes value Xi in the
  absence of function clause a

11
Satisfiable and Unsatisfiable Sets
12
(No Transcript)
13
(No Transcript)
14
BP Example
Blue is the cavity field of the variable node,
red is the message from function to variable.
Feed these values into another function to
compute probabilities of being true
15
WP and BP

• Work 100 of the time when the factor graph is a
  tree
• Can be used as heuristics when it is not a tree,
  but no guarantee on solution
• Goal Create a more efficient scheme for the
  general case

16
Basics of Survey Propagation

• Generally the same as BP, but deals better with
  clusters
• Details of the message change
• A message sent from function to variable is the
  probability that a warning is sent to the
  variable node
• A warning is sent if all other variable nodes
  connected to the function do not satisfy the
  function.
• Note Behaves exactly like WP on tree factor
  graphs

17
Clustering of Configurations
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
SAT Assignments
.
.
.
.
The problem with BPs approximations is that they
do not hold globally, but do hold if you restrict
the computation to a specific cluster. SP fixes
this.
18
(No Transcript)
19
(No Transcript)
20
SP Results with Dont Care State
21
Testing Data
22
Results

• New way to solve SAT using cavity fields and
  message passing
• Works well on generic input, as well as the tree
  structure
• Has not been rigorously examined, but is based
  off of similar work

23
Why does SP Work?

• As of 2008, SP is the only method of solving SAT
  problems with 1,000,000 variables (and larger)
• Does so in near linear time (even in the hardest
  region)
• SP behaves like backtrack searches, except it
  almost never has to backtrack

24
SPs relation to BP

• BP works well when the number of SAT clusters is
  low
• SP seems to work well ALL the time
• Work in 2005 shows that SP is equivalent to BP
  search over a new object, covers
• This is a generalization of the clusters
  described previously

25
Covers
(x V y V z)?(x V y V z) ?(x V y V z)

• s ? 0,1,n is a cover of F if
• Every clause has at least one satisfying literal,
  or two -ed literals
• No unsupported variables are assigned 0 or 1

• X
• Y
• Z

26
Computing Covers

• New Factor Graphs

27
Solving SAT using Covers

• Let P(F) be a cover of F
• The factor graph of P(F) can bee seen as a
  Bayesian Network, so marginals can be computed
  using Bayes Theorem
• These marginals, when fed to the BP algorithm,
  computes a solution for the original SAT problem
  (and results in the SP algorithm).
• The SP algorithm has been shown to compute
  marginals over these covers.

28
SP, BP and Marginals
Magnetization of a Variable The difference
between the marginal being positive vs. negative
29
Results

• The reason why SP works so well seems to be
  related to these covers
• These covers seem to expose hidden properties of
  a SAT instance, such as backbone variables

30
Interesting Aside

• SAT and Decision SAT are self-reducible
• Finding covers is NOT self-reducible

(x V y V z)?(x V y V z) ?(x V y V z)
x 1? Oracle yes
(y V z) ?(y V z)
y 0? Oracle yes
(z)
(1,0,0) a cover of the original? No!