Belief%20Propagation%20Revisited

1
Belief Propagation Revisited

• Adnan Darwiche

2
Graphical Models
Battery Age
Alternator
Fan Belt
Charge Delivered
Battery
Fuel Pump
Fuel Line
Starter
Distributor
Gas
Battery Power
Spark Plugs
Gas Gauge
Engine Start
Lights
Engine Turn Over
Radio
3
Graphical Models
Battery Age
Alternator
Fan Belt
Charge Delivered
Battery
Fuel Pump
Fuel Line
Starter
Distributor
Gas
Battery Power
Spark Plugs
Gas Gauge
Engine Start
Lights
Engine Turn Over
Radio
4
Graphical Models
Battery Age
Alternator
Fan Belt
Charge Delivered
Battery
Fuel Pump
Fuel Line
Starter
Distributor
Gas
Battery Power
Spark Plugs
Gas Gauge
Engine Start
Lights
Engine Turn Over
Radio
5
Probabilistic Reasoning in

• Diagnosis
• Planning
• Learning
• Channel coding
• Vision
• Speech recognition
• Language comprehension
• Bioinformatics

6
Treewidth w
7
Smaller Shift ? Further Object
Larger Shift ? Closer Object
8
Bayesian Network
Reasoning inBayesian NetworkEstimates Depth
Images Define aBayesian Network
9
Belief Propagation
10
Belief Propagation
11
Belief PropagationWhat if there are loops?
?
?
12
Belief PropagationWhat if there are loops?
13
Loopy Belief Propagation
p.235
14
The Merit of Loopy Belief Propagation

• Revolutionary error correcting codes
• Turbo Codes (BerrouGlavieux 1993)
• LDPC Codes (MacKayNeal 1995, Gallager 1962)
• Can closely reach the theoretical limit of
  communications in noisy channels
• Turbo LDPC decoders Loopy BP in BNs!
  (McEliece, MacKay Cheng 1998)

15
Stereo Vision
Two Images
Depth Map
16
Stereo Vision
http//vision.middlebury.edu/stereo/eval/
17
Edge Deletion Semantics(Joint work with Arthur
Choi)
Energy-Based Semantics (Statistical Physics)
18
The Idea
19
The Idea
A
B
C
D
20
The Idea
A
B
Y
X
C
D
New Edge Parameters for each Query
21
Specifying the Approximation

• How do we parametrize edges?
• Quality of approximation
• Which edges do we delete?
• Quality of approximation
• Computational complexity

22
Parametrizing Edges ED-BP
U
s'
U'
X
23
Parametrizing Edges Iteratively ED-BP
Iteration t 0Initialization
24
Parametrizing Edges Iteratively ED-BP
Iteration t 1
25
Parametrizing Edges Iteratively ED-BP
Iteration t 2
26
Parametrizing Edges Iteratively ED-BP
Iteration tcConvergence
27
Belief Propagation as Edge Deletion
Iteration t
Iteration t
28
(No Transcript)
29
(No Transcript)
30
(No Transcript)
31
Which Edges To Delete?
U
X
32
Which Edges To Delete?
U
s'
U'
X
33
ED-BP Improving on the Quality of IBP
BP
Exact Inference
34
ED-BP Improving on the Quality of IBP
BP
Exact Inference
35
ED-BP Potentially Bad Approximations
Unimproved, but costly,approximation,
BP
Exact Inference
36
ED-BP Improving on the Convergence Rate
37
ED-BP Improving on Running Time
38
Edge Deletion in Undirected Models
Original Network
Approximate Network
39
Correcting the Partition Function I
Theorem If MI(Xi,Xj) 0 in ED-BP network M',
then where
i
j
40
Deleting Many Edges
This will yield the Bethe free energy
approximation!
41
Correcting the Partition Function II
Theorem For an ED-BP network M', we have where
i
j
42
Which Edges Do We Recover?EC2?
Recover edges with largest ?kl MI(Xi,XjXkXl)!
i
j
43
Experiment Random Grid
Bethe
exact
yfq
44
Beyond Treewidth

• Exact Inference Exploit Non-structural
  Independence
• Approximate inferenceExact inference on an
  approximate network obtained by deleting edges

45
What next?

• Constant factors!
• Guarantees/bounds on approximations
• Edge recovery heuristics getting the most out of
  the extra time
• Controlling tradeoff between quality complexity
• Dynamic models
• Logical reasoning Survey propagation

46
http//reasoning.cs.ucla.edu
47
(No Transcript)