?????????????? Physical Fluctuomatics ?9? ????? 9th Belief propagation

1
??????????????Physical Fluctuomatics?9?
?????9th Belief propagation

• ???? ?????????? ????????
• ?? ??(Kazuyuki Tanaka)
• kazu_at_smapip.is.tohoku.ac.jp
• http//www.smapip.is.tohoku.ac.jp/kazu/

2
???????????
?????????????????????,?8?,????,2006.
????
J. Pearl Probabilistic Reasoning in Intelligent
Systems Networks of Plausible Inference, Morgan
Kaufmann, 1988. ???, ????, ???, ????, ????
???????????/???? I ---??????????, ????, 2003.
3
????????????

• 2L ????????????

????????? L10??????1???????? L20???17?, L30???1
2?, L40???34????.
???????????????????????.
L ????

• ?????????????
• ?????

??
??
4
???????????
???????????
??????
?????
???????
????? (Belief Propagation)
J. Pearl Probabilistic Reasoning in Intelligent
Systems Networks of Plausible Inference (Morgan
Kaufmann, 1988). C. Berrou and A. Glavieux Near
optimum error correcting coding and decoding
Turbo-codes, IEEE Trans. Comm., 44 (1996).
5
?????????

• ??????????????????
• Y. Kabashima and D. Saad, Belief propagation vs.
  TAP for decoding
• corrupted messages, Europhys. Lett. 44 (1998).
• M. Opper and D. Saad (eds), Advanced Mean Field
  Methods
• ---Theory and Practice (MIT Press, 2001).
• ??????????????
• S. Yedidia, W. T. Freeman and Y. Weiss
  Constructing free-energy
• approximations and generalized belief propagation
  algorithms,
• IEEE Transactions on Information Theory, 51
  (2005).
• ?????????????
• S. Ikeda, T. Tanaka and S. Amari Stochastic
  reasoning, free energy,
• and information geometry, Neural Computation, 16
  (2004).

6
?????????????????

• ???????????
• J. S. Yedidia, W. T. Freeman and Y. Weiss
  Constructing free-energy approximations and
  generalized belief propagation algorithms, IEEE
  Transactions on Information Theory, 51 (2005).
• ???????????????????????????
• R. Kikuchi A theory of cooperative phenomena,
  Phys. Rev., 81 (1951).
• T. Morita Cluster variation method of
  cooperative phenomena and its generalization I,
  J. Phys. Soc. Jpn, 12 (1957).

7
????????????????
??????????????????????????????????.
??????????????????????????????????????????????????
????????????(Yedidia, Weiss and Freeman,
NIPS2000).
?????
????? (???) ??????????
???????? (????)
???????????
8
????????????????

• Image Processing
• K. Tanaka Statistical-mechanical approach to
  image processing (Topical Review), J. Phys. A, 35
  (2002).
• A. S. Willsky Multiresolution Markov Models for
  Signal and Image Processing, Proceedings of IEEE,
  90 (2002).
• Low Density Parity Check Codes
• Y. Kabashima and D. Saad Statistical mechanics
  of low-density parity-check codes (Topical
  Review), J. Phys. A, 37 (2004).
• S. Ikeda, T. Tanaka and S. Amari Information
  geometry of turbo and low-density parity-check
  codes, IEEE Transactions on Information Theory,
  50 (2004).
• CDMA Multiuser Detection Algorithm
• Y. Kabashima A CDMA multiuser detection
  algorithm on the basis of belief propagation, J.
  Phys. A, 36 (2003).
• T. Tanaka and M. Okada Approximate Belief
  propagation, density evolution, and statistical
  neurodynamics for CDMA multiuser detection, IEEE
  Transactions on Information Theory, 51 (2005).
• Satisfability Problem
• O. C. Martin, R. Monasson, R. Zecchina
  Statistical mechanics methods and phase
  transitions in optimization problems, Theoretical
  Computer Science, 265 (2001).
• M. Mezard, G. Parisi, R. Zecchina Analytic and
  algorithmic solution of random satisfability
  problems, Science, 297 (2002).

9
?????????????????????
????????????????????????.

• ????????????
• ?????????????????????? ?????????????????????.
• ? ????????????????
• ?????????

10
????? (Belief Propagation)

• ???????????????

?????????????????.
x

• ???????????????

?????????????????.
x
11
????? (Belief Propagation)
???????????????
12
????? (Belief Propagation)
???????????????
13
???????????????
???1?2???????? ????????????????????????????????.
14
????? (Belief Propagation)
???????????????
???1???????2???????????(???2???)???1???????????1??
??????????????????.
??????????????? (Message Passing Rule)
15
??????????(1)

• 1???

16
??????????(2)
??????

• ???

17
???????????????
??????????!!
??????2?????
18
?????
???????????????
B??????????
19
???????????????
???????????????????????????????????????. ???,?????
????????????????
3
1
4
5
???????????????
20
??????????
??????
??????????????????,?????????????????.
???
21
????????????(1)
Kullback-Leibler Divergence
Free Energy
22
????????????(2)
Free Energy
KL Divergence
23
????????????(3)
Free Energy
KL Divergence
Bethe Free Energy
24
????????????(4)
25
????????????(5)
Lagrange Multipliers to ensure the constraints
26
????????????(6)
Extremum Condition
27
????????????(7)
Extremum Condition
28
????????????(8)
Message Update Rule
29
????????????(9)
Message Passing Rule of Belief Propagation

????????????????????????.
30
???????????????

1. ?????(?????)
2. ?????
3. ????????(????)
4. ???(L. Onsager)

31
??????

• ?????
• ?????
• ?????????????
• ???

?? ?11? ???????????????? ?12? ?????????????
---????????
32
????10-1
???? a, b, c, d, x, y ????? P(a,b,c,d,x,y) ????.
???? x, y ??????? PXY(x,y) ????????????????.
33
????10-2
??????????2????????