Dynamics and its stability of Boltzmann-machine learning algorithm for gray scale image restoration

1
Dynamics and its stability of Boltzmann-machine
learning algorithm for gray scale image
restoration

• J. Inoue (Hokkaido Univ.) and K. Tanaka (Tohoku
  Univ.)

The 3rd International Symposium on Slow Dynamics
in Complex Systems in Sendai November 2003
2
Plan of this talk

• Bayesian image restoration and hyper-parameter
  estimation
• Boltzmann-machine learning algorithm for the
  hyper-parameter estimation
• Dynamic behavior of the BML algorithm
• Stability of the solution
• Concluding remarks

3
Bayesian image restoration
Original
Corrupted
We treat images and the degrading process as spin
systems
4
Definitions of the model by spin systems
Original
Hyper-parameters (true value)
Corrupted
5
Bayesian approach and MPM estimation
takes its minimum at
Inoue and Carlucci (2001)
6
Maximization of the marginal likelihood
via Boltzmann-machine learning algorithm
takes its maximum at
on average
Inoue and Tanaka (2003)
We evaluate the data-averaged BML algorithm at
the mean-field level
7
Dynamic behavior of the hyper-parameters
are integrated numerically
8
Analysis of the stability
Expand the BML equations around
and check the sign of eigenvalues of the Hessian
A
The solution
is asymptotically stable
9
True hyper-parameter dependence
of the stability
(fixed)
(fixed)
The solution of the BML algorithm is
asymptotically stable as long as the solution is
identical to the true value of the
hyper-parameters
10
Behavior of the BML algorithm
around the solution
Trajectories in the hyper-parameter space
(around the solution)
11
Concluding remarks

• We investigated dynamic behavior and its
  stability of the BML algorithm for gray scale
  image restoration
• We derived the data-averaged BML equations
• The solution is asymptotically stable as long as
  the solution is identical to the true value of
  the hyper-parameters
• More details of the present study are available at

http//chaosweb.complex.eng.hokudai.ac.jp/j_inoue
/
Send email j_inoue_at_complex.eng.hokudai.ac.jp