Applications of Cellular Neural/Nonlinear Networks in Physics

1
Applications of Cellular Neural/Nonlinear
Networks in Physics
Babes-Bolyai University Péter Pázmány Catholic
University

• Mária-Magdolna Ercsey-Ravasz

Scientific advisors Dr. Prof. Zoltán Néda Dr.
Prof. Tamás Roska
2
Outline

• CNN computing
• A realistic random number generator
• Stochastic simulations on CNN computers
• The site-percolation problem
• The two-dimensional Ising model
• Optimization of spin-glasses on a space-variant
  CNN
• Pulse-coupled oscillators communicating with
  light pulses

3
The standard CNN model

• Each cell has a circuit with
• Input voltage u
• State voltage x
• Output voltage y

Template
A , B , z
L. O. Chua , L. Yang, IEEE Transactions on
Circuits and Systems 35. No. 10, 1988
,
.
4
The CNN Universal Machine
ACE16K CNN chip 128128 cells Bi-i V2

• programmable
• parallel processing
• continuous in time
• continuous (analog) in values
• discrete in space
• Universal (in Turing sense) on integers and
  on analog array signals

T. Roska, L. O. Chua, IEEE Transactions on
Circuits and Systems II, 40, 1993
5
CNN computing
Name Year Size
-- 1993 1212
ACE440 1995 2022
POS48 1997 4848
ACE4k 1998 6464
CACE1k 2001 32322
ACE16k 2002 128128
XENON 2004 128962
EYE-RIS 2007 176144

• image processing
• real-time algorithms
• fast and smart camera computer
• robot eyes, bionic eye-glass

• cellular automata models

• partial differential equations

• Research goals
• applications in physics
• how should CNN computers be further developed?
  from physicist perspectives

6
Generating realistic random numbers

• A good pseudo-random generator

Yalcin, et alle., Int.J. Circ. Theor. Appl., 32,
591-607, 2004

• Chaotic cellular automaton perturbed
• with the natural noise of the chip
  
• P(t)P(t) xor N(t)
• N(t) - very few black pixels
• - strong correlations but
• real stochastic fluctuations

P(t) P1(t) XOR P2(t)
7

• a good random binary image in t116 µs
• 1 single random value
• ACE16K ?
  7ns
• Pentium 4 at 2.8 GHz (Linux) ? 33ns

Increasing the size of the chip in the future
will assure even much bigger advantage for CNN
chips
Trend for the simulation time as a function of
the chip size
M. Ercsey-Ravasz, T. Roska, Z. Neda, Int. J. of
Modern Physics C, Vol. 17, No. 6, p. 909 (2006)
8

• generating random images with different p
  density of the black pixels --- using more
  images with ½ density
• --- if p is an n bit
  number we need n images

p0.25 p0.375 p0.03125
P Measured density
1/2 0.5 0.4999529
1/40.25 0.254261
1/80.125 0.124140
1/160.0625 0.061423
1/320.03125 0.031561
1/640.015625 0.015257
1/1280.0078125 0.007470
1/2560.00390625 0.004154
1/4 1/80.375 0.377712

• Correlations
• in space (first neighbors) 0.05 - 0.4
• in time (consecutive steps) 0.7 - 0.8

M. Ercsey-Ravasz, T. Roska, Z. Neda, Int. J. of
Modern Physics C, Vol. 17, No. 6, p. 909 (2006)
9
Stochastic simulations on CNN computers
The site percolation problem

• Used for modeling
• conductivity or mechanical properties of
  composite materials
• magnetization of dilute magnets at low
  temperatures
• fluid passing through porous materials
• propagation of diseases
• Probability of percolation - density of black
  pixels
• 2nd order geometrical phase-transition

• With CNN 1 single template detecting
  percolation
• input the random image
• initial state the first row
• output the parts connected to the first row

ACE16k chip
10

• Percolation probability
• - for each p 10000 different initial conditions
• - results agree with the accepted critical value
• pc0.407

Trend for the simulation time in function of the
chip size

• Time needed
• CNN t L
• digital computers t L
• For L128 CNN is slower
• if L grows still promises advantage

2
M. Ercsey-Ravasz, T. Roska, Z. Neda, Int. J. of
Modern Physics C, Vol. 17, No. 6, p. 909 (2006)
11
The two-dimensional Ising model
Energy of the system
Metropolis algorithm - randomly choose a spin
and flip it with p probability

• On CNN A parallel Metropolis algorithm is
  used
• Because parallel computing we have to avoid
  flipping 2 neighbors simultaneously
  chessboard mask
• Odd (even) step spins marked with black
  (white) are updated
• Equivalent with a Metropolis algorithm in which
  spins are chosen in a well
  defined order

12

• Algorithm scheme for 1 MC step
• Build 3 masks marking
  -generate 2 random images
• Spins with 4 similar neighbors (?E8J) M1
  ----AND------ P1 with exp(-8J/kT)
• Spins with 3 similar neighbors (?E4J) M2
  ----AND------ P2 with exp(-4J/kT)
• Other spins (?E?0) M3
• Build the composed mask M(M1 AND P1) OR (M2 AND
  P2) OR M3
• Use the (inverse) chessboard mask M M AND C in
  (even) odd steps
• Flip the spins marked on M

T2 T2.3 T2.6 (
J/k1)
Movies obtained with the ACE16K chip
M. Ercsey-Ravasz, T. Roska, Z. Neda, Eur. Phys.
J. B, Vol. 51, No. 3, p. 407, (2006)
13
Specific heat
Results for the Ising model
Magnetization
Susceptibility

• Initial state homogeneous
• Boundary conditions fixed
• 5000 transition MC steps
• Averaging over 10000 MC steps

M. Ercsey-Ravasz, T. Roska, Z. Neda, Eur. Phys.
J. B, Vol. 51, No. 3, p. 407, (2006)
14
Time needed for 1 MC step (128128 lattice)
? 4.3 ms on the ACE16K chip ? 2.2 ms on a
Pentium 4 at 2.4 GHz
M. Ercsey-Ravasz, T. Roska, Z. Neda, Eur. Phys.
J. B, Vol. 51, No. 3, p. 407, (2006)
15
Optimization of spin-glasses on a space-variant
CNN
only simulated

• final state after an operation yij ?1
• Lyapunov function (energy) of the CNN

CNN
Spin-glass

• monotone decreasing
• final state local minimum (dE/dt0)

y?-1,1
y1
same local minima 1 operation ? 1 local minimum
16
The stochastic optimization algorithm

• Same principles as in simulated annealing
• noise random input U
• b ? strength of noise
• b slowly decreases
• we choose
• b05
• ?b0.05

1 cooling process
17
NP-hard problem hard for plt0.6

• Speed estimation
• the A templates must be introduced only once for
  each problem
• we can use the characteristic parameters of the
  ACE16k chip

1000- 5000 steps / second Independent of size !
Many applications error-correcting codes,
econophysics, computer science etc.
M. Ercsey-Ravasz, T. Roska, Z. Neda, Physica D
Nonlinear Phenomena, Special issue Novel
computing paradigms Quo vadis?, accepted,
(2008), http//dx.doi.org/10.1016/j.physd.2008.03
.028
18
Pulse-coupled oscillators communicating with
light pulses
Motivations - studying a CNN with pulse-coupled
oscillators -
communicating with light global coupling
- perspectives separately programmable
oscillators - first part of the study
collective behavior of identical units

• The oscillators
• electronic fireflies
• simple integrate-and-fire type neurons
• Photoresistor (R,U) LED
• light R U
• G threshold
• if UgtG LED fires
• not before Tmin
• not after Tmax

19
Collective behavior
Tmin? 800 ms Tmax ? 2700 ms Firing ? 200
ms Reaction time of the photoresistor ? 40
ms Deviations 2-10
20
Order parameter - normalized phase-histogram
smoothing
21
Perspectives

• Separately programmable oscillators
• Tmin, Tmax, Tflash, light intensity A, threshold G

CNN model using pulse-coupled oscillators
Benefits - global coupling - dynamical
inputs - time-delays No independent A(i,jk,l)

• pattern recognition, detecting spatio-temporal
  events
• studying the role of reaction time ?
• dynamically changing the parameters

22
Conclusion

• Realistic random numbers
• Stochastic simulations on lattice models
• The site-percolation problem
• The two-dimensional ising model
• - many related problems could be also simulated
• Locally variant CNN very fast stochastic
  optimization algorithm for spin-glass models
• Further motivating the development of CNN-UM
  hardwares
• CNN built up by pulse-coupled oscillators
• Ineteresting collective behavior further
  studies
• Communication with light could be useful idea
  also in hardware projects

23
Journal publications

• M. Ercsey-Ravasz, T. Roska, Z. Néda,
  Perspectives for Monte Carlo simulations on the
  CNN universal machine, Int. J. of Modern Physics
  C, Vol. 17, No. 6, pp. 909-923, 2006
• M. Ercsey-Ravasz, T. Roska, Z. Néda, Stochastic
  simulations on the cellular wave computers, Eur.
  Phys. J. B, Vol. 51, No. 3, pp. 407-412, 2006
• M. Ercsey-Ravasz, T. Roska, Z. Néda,
  Statistical physics on cellular neural network
  computers, Physica D Nonlinear Phenomena, vol.
  Special issue Novel computing paradigms Quo
  Vadis?, 2008, accepted, http//dx.doi.org/10.1016
  /j.physd.2008.03.28

24
International conferences

1. M. Ercsey-Ravasz, T. Roska, and Z. Neda, Random
   number generator and monte carlo type simulations
   on the cnn-um, in Proceedings of the 10th IEEE
   International Workshop on Cellular Neural
   Networks and their applications, (Istanbul,
   Turkey), pp. 4752, Aug. 2006.
2. M. Ercsey-Ravasz, Z. Sarkozi, Z. Neda, A.
   Tunyagi, and I. Burda, Collective behavior of
   electronic fireflies, SynCoNet 2007
   International Symposium on Synchronization in
   Complex Networks, July 2007.
3. M. Ercsey-Ravasz, T. Roska, and Z. Neda,
   Statistical physics on cellular neural network
   computers. International conference
   Unconventional computing Quo vadis?, Mar.
   2007.
4. M. Ercsey-Ravasz, T. Roska, and Z. Neda,
   Spin-glasses on a locally variant cellular
   neural network. International Conference on
   Complex Systems and Networks, July 2007.
5. M. Ercsey-Ravasz, T. Roska, and Z. Neda,
   Applications of cellular neural networks in
   physics. RHIC Winterschool, Nov. 2005.
6. M. Ercsey-Ravasz, T. Roska, and Z. Neda, The
   cellular neural network universal machine in
   physics. International Conference on
   Computational Methods in Physics, Nov. 2006.
7. M. Ercsey-Ravasz, T. Roska, and Z. Neda,
   NP-hard optimization using locally variant CNN,
   accepted in the Proceedings of the CNNA2008.

25
Thank You!