Project Goals:

1

2
Project Goals Developing the biologically
motivated mathematical theory of random
percolations and first-order phase transitions in
spatially extended probabilistic systems and
explaining the spatial-temporal characteristics
of phase transitions in a neural network model,
whose architecture is inspired by the neuronal
architecture. This theory will be used for the
interpretation of the operation of dynamical
memories in artificial and biological systems.
Ideas Tools Support by EIA enables to
establish a link between pattern-based
information processing using dynamical memories
and percolation phenomena, called
neuropercolation. Based on the theory of
neuropercolation, we describe dynamical behavior
of the neuropil in cortices. Massive
computational resources are used to gain
information about the behavior of large-scale
random systems near the phase transition. The
results of simulations will provide conjectures
which will be explored using random graph methods.
Impact This project generates a synergy between
various disciplines to achieve a revolutionary
new understanding of information encoding in the
central nervous system and to implement these
principles in a new generation of computational
devices.
Barriers Opportunities The problem of phase
transitions in random systems embedded in space
is an immensely complicated task and, in most
cases, it still defies rigorous analysis.
Biological intuition provides vital help in
setting up suitable models with desired behavior.
We propose to use statistical tools in the
construction of the models themselves, replacing
the differential equations by stochastic models
to derive spatial-temporal oscillations with
biologically plausible characteristics. This
approach has the advantage of the natural
introduction of noise required to stabilize the
dynamics, a greater degree of plausibility of the
model, and less empirical foundation for neural
modeling.
3
INSPIRATION FROM NEURONAL ARCHITECTURE
Neuropil Densely connected filamentous texture
of nervous tissue
schematic view of axon
mesoscopic state variables are describe by the
collective activities of neurons in the pulse and
wave modes. Transformation of the neurons from
one mode of existence to another is example of
state transition.
microscopic state variables are described by
the activity of the single neuron, which converts
incoming pulses to waves, sums them, converts its
integrated wave to a pulse train, and transmits
that train to all its axonal branches.
4
MODEL WITH NON-LOCAL CONNECTIONS
Assignment of Neighbors
site with local neighbors only
site with additional remote neighbor
NxN Lattice/Torus
1) every site has 4 local neighbors with same
relative position 2) randomly selected sites with
the additional remote neighbor 3) one way
directions for remote neighbor 4) once assigned
neighbors never change
Site activation changes in time, which is
measured in discrete steps. At each step
activation is defined by the transition rules,
(see next).
5
Transition Rules
step i
.5
e
1-e
e
.5
1-e
step i1
(c)
(b)
(a)


e if majority active 1-e if majority inactive
0.5 if number of inactive number of
active e if majority inactive neighbors 1-e
if majority active 0.5 if number of inactive
number of active
chance of being inactive


chance of being active
6
Statistical Description
schematic view of statistical description and an
example of behavior at critical e
di the ratio of the number of active sites and
total number of sites e the probability of the
site to change its state T Number of runs r
difference between upper and lower density level
in mostly active/inactive mode of behavior µ
average d within the given range Ki time with
density within given r between transitions ti
wait time for the oscillation si number of
steps from the end of ti to the beginning of
ti1 A amplitude of the oscillation MAX
maximum value of d for the experiment with at
least one transition Critical e is the value for
which M gt E M the number of di in the
range(0.25 MAX, 0.75MAX) E the number of di
in the range(0. 0.25MAX) and range(0.75MAX,
MAX).
7
DYNAMICS OF MESOSCOPIC STATES Simulation Results
on Mesoscopic Densities
densities in time and typical spatial
configurations depicted in the lower right corner
8
e vs. Average Activation Density for Different
System Configurations

• Changed structure changes the critical e

different systems have different structure of
connections and/or different number of remote
connections
9
Critical e vs. Remote Connections

• Adding neighbors and/or changing the connection
  architecture changes the critical e with lower
  and upper bound when the lattice has no remote
  neighbors and when lattice is fully connected
  respectively.
• It is possible to simulate critical e between
  lower (0.13428) and upper bound (0.233) to any
  desired precision by arbitrary lattice size and
  connection architecture.

100 corresponds to 400 remote neighbors, 20 to
80 remote neighbors and so on.
10
Average Waiting Time of Phase Transitions

• Given the e, which is below the the critical e,
  in the lattice with fewer sites there is shorter
  wait time.
• Given the lattice, the higher the e, up to the
  critical one, the shorter the wait time.
• Waiting time for the oscillation to happen
  decreases with addition of remote connections
  when comparing different systems within same
  relative distance from the critical e.

(a) corresponds to the system with no remote
neighbors, and (b) to the system with 100 (4)
remote neighbors
11
Average Residence Time of Phase Transitions

• Given the e, which is below the critical one,
  lattice with more sites needs more runs to
  accomplish the transition.
• Given the lattice size without enough remote
  connections, and the e level below the critical
  one, the lower the e the higher the number of
  runs to accomplish the transition.
• With enough remote connections, time needed for
  transition stays about the same for any e level.

(a) corresponds to the system with no remote
neighbors, and (b) to the system with 100 (4)
remote neighbors
12
CONCLUSIONS AND FUTURE PLANS
Adding remote neighbors causes the increase of
critical e level. There is lower and upper bound
for the critical e, when the lattice has no
remote neighbors and when lattice is fully
connected respectively.
Computations
(a)
(b)
Given the model described with the enough remote
connections it is necessary to have layers of
excitatory and inhibitory sites that cause the
system to oscillate and bring them back to
chaotic behavior. The given stimulus excite the
network. If the network starts to oscillate it
recognizes the input, (b), else it stays the
same, (a), which means it is time to learn the
new input. To learn means to gradually adjust the
weights of simultaneously stimulated neurons.
Eventually with enough stimulus connections get
strong enough and phase transitions occur. After
the oscillation period the system gets back to
chaotic behavior.
13
ACKNOWLEDGMENTS This research is supported by NSF
grant EIA-0130352.
REFERENCES 1 W. J. Freeman (1999) How
Brains Make Up Their Minds, Weindenfeld
and Nicolson Publication. 2 P. Erdos, A.Renyi
(1960) On the Evolution of Random Graphs,
Publ. Math. Inst. Hung. Acad. Sci., 5,
17-61. 3 B. Bollobas (1985) Random
Graphs, Academic Press, London
Orlando. 4 W. J. Freeman (1975) Mass
Action in the Nervous System, Academic
Press, N.Y. 5 R. Kozma, P. Balister, B.
Bollobas and W. J. Freeman (2001) Dynamical
Percolation Models of Phase Transitions in the
Cortex, Proceedings NOLTA'01 Nonlinear
Theory and Applications Symposium, Vol. 1, pp.
55-59 6 M. Puljic and R.Kozma (2003)
Phase Transitions in a Non-Local Neural Network
Model Having Partially Random Connections,
submitted to IJCNN'03 Conference, Portland, OR