A Hybrid neural network model for consciousness

1
A Hybrid neural network model for consciousness

• LIN Jie, JIN Xiao-gang, YANG Jian-gang
• (Institute of Artificial Intelligence,
  Zhejiang University, Hangzhou 310027, China)
• Date of publication Mar, 2004

Presented by Bhuban M Seth, Joydip Datta Under
the guidance of Prof. Dr. Pushpak Bhattacharyya
2
Motivation

• Ultimate goal of Artificial Neural Net is to
  imitate a human brain.
• But human brain is too complex to understand.
• Question What is a consciousness and How it is
  generated in brain?
• Is there any hierarchical organization in the
  brain?
• How can we incorporate these newfound insights of
  human brain into an ANN?

3
Understanding the brain(Different Approaches)

• Taylor (1994) Relational Mind
• Rakovic (1997) hierarchically organized and
  interconnected paradigm for information
  processing inside the brain.
• Vitiello (2003) Quantum Model
• Rennie et. al. (2002) Evoked potential

4
Where all these things leads to?

• Cognitive processes are carried out at different
  levels in the brain.
• Higher levels may be reduced to lower
  levels.
• Thus, higher levels of complex brain
  functions require a number of neural modules
  to cooperate together.
• Example We see a rose, smell the fragrance and
  remember some memory this way a conscious state
  of mind emerges in a thinking process.

5
Level 1 Physical Mnemonic Layer

• Physical Mnemonic Layers (PML) capture input from
  external senses and produce a feature vector
  (patterns) from them. Many modular PMLs run in
  parallel.
• There may be two kinds of external inputs
• Arousal Inputs Reach only up to recognition
  Layer Do not take part in Associative
  Recognition
• Aware Inputs Reaches Abstract Thinking Layer and
  may take part in Associative Recognition
• The feature vectors are input to recognition
  layer

6
Level 2 Recognition Layer

• It is a searching tree composed of layered
  storage neurons.
• It receives a pattern from the PML.

7
Level 3 The Global Workspace

• It belongs to the Abstract Thinking Layer.
• It describes the state of Consciousness.
• It can project the abstract information it has
  and mobilize different parts of the brain.
• This global availability of information define
  the conscious state of mind.

8
(No Transcript)
9
Recognition Layer

• Divided into levels
• Each level consists of number of knowledge
  clusters
• Input is the pattern formed by Physical Mnemonic
  Layer (PML)
• This pattern is compared with stored patterns at
  all levels

10
Recognition Layer

• If the pattern is similar to some existing
  patterns
• it will be recognized.
• Else, the pattern will be saved (New neurons will
  be created)
• Similarity is measured by resonant coefficient

11
Inherent Frequency
Definition It is the feature vector of a
pattern. Inherent frequency of a neuron
group k that memorizes a knowledge pattern
can be described by the weights from one
neuron in the group to other members, as
Kwl, w2 ..... wi .... .
12
Similar Patterns

• Similarity of two patterns A, B are determined by
  their Resonant Coefficient R(A,B).
• The resonant coefficient is a kind of
  delta similarity relation satisfying the
  following properties
• Reflexive R(A, A)1
• Symmetric R(A,B) R(B,A)
• And 1 - R(A,C) R(B,C) gt R(A,B) --(Upper
  bound) R(A,B) gt max(0, R(A,C)R(B,C)-1 )
  --(Lower bound)

13
Example

• Suppose a series of four-dimension patterns Pi
  (i0,1,2 .... ,9) formed by PML models
  enter RL. Say, Pi is the binary format of i
  as
• P30,0,1,1, P50,1,0,1, P10,0,0,1.
• We can define resonant coefficient R(Pi, Pj)
  as R(Pi,Pj) 1 (XOR(Pi and Pj)/ 4)
• Then R(P0,P0)1,R(P0,P1)0.75,R(P0,P2)0.75 ,
  R(P0,P3)0.5 and so on.

14
Resonant Space

• It is a representation of pattern showing
  similarity between them.
• Definition It is a space of patterns to which
  any other pattern can be compared to evaluate
  resonant coefficient.
• A pattern P is represented in resonant space by a
  single point, whose projection on an axis
  represents the resonant coefficient between the
  pattern corresponding to the axis and the pattern
  P.

15
Resonant Space(contd)
The resonant space formed by patterns P0 and
P5
16
Cntd

• Consider a resonant space Rn with n patterns
  Pi and the resonant coefficient R(Pi, Pj)
  between any two patterns Pi and Pj .
• From the resonant space formed by n patterns , a
  pattern Pm may be represented on Rn as
• where is the unit vector along Pi axis.

17
Threshold in Recognition Layer

• Definition The thresholds exist in RL
  corresponding to different levels (numbered from
  zero to TOP) togttLgttLlgttTOP, patterns are
  clustered at those levels. For example, at
  level L, two patterns belong to the same
  cluster if and only if tLgtf(u,v)gttLI. There
  also exists a highest threshold tmax and two
  patterns are recognized to be the same if
  f(u,v)gttmax.

18
Abstract Thinking Layer

• It can associatively compare (and recognize)
  different types of inputs.
• It can broadcast its contents to the nervous
  system as a whole allowing different modules to
  interact.
• E.g. The ATL cat take input from the auditory and
  the vision subsystem and while associatively
  recognizing the inputs it can mobilize the
  olfactory subsystem.

19
Abstract Thinking Layer
20
Abstract Thinking Layer

• The ATL is an Bi-directional Backpropagation
  network (BBP).
• A1 and A2 are both input to of the BBP.
• The computation is interleaved only one-way
  learning is going on at a particular interval.
• The structure (no of neurons in different layers
  of the BBP) of the ATL may vary depending on the
  inputs.
• A subset of the neurons are excited at a time
  while rest of them are inhibited. This in general
  represents the consciousness.

21
Consciousness in ATL

• Dynamic workspace states are self sustained and
  follow one another in a continuous stream,
  without external help
• Consciousness generation requires a stable
  activation loop.
• The system enters a stable state V (attractor)
  when there no more change in the state
  possibleV V(t?t) V(t), ?t gt 0

22
Time span threshold in GW

• Establishment of stable state requires a minimal
  duration.
• There is a temporal span of successive workspace
  states.
• If patterns from several subsystems appear in the
  ATL longer than some Time span threshold then a
  conscious state emerges.
• Otherwise they can not establish a self sustained
  activation loop They are called
  sub-consciousness.

23
Conclusion

• Different levels exists in consciousness
  generation process.
• Partial recognition layer threshold helps to form
  clusters within RL unconsciously.
• Strong pattern that persists for more than a time
  span threshold can accomplish associative
  recognition resulting in consciousness.

24
Background Study

• Wikipedia articles on Brain, Human Brain,
  Cerebral Cortex, Hippocampus etc (different parts
  of brain), Neuron, Action Potential,
  Depolarizing, Hyperpolarizing, Inhibited Neurons,
  Excited Neurons, Axon Hillock, Back-propagation,
  Neural Back-propagation, Resting potential,
  Layered perceptron, MLP, Electrical Inductance,
  Electrical Resonance etc.
• Hierarchical Learning in Neural Network
  http//www.cs.iastate.edu/baojie/acad/current/hnn
  /hnn.htm
• A Bi-Directional Multilayer PerceptronM. JEDRA,
  A. EL OUARDIGHI, A. ESSAID and M.
  LIMOURILaboratoire Conception Systèmes,
  Faculté des Sciences, Avenue Ibn Batouta, B.P.
  1014, Rabat10 000, Morocco, e-mail
  jedra_at_fsr.ac.ma