Adaptive Resonance Theory 3

1
Adaptive Resonance Theory 3

• A Synapse Model
• Alice Kirchheim
• Luis Servin
• Luis Torres

2
Contents

• Introduction
• Motivation for ART plasticity-stability dilemma
• Different ART types
• ART3
• Basic Operation Principles
• Modeling
• Reset Function
• Vigilance Parameter

3
Contents (2)

• ART 3 (Cont.)
• Architecture
• Pattern match and Weight size trade-off
• Algorithm
• Applications

4
Data Classification

• Partition of the input set according to some
  given rules, like similarity and class sizes.
• Clustering Algorithms
• Neural Networks MLP
• Properties
• Stability
• Plasticity
• Dilemma

5
ART

• Solves Stability-Plasticity dilemma
• A set of critical features is extracted from the
  input vector and compared to the internal memory.
• Result
• Resonant state
• Parallel memory search
• Vigilance Parameter defines class sizes

6
ART Types

• ART-1
• Binary input vectors
• Unsupervised NN that can be complemented with
  external changes to the vigilance parameter
• ART-2
• Real-valued input vectors

7
ART-3

• Parallel search of compressed or distributed
  pattern recognition codes in a NN hierarchy.
• Search process leads to the discovery of
  appropriate representations of a nonstationary
  input environment.
• Chemical properties of the synapse emulated in
  the search process

8
Synaptic Representation

• Neurotransmitters dynamics
• Transmitter accumulation
• Release
• Inactivation
• Modulation
• Net effect is to change the permeability of the
  membrane site, leading to excitation of
  inhibition of the cell

9
Synaptic Representation

• Synapse process between ith and jth node
• Si Action potential
• zij Long-term traces
• uij Accumulated transmitter
• vij Released transmitter
• xj Membrane potential

10
ART-3 Dynamics (1)

• Presynaptic transmitter production /
  mobilization of transmitter uij until maximum
  level zij is reached at this rate
• where zij works as a long term memory, changing
  slowly with respect to uij in time

11
ART-3 Dynamics (2)

• Bound Transmitter
• Fraction of the released transmitter reaching
  node j
• Inactivated by a reset signal

12
ART-3 Dynamics (3)

• Post-Synaptic activation
• Excitatory inputs, from all converging synapses
  on node j, drive xj (Membrane potential) to a
  maximum level
• Inhibitory inputs (reset) drive xj to a minimum
  level
• In the absence of input, activity decays passively

13
Architecture

• Hierarchy of ART networks
• No sharp distinction between input representation
  field and category representation field
• Layer 1 reads bottom-up input
• Layer 2 matches patterns from top and bottom
  layers before sending a composite pattern back
  through the feedback loop
• Layer 3 reads up-bottom input

14
Inhibition (Reset)

• Functions
• Correct erroneous category choices
• Learn from reinforcement feedback
• Respond to changing input patterns
• Activation
• Input reset
• Mismatch reset

15
Algorithm

• Step 1. Set vigilance parameter and membrane
  potential value at the input of 2nd field
• Step 2. Compute and check for reset
• Step 3. Iterate short term memory equations
• Step 4. Iterate transmitter equations
• Step 5. Compute bottom-up and top-down released
  transmitter between ith and jth nodes
• Step 6. Increment time and go to step 1

16
Applications of ART 3

• Mobile robot control
• Facial recognition
• Land cover classification
• Target recognition
• Medical diagnosis
• Signature verification