Principles of associative memory in brain networks

1
Principles of associative memory in brain networks

• Minryung Song

2
David Marr
Mental Image of David Marr
3
David Marr
Neuroscientist
Mental Image of David Marr
4
David Marr
Neuroscientist
I dont like him! His ideas are difficult!
Mental Image of David Marr
5
Hebbs principle of associative learning

• Neurons that fire together wires together
• -gt Neural assemblies that fire together wires
  together

6
What is memory?

• In terms of the brain, retrieval means generating
  a full pattern of neural activity from some of
  its components

7
What is memory?

• In terms of the brain, retrieval means generating
  a full pattern of neural activity from some of
  its components

8
What is memory?

• In terms of the brain, retrieval means generating
  a full pattern of neural activity from some of
  its components
• and/or linking patterns of activity in one
  brain area with the corresponding pattern in
  another. 

9
What is memory?

• In terms of the brain, retrieval means generating
  a full pattern of neural activity from some of
  its components
• and/or linking patterns of activity in one
  brain area with the corresponding pattern in
  another. 
• Patterns of activity (and hence the memory) does
  not need to be exactly identical from generation
  to generation

10
Lets pretend to be Dr. Marr
11
Brindley Synapse fixed strength Hebb synapse
12
OUTPUT
INPUT
Hebbs learning
13
Oops!
OUTPUT
INPUT
14
Feed-forward Inhibition

• INPUT

OUTPUT
15
OUTPUT
Feed forward inhibition -1
INPUT
Substractive inhibition Normalize input
16
RepeatX 100

• INPUT

1 0 0 1
Brindley synapse allows Hebbian synaptic
modification
OUTPUT 1
17
Oops!

• INPUT

1 0 0 1
OUTPUT 10
18
Feedback Inhibition

• INPUT

Substractive inhibition Normalize output
OUTPUT
19
Oops!
. . .
20
Divisive inhibition

• Increase contrast
• Can be implemented by shunting inhibition

21
(No Transcript)
22
Put things together
Recurrent loop
23
David Marr
Neuroscientist
I dont like him! His ideas are difficult!
Mental Image of David Marr
24
Neuroscientist
25
David Marr
I dont like him! His ideas are difficult!
26
Neuroscientist
27
Neuroscientist
28
David Marr
I dont like him! His ideas are difficult!
Neuroscientist
Pattern completion through recurrent
loop Recall other memories
29
Neuro. What?
30
Neuro. What?
31
David Marr
I dont like him! His ideas are difficult!
Neuro. What?
32
David Marr
I dont like him! His ideas are difficult!
Neuroscientist
Pattern completion through recurrent loop Error
correction
33
Actual example in hippocampus
34
Hippocampus
35
(No Transcript)
36
Pattern seperation
37
Properties of interneurons predicted by the
simple Hebb-Marr net model

• The inhibitory mechanism must implement a
  division operation on the excitation that reaches
  the cell body from the dendrites.
• Inhibitory cells can be much fewer in number than
  principal cells, but they must have extensive
  connectivity.
• Inhibitory cells must be driven by the same
  excitatory afferentsthat activate the principal
  cells .
• Inhibitory cells must respond to a synchronous
  input at lower threshold and at shorter latency
  than principal cells. Shorter latency is
  necessary to ensure that the appropriate division
  operation is already set up at the somataof the
  principal cells by the time the excitation from
  the same input arrives there via the principal
  cell dendrites.
• Whereas principal cells will be quite selective
  in their response characteristics, inhibitory
  neurons will not be particular about which
  afferents are active at a given time, only about
  how many are active. Thus they will convey little
  information in the principal cells' response
  domain.
• Excitatory synapses onto interneurons should not
  be plastic
• In unfamiliar situations (i.e. when current input
  elicits reduced output) extrinsic modulation of
  inhibitory neurons might lower output threshold,
  successively probing for a complete pattern. This
  might also serve as a gate enabling the
  activation of the synaptic modification process.