Neurobiology of Learning and Memory

1
Neurobiology of Learning and Memory

• Prof. Stephan Anagnostaras
• Lecture 2 Learning Theory

2
Classical (Pavlovian) conditioning
Twitmyer (1902) Paired bell with patellar
tendon tap Previously neutral bell could now
elicit knee jerk Ivan Pavlov Studied
digestion, and noticed that after he worked with
a particular dog for a while, the dog salivated
when it first saw him. Paired metronome with
food Previously neutral metronome elicited
salivation. Called this conditioning
3
Classical (Pavlovian) conditioning
A conditional relationship emerged between the
meaningful and previously neutral stimulus. US -
unconditional stimulus - biologically
significant stimulus (food) UR - unconditional
response (salivation) CS - conditional stimulus
- previously neutral stimulus (bell) CR -
conditional response (salivation) The UR and CR
can be different, but it bears some relationship
to the UR.
4
Classical (Pavlovian) conditioning

• After pairing, how do you know you have a CR?
• Present the CS alone (without the US)
• Measure the response at the beginning of the CS
  (metronome) before the US is presented (food)
• One theory is that the purpose of CSs is to
  predict USs and the CR is a prepatory response.

5

Basic Phenomena
Asymptote
Negatively accelerating growth curve The
stronger the US, the stronger the CR (same
growth rate)
Growth rate
6
Basic phenomena
1.Acquisition from CSUS pairings the curve is
negatively accelerating the stronger US
produces a higher asymptote the CR gets
stronger with repeated trials 2. Extinction
the CS is presented alone after conditioning
CS same curve as acquisition not unlearning
or erasing memory 3. Generalization if you
present a similar CS you will get a similar
reaction generalization decrement
7
Basic phenomena
4. Discrimination Train CS and CS that are
similar Inhibition Associative learning
theory Tries to explain what is going on and
relies on 3 processes to explain everything 1.
Excitation (excitatory association) 2. Inhibition
(inhibitory association) 3. Generalization
Discrimination explained using learning theory
Extinction explained
8
Basic phenomena
Inhibition is a weaker process than
excitation Spontaneous recovery in extinction
Disinhibition in extinction Excitatory
association not lost, its only the buildup of
inhibition that suppresses excitation Law of
parsimony Power of a theory of things
explained---------------------------- of
explanatory principles
9
Procedure, Process, Behavior
Procedure what we do (e.g., pair CS and
US) Process what intervenes between procedure
and behavior (e.g., excitation,
inhibition) Behavioral result what we observe
(e.g., after extinction we see a reduction of
the CR) Our explanation involves all three
Must be aware of this distinction -- procedure is
not what is learned by the animal Skinner
argued only talk about procedure-result laws
(radical behaviorism)
10
Control procedures
In order to study associative learning, must show
change in behavior is due to pairing of the CS
and US Presentation of stimulus alone
increases CR Sensitization Control present
the US alone Presentation of CS alone
increases CR Pseudoconditioning Control
present the CS alone
11
Control procedures
How could we combine the two control
groups? Unpaired group receives both the US
(sensitization) and CS (pseudoconditioning) but
not together. Alternative is the truly random
control. The main point is subject has same
experience with CS and US as the Conditioning
group.
12
Several acquisition procedures
Forward works best. Interestingly this is a test
of Contiguity Theory
13
Several acquisition procedures
Delay conditioning is another term for forward
conditioning. Trace conditioning is quite
special in terms of mechanistic models of animal
learning.
14
Higher order conditioning
Second-order conditioningPhase I Phase
II TestCS1-US CS2-CS1 CS2--gtCRtone-food ligh
t-tone light Sensory pre-conditioningPhase
I Phase II TestCS2-CS1 CS1-US CS2--gtCRlight-
tone tone-food light
15
Generality of conditioning
Conditioning permeates everything you docan
condition pancreas and most glands, voluntary
and involuntary muscles, and immune system Coke
(CS)-----gt Sugar US----gt UR (insulin
release) after a few pairingsCoke (CS) ---gt
CR (insuline release) Abrupt switch to Diet
Coke can cause hypoglycemia Pavlovian
conditioning prepares the body for impending
URs
16
Generality of conditioning
Hollis (1989) blue gouramis mating behavior - if
a male enters territory drives it away Exp 1
Males were subjects TrainingPaired light
(CS) paired with access to males (US)Unpaired
light unpaired with access to males Testing the
light was turned on and barrier removed. Paired
male always won against unpaired male. But also
drives away female.
17
Generality of conditioning
Hollis (1989)Exp 2Paired light (CS) paired
with access to females (US)Unpaired light
unpaired with access to females Testing get
light then access to femaleResult when light
turned on paired group started mating much more
rapidly than unpaired. Exp 3 Design the same
as 2, except female now in between paired and
unpaired male -- female always picks paired male
18
Generality of conditioning
Hollis (1997) Exp 4 Reproductive
success Training Paired got light with access to
female for 2h, Unpaired got light unpaired with
access. Testing present light then give
access to female for 2 h for both groups. Six
days later count baby gouramis
19
xWhat is learned?
Emotional Learning Little Albert
study Conditioned emotional response
(CER) (Pavlovian fear conditioning) Estes
Skinner (1941) Conditioned Suppression Trained
to bar-press for food Paired tone with shock
When tone came on fear suppressed
bar-pressing Suppression became the dominant way
to measure CR
20
What is learned?
Why not just measure fear? No attention to
evolution. Why do rats stop bar-pressing? They
freeze. Nowadays people just measure freezing or
other defensive CR. E.g. Fanselow Bolles 1979
Did fear conditioning with backward (unpaired
group) Evolution heavily influences what is
learned, and even what can be learned
21
What is learned?
S-S vs S-R Two views on learning S-S
CS---gtUS---gt RS-R CS---gtR (US serves to
stamp in this association) Strong evidence for
S-S learningRescorla (1973) Devaluation
Experiment Conditioned Suppression 1. Light
(CS) paired with loud noise (US) 2. US alone -
habituate (control no habituation) 3. Test to
CS - habituation group much less fear
22
What is learned?
S-S vs S-R Rescorla (1974) Inflation
experiment 1. Tone-shock (0.5 mA) 2. US alone
groups - 3 mA - 1 mA - 0.5 mA - no
shock 3. Test CS alone - little devaluation in
0.5 mA group - massive inflation in 1 and 3 mA
groups - Memory of the shock changed and CR
changed
23
What is learned?

• What causes conditioning?
• Contiguity theory things have to occur together,
  that is necessary and sufficient
• Challenges Simultaneous conditioning doesnt
  work well
• Garcia Koelling (1966) Conditioned Taste
  Aversion (CTA)

24
Typical CTA Procedure
Good conditioning with CS-US delay of up to 75
min -Contiguity not necessary
Avoidance
25
What is learned?

• Is contiguity sufficient?
• Kamin (1968) Blocking effectA CS US
  AB two different CSs with US
• Train TestAB B alone good
  conditioninglight-tone-shock light
• HoweverPhase I Phase II TestA AB B alone
  no conditioning!!
• US must be SURPRISING. Note that contiguity is
  the same in both experiments

26
What is learned?

• Is contiguity sufficient?
• Un Blocking effect
• A CS US AB two different CSs
  with US
• Phase I Phase II Test
• A AB B alone conditioning!!
• Big US was SURPRISING.

27
What is learned?
It is also surprising if you dont get the
US Conditioned inhibition procedurePhase
I Phase II TestA AB B cond inhibitor US
was expected but didnt occur!
28
Relationship between cue and consequence

• Garcia Koelling (1966)
• Bright Noisy Water Experiment
• taste associated with illness
• audio/visual stimuli associated with shock

29
Garcia Koelling (1966)
30
Garcia Koelling (1966)
31
Garcia Koelling
Biological constraints on learning
32
Modern learning theory
Wagner, Logan, Haberlandt Price
(1968) Relative validity Theory Two cmpd CSs
AX (tone, light),BX (buzzer, light)Animal
sometimes get AX, sometimes BX In group 1
(correlated conditioning group) AX is reinforced
100 (AX) and BX is never reinforced (BX-) In
group 2 (the uncorrelated group) AX is
reinforced 50 of the time, and BX is reinforced
50 of the time.
33
Modern learning theory
In training Correlated group Uncorrelated
Grp AX 100 reinf AX 50 BX 0 BX
50 A predicts US neither A or B B predicts
no US perfectly predicts US Both get 50
reinforcement overall. But what is happening to
X? X is reinforced 50 of the time in both
groups. According to contiguity theory should
have the same conditioning. What happens?
34
Modern learning theory
In test phase Correlated gp Uncorrelated gp A
alone Strong cond No cond B alone No Cond No
cond X alone No cond Strong cond X has the
same number of pairings in both groups, so
contiguity theory is screwed Wagner says the
cue must be the most valid predictor of the US in
the situation in order to get associated.
Relative validity to other CSs.
35
Modern learning theory
Correlated group A perfectly predicts shock,
and X only half the time predicts
shock Uncorrelated group A predicts shock half
the time when its on, the same for B. But X
predicts shock half the time whether A or B are
on or not. So X is the most valid cue in this
situation.
36
Modern learning theory
Rescorla (1968), Contingency experiment CS
tone, US shock For all groups, P (USCS) 0.8
(80 of the time you get the CS you will get the
US also). Rescorla varied the P(USno CS) for
all groups.
37
Modern learning theory
38
Modern learning theory
Rescorla called this contingency theory
If P(USCS) gt P (USno CS) then excitatory
conditioning If P(USCS) lt P(USno CS) then
inhibitory conditioning (e.g., safety
signal) If P(USCS) P(USno CS) then no
conditionin occurs (truly random control)
39

Rescorla-Wagner Model (1972)

• Key Assumptions
• Emphasize CS-US pairings as criticial for
  conditioning
• Formalize the notion of Kamins suprirse
• Assume that any US can only support a limited
  amount of conditioning/reinforcement
• All the CSs compete with echother for the limited
  amount of conditioning/reinforcement
• Competition occurs through summation of all the
  CSs present on a given trial
• The US has a certain amount it can condition,
  meaning this is a US-limiting model.
• Stimuli compete for ability to predict the US.

40

Rescorla-Wagner Model (1972)
41

Rescorla-Wagner Model (1972)
Can explain a number of phenomena Acquisition,
extinction Blocking (A, AB, B) Unblocking
(A, AB, B) Conditioned Inhibition (A, AB,
B) Contingency Can deal with a number of
phenomena and makes several new predictions which
were testable Cannot deal with latent
inhibition (CS pre-exposure) Can deal with US
pre-exposure effect