Law of Effect

1
Law of Effect

• Remember the Law of effect (Thorndike, 1911)
  behavior that is followed by a satisfying state
  of affairs is more likely to occur again and vice
  versa
• With each trial, latency response increases (up
  to some point)
• With each trial, rate of responding increases (up
  to some point)
• Learning theorists note that animals improve on
  performance optimize, not just repeat same
  behavior but make it better and more efficient
• Note that considers adaptation a question, and
  not an answer (e.g., what are adapting to how
  are adapting how know to adapt, etc)

2
Violations of Law of Effect and adaptation

• Ferster and Skinner change in response rate
  between VR and VI schedules why if just stamping
  in of behavior? Why do this?
• behavior on conjunctive schedules
• conjunctive schedule reinforces first response
  after certain number of responses AND first
  response after set amount of time
• double criteria
• CONJ FI(t) FR(n1)
• Difference between FI and FR
• for FI schedule rapid responding is penalized
• FR or ratio schedules rapid responding more
  reward
• CONJ schedule combines
• rate of Sr is directly proportional to rate of
  responding only for rates of responding on FI
  schedule
• not for FR schedule.

3
Question why do animals respond differently for
different schedules, and why or how do they know?

• Need to conduct research on this
• Much of early behavior studies investigated basic
  scheduled reinforcers and behavior questions

4
Herrnstein and Morse (1958)

• Used a CONJ FI 15 FR (0 to 240) with pigeons
• Found
• Rate of responding decreased with increases in FR
  value
• This resulted in reduced rate of reward
• Why is the animal doing this? cant be
  stamping in of response or wouldnt get
  fluctuations
• May have to do with response strength- how?
• Response strength strength of association
  between R and Sr
• Does this strength differ with different kinds of
  schedules?

5
Reinforcement as strength

• making stronger link between responding and
  reward
• Use relative frequency of responding as a measure
  of strength
• response rate as function of reinforcer rate
• R/time f(Sr/time)
• plot proportion of responses as function of
  proportion of reward
• Use INTERVAL schedules rather than RATIO
  schedules
• note is continuous measure, and not discrete
  trial animal has more choice
• reinforcers come on interval, not ratio schedules
• no proportionality between number of responses
  and number of reinforcers-
• all time based
• Schedule controls reinforcement rate, not the
  animal
• this becomes basis of matching law

6
Side Bar Explain use of COD

• Affects Response strength and choice
• Shull and Pliskoff (1967) used COD and no COD
  and ICS as reward
• Used Findley procedure
• On concurrent schedules, two independent
  reinforcer schedules, one for left and one for
  right
• Typically, timers for these run independently
• On Findley procedure, when a reinforcer becomes
  available, it is HELD until it is taken, that is,
  the clock stops and starts only after the
  reinforcer for that choice is obtained.
• matching occurred as long as used COD
• why important
• COD not controlling factor- response ratio was
• COD makes the choice setting MORE SALIENT
• Extended matching to rats
• If use COD, rats will match (they are rather
  dumb other wise!)
• Extended matching to another Sr
• Shows several other studies that also found
  matching

7
Herrnsteins Matching Equation

• P1 kR1
• ------------
• R1Ro
• P1 rate of responding to alternative 1
• R1 rate of reinforcement for alternative 1
• Ro rate of unaccounted sources of reinforcement
• k asymptote of response rate

8
Can derive a more general two-choice equation

• P1 kR1
• ------------
• R1 R2 Ro
• ---------------------------------------------
• P2 kR2
• ------------
• R1 R2 Ro

9
Cancelling out

• P1 kR1
• ------------
• R1 R2 Ro
• ---------------------------------------------
• P2 kR2
• ------------
• R1 R2 Ro

10
Two-Parameter Matching Equation

• P1 R1
• ---- ----
• P2 R2
• Must assume that Ro is equal for both P1 and P2
• Note that are everything is measurable!

11
Objections to matching

• Several alternative models that suggest that
  matching is a special case, for example
• Probability models or probability learning
• Melioration (make the best choice) models or VR
  schedules
• But note here matching would be exclusive choice
• Can show mathematically can produce matching from
  probability learning
• Do get some predictable deviations from matching
  under certain circumstances-
• need to address these circumstances from the
  matching law model or else matching law becomes
  too limited-
• Baum will do this next

12
Choice as Behavior and Vice Versa

• Know that relative rate of responding varies with
  relative rate of reinforcement must affect
  absolute rate of responding as well.
• Difference between relative rate and absolute
  rate
• Absolute rate P1/time
• Relative rate P1/P2 or P1 relative to P2

13
Herrnsteins matching equation

• P1 kR1/R1 Ro
• P1 absolute rate of responding for alternative
  1
• K constant multiplier for maximum response rate
• R1 absolute rate of reinforcement for
  alternative 1
• Ro absolute rate of reinforcement from other,
  unaccounted sources!
• makes a hyperbola function
• some maximum rate of responding
• Why?

14
How plot?

• Plot reinforcement rate (responses per minute) as
  a function of reinforcer rate (reinforcers per
  minute)
• Again, forms a hyperbola
• Decelerating ascending curve
• Why decelerating- why reach asymptote?
• Note is a STEADY STATE theory, not an acquisition
  model

15
Example

• Plot response rate as a function of reinforcer
  rate

16
Now look at Herrnsteins figures

• figures on pages 255-256

17
Factors affecting the hyperbola

• Absolute rates are affected by reinforcement
  rates
• Higher the reinforcement rate the higher the rate
  of responding
• True up to some point (asymptote)- why?
• Function is affected by changes in Ro
• more or less other reinforcers available
• What would happen if put sugar water in tube and
  made it freely available?
• Can also plot for P1/R2 R1/R2 and get same
  general trend

18
Enough!

• next section explains multiple schedule and how
  the model fits this, we will skip that for now