Deconstructing the Law of Effect

1
Deconstructing the Law of Effect

• C. R. Gallistel
• Rutgers Center for Cognitive Science

2
Matching Is

• An innate strategy foraging strategy (or,
  possibly, an innate principle of choice
  melioration)
• The apportionment of foraging time (and effort)
  in accord with the incomes obtained from the
  foraging locations
• T1/T2I1/I2, where Ti time invested in Location
  i Ii income obtained from Location i
• Equivalently T1/(T1T2) R1/(R1R2)
  T2/(T1T2) R2/(R1R2) Herrnstein
  fractions
• An evolutionarily stable strategy

3
Income vs Return

• Income (Ii) (reward from Loc i)/(unit time
  spent foraging, without regard to where time is
  invested, that is, summed over i)
• Return (Ri) (reward from Loc i)/(unit time
  spent foraging at a given Loc i), i.e., per unit
  time invested in a location
• Matching equates returnsT1/T2I1/I2 ---gt I1/T1
  T2/I2

4
Model 1 Law of Effect

• Matching driven by inequality of returns
• Subject compares returns and reapportions
  investments to favor location yielding greater
  return
• If income is only loosely connected to
  investment, then this decreases return on favored
  side and increases it on disfavored side
• Eventually, a dynamic equilibrium is attained
  (the equation of returns)
• No change-detector necessary
• Hill climbing slow!

5
Model 2 Innate Strategy Pure Feed Forward

• First proposed in a general way by Gene Heyman
  (matching is unconditioned behavior)
• Leaving rate parameters the inverse of the
  proportions among the incomes
• A separate process detects changes in income
• Income estimator works on small samples taken
  from the period between the time when a change
  point is (retrospectively) estimated to have
  occurred and the time at which the change is
  detected

6
Old Rat Results
7
Appearance of Matching in the Experimentally
Naïve Mouse

• Mouse shuttles back and forth between two feeding
  hoppers
• Hopper entries monitored by IR beams

8
Coupling between investment and income

• Slightly coupled traditional concurrent VIs
  with indefinite hold for harvesting
• Income completely independent of investment
  (pellets accumulate at random rates the mouse
  harvests the accumulation whenever it comes
• Complete coupling the schedule clock only for a
  hopper only runs when the mouse has its head in
  that hopper

9
Tracking Matching

• Compute difference in Herrnstein fractions
  after each feeding
• R1/(R1R2)- R2/(R1R2)(R1- R2)/ (R1R2)
  Feeding Imbalance possible values -1
  1,depending on whether feeding was at 1 or 2
• T1/(T1T2)-T2/(T1T2)(T1- T2)/(T1T2)
  Investment Preference ranges -1 to 1, depending
  on how mouse spent its time between feedings
• Cumulate differences slope of cumulative record
  gives average values when slopes of the 2
  cumulative records are the same mouse is matching

10
Condition 1

• Relative income jointly determined by programmed
  ratio and sampling behavior
• Naïve mouse samples rarely
• Hence initial ratio of incomes may be 11 no
  matter what the programmed ratio
• Mouse is always matching
• Changes in relative income and changes in
  relative investment coincide (even though income
  only very loosely connected to investment)

11
All Mice in Cond 1

• They are, in general, as close to matching at the
  beginning of training as they are at the end of it

12
Condition 2

• Income imbalances now constant from beginning (as
  intended)
• Mice match from the beginning

13
Condition 3

• When income is directly tied to investment, the
  only place in behavioral parameter space for
  matching is exclusive devotion to one or the
  other option
• Thus, on our model, preference should be (almost)
  all-or-none, but unstable

14
A Closer Look

• The session in which the major reversal occurred
• Nothing in the returns justifies the switch!
• The change in income does justify the switch
• This change is caused by the switch, but, in our
  model, the animal doesnt know that, because it
  takes no account of the impact of its behavior on
  what it observes

15
Terminal Economic Weirdness

• Animals are oblivious to the impact of their own
  behavior on the rewards they receive (pace
  Skinner the entire economics profession)

16
Matching theHunter-Gatherer World

• Hunting gathering, unlike farming, do not bring
  food into existence, they merely exploit what is
  there
• Intelligent sampling of a changeable environment
  is essential in hunter-gathering
• Matching maximizes income when income is
  environment-limited rather than
  investment-limited
• Matching samples the world intelligently