Recursive Appetite Recruitment:

1
Recursive Appetite Recruitment

• The mechanism of sudden craving in addictions
• George Ainslie
• Veterans Affairs Medical Center, Coatesville
• Presented at WHAT IS ADDICTION?
• the Third Mind and World Conference
• UAB May 5, 2007

2
What Is Addiction?

• Substance not necessary Gambling, risk-taking
• Arousal probably not necessary Internet,
  procrastination
• Captivation not sufficient Could be ego-syntonic
• NEED REPEATED IMPULSIVENESS

3
Impulse Temporary Preference

• Not primrose path
• -- You know youll regret it
• Not rational addiction (Becker)-- You take steps
  to avoid it
• Thus Rational Choice Theory cant account for it

4
Exponential Discounting

• RCTs formula

Exponential Present value Value0 x
dDelay d 1 discount rate
5
(No Transcript)
6
Why do impulses occur?

• Rational choice theory (RCT) holds that an
  individual always maximizes prospective reward
• Temporary preferences in the absence of new
  information require additional factor
• -- conditioned responses
• -- hyperbolic discount curves
• -- hyperbolOID discount curves
• -- shifting cognitive frames

7
Why Not Conditioned Responses?

• Impulses are unwanted
• Watson proposed trigger by associated stimuli
• But conditioning mistranslated
• Transmits only information (Rescorla)

8
Hyperbolic Discounting Predicts Temporary
Preferences

• Derived from larger matching law (Herrnstein)
• --perhaps in turn from Weber-Fechner law
• Widely replicated
• -- traditional (Read Roelofsma 03)

9
Hyperbolic discounting

• Exponential
• Present value Value0 x dDelay
• Hyperbolic
• Present value Value0 / 1 (k x Delay)
• d 1 discount rate

10
(No Transcript)
11
(No Transcript)
12
Objections to Hyperbolic Discounting

• Does not seem to predict craving for
  non-immediate rewards
• fMRI shows two or more motivational centers
• Several findings said to be inconsistent with
  hyperbolae

13
Why Not Conditioned Craving?

• Relapses often caused by sudden cravings
• Cravings often occur when substance close, but
  not necessarily
• Can be triggered by any associated stimulus

14
Are Visceral Factors Exceptional?

• At high levels, drug craving and other visceral
  factors overwhelm decision making altogether,
  superseding volitional control of behavior
  (Loewenstein, 1999)

15
A Hybrid HyperbolOID Discounting

• Originally meant to preserve exponential
  discounting (Laibson)
• Combined with visceral factors (Loewenstein) via
  conditioning

16
More than a single valuation mechanism

• There are two canonical mechanisms one that
  has a very steep discount function (in the limit,
  valuing only immediate rewards), and one that
  treats rewards more judiciously over time with a
  shallower discount function (Montague et.al.,
  2006) 

17
Hyperboloid Discount Curves

• Exponential
• Present value Value0 x dDelay
• Hyperbolic
• Present value Value0 / 1 (k x Delay)
• Hyperboloid
• Present value Value0 x ß x dDelay
• (0 lt ß lt 1 if ß not 0 d 1 discount rate)

18
A Hybrid HyperbolOID Discounting

• Originally meant to preserve exponential
  discounting (Laibson)
• Combined with visceral factors (Loewenstein) via
  conditioning
• Easy to plot only for immediate case

19
(No Transcript)
20
Objections to Hyperboloid Discounting

• Least-squares curve fitting favors hyperbolae
• Preference reversals occur when all delays are
  nonzero
• Conditioning per se does not magnify the
  probability of reward

21
(No Transcript)
22
(No Transcript)
23
Objections to Hyperboloid Discounting

• Least-squares curve fitting favors hyperbolae
• Preference reversals occur when all delays are
  nonzero
• Conditioning per se does not magnify the
  probability of reward

24
Conditioning sticks to the facts

• Meyer Addicts in program develop craving only
  on days when drug scheduled
• Schachter Orthodox Jews do not crave cigarettes
  on the Sabbath
• Animal experiments Conditioned responses are
  precisely timed to best expectations

25
(No Transcript)
26
(No Transcript)
27
Then why explosive appetite?

• In experiments consumption is limited by
  availability
• In daily life consumption is limited by choice
• If appetite? ? likelihood of consuming
• and ? likelihood of consuming?? appetite
• then a positive feedback loop is created

28
Recursive self-prediction is common

• Darwin-James-Lange phenomenon
  The free expression
  by outward signs of an emotion intensifies it.
  On the other hand, the repression, as far as this
  is possible, of all outward signs softens our
  emotions. (Darwin, 1872)

29
Recursive self-prediction is common

• Darwin-James-Lange phenomenon
• Testing whether youre getting seasick
• Testing whether youre about to panic
• Many forms of performance anxiety

30
Recursive conditioning?

• The process
• ? likelihood of consuming ? ? appetite could
  be because either
• -- Appetite is a CR to self-prediction, or
• -- Self-prediction provides cues for appetite
  as an operant

31
Operant Learning vs. Conditioning

• stimulus? operant response? reward .
• conditioned stimulus (CS)? conditioned response
  (CR)? unconditioned stimulus (UCS)

32
Recursive conditioning?

• The process
• ? likelihood of consuming ? ? appetite could
  be because either
• -- Appetite is a CR to self-prediction, or
• -- Self-prediction provides cues for appetite
  as an operant
• But recursive conditioning should dampen, not
  amplify, increases in CRs

33
Damping of recursive conditioning

• When appetite responds to ? expectation
• 10 more expec should? 10 more app
• if max is app for certainty and immediacy
• When expectation responds to ? appetite
• 10 more app should? lt10 more expec
• unless there is 0 chance of abstention
• Thus successive cycles should add less

34
And conditioning is probably not an independent
selective principle

• In lab only information is conditioned
• All UCSs are also incentives
• CRs are shaped from UCRs how?
• if not reward then a third selective factor
• Willed CRs become easier, not extinguish
• bulimia, acting
• De-conditioning doesnt work as treatment

35
But Can Appetite/Emotion Depend on Reward?

• Often fostered
• -- actors, bulimics
• -- rage, panic, grief urges, therapies
• The big objection is negative appetites/emotions
• -- if not conditioned, what?

36
Wanting but not Liking

• Nail-biting, poor reward schedules, brain
  stimulation
• Sought when close, avoided when distant
• Berridge ?conditioned behaviors
• But could be just faster version of temptation ?
  regret

37
Durations of Temporary Preference

• Longer than temptations Compulsions
  (side-effects of bundling)
• Shorter than temptations Itches, things wanted
  but not liked
• Much shorter Negative emotions and anticipation

38
Negative Emotions Vivid but Aversive

• Negative emotions including pain are hard to
  resist
• Treatments control attention Lamaze, painless
  dentistry
• ? An extreme form of wanted but not liked

39
Aversion High reward ?nonreward
40
Summed curves spike vs. block
41
Continuum of Preference

• Threshold for positive emotions may rise when
  occasions are too common
• Anger may become preferred more than temporarily
  when alternatives are poor
• Fear and grief can be attractive when limited,
  addictive when not

42
Aversive experiences win by seduction rather than
force

• Addiction range High?hangover
• Itch range Relief of urge?interrupted activity,
  repeatedly
• -- Negative emotions are in this range,
  conspicuously anger
• Pain range Attract attention but repel all
  behavior, resist only by hypnosis

43
Emotion both seeks reward and produces reward

• Whats rewarding about fear, grief, and anger?
• -- Pain and painful feelings as seductive
  rapid addictions
• THUS CONDITIONING NOT NEEDED FOR APPETITE/EMOTION

44
Hyperbolae solve RCTs preference reversal problem

• Appetites can be reward-seeking
• Neal Miller from 1968
• Ainslie 1992
• A reward-seeking appetite can exceed expectation
  of consumption
• ?explosive positive feedback loop

45
Conditioned Craving as the Recursive
Recruitment of Appetite

• Appetite itself is mildly rewarding
• Appetite is further rewarded when it predicts an
  imminent rewarding event
• Where choosing the event depends on the appetite,
  self-prediction is volatile (Darwin-James-Lange)
• A problem only in face of self-control?

46
(No Transcript)
47
Objections to Hyperbolic Discounting

• Does not seem to predict craving for
  non-immediate rewards
• fMRI shows two or more motivational centers
• Several findings said to be inconsistent with
  hyperbolae

48
fMRI as a Function of Delay

• Students chose Amazon.com certificates
• SS at 0 or 2 weeks LL at 2 or 4 weeks
• One of their choices wins by lottery
• McClure, Laibson, Loewenstein, Cohen Science
  306, 503-507, 2004.

49
fMRI as a Function of Delay

• If one option was immediate, activity in
• -- ventral striatum
• -- medial orbito-frontal cortex
• -- medial prefrontal cortex
• -- posterior cingulate

50
(No Transcript)
51
fMRI as a Function of Delay

• For all choices, activity in
• -- Intraparietal cortex
• -- R dorso- and ventrolateral prefrontal
• -- R lateral orbitofrontal
• -- various visual and motor areas

52
(No Transcript)
53
(No Transcript)
54
Striatum Does Respond to Delayed Rewards

• Deprived smokers, getting a half or one
  cigarette, right after the fMRI or in one week
  (also deprived)

55
Experimental Design

• fMRI with 8 cigarette smokers (12 h abstinent)
• Playing to win puffs for immediately after the
  scan, or for another session after 12 h
  abstinence 1 week later.
• Each of 100 trials begins by informing the
  participant what they could win (1 puff for the
  present session, ½ puff for the present session,
  1 puff for next weeks session, ½ puff for next
  weeks session).
• Winnings accumulate

56
Full puff now
Half puff now
Full puff 1 week
Half puff 1 week
57
Full puff now
Half puff now
Full puff 1 week
Half puff 1 week
Monterosso, Ainslie et al, in preparation
58
Total Ventral Striatum
59
How Do Cortical and Limbic Centers Interact? Take
your pick

• McClure et.al.s grasshopper and antor Freuds
  id and ego, unpleasure for one system and
  simultaneously satisfaction for the other
• Limited warfare with each other, literal sites of
  the strategically competing interests predicted
  by hyperbolic discounting (cf. Freuds bickereing
  married couple)
• Farsighted centers may derive motivation from
  nearsighted centers (MacLeans rider on a horse,
  who ultimately has to go where the horse wants,
  or Freuds reality principle safeguarding the
  pleasure principle
• A nearsighted center, as our evolutionary
  starting place, may be motivated to access
  farsighted centers to the extant that their
  information can excite present reward

60
(No Transcript)
61
Autonomous Cortical Centers Would Be Unusual

• Later-evolved centers usually work through
  earlier ones
• -- e.g. motor center rides the crossed
  extension reflex
• Whatever the anatomy, there must be a common
  currency (Schultz)

62
(No Transcript)
63
Objections to Hyperbolic Discounting

• Does not seem to predict craving for
  non-immediate rewards
• fMRI shows two or more motivational centers
• Several findings said to be inconsistent with
  hyperbolae

64
Failure to Get Hyperbolae

• Representative Danish adults
• 450 (3000DKK) today vs. up to 1840 in up to 24
  months
• 10 chance of getting it
• Implied interest rates listed
• 15-35 chosen, constant within each subject
  (Harrison, Lau, and Rutstrom, 2005)

65
Subadditive Discounting

• Dutch students, English staff
• Imaginary money
• Amount vs. delay over 2 years and over each equal
  part (e.g. 6 mos.)
• Implied discount rates higher for parts, add up
  to more than whole
• Hyper-concave curve but not hyperbolic, from
  magnitude effect (Read Roelofsma 03)

66
One Experiment with Constant Intervals

• Pigeons chose 2 sec access to grain vs. 4 sec
  access 4 sec later
• Change of preference implies hyperbolic
  discounting (Ainslie Herrnstein, 1081)

67
Food 2 sec. vs. 4 sec.
Ainslie Herrnstein, 1981
68
Framing Theorists Disregard Animal Data

• The connection between findings on pigeons or
  even monkeys and the behavior of humans seems
  rather tenuous. We commonly believe that an
  animal does not understand the choice it is
  facing in the same way that a human being does
  (Rubinstein, 2003)

69
Animal Experiments Are Crucial

• People change greatly with framing (Kahneman,
  Tversky, Loewenstein)
• K varies lt7 in animals, 100s in people
• Only people discount larger amounts less
• THUS Human choices reflect a variable admixture
  of controls

70
The Magnitude Effect

• Larger amounts of money discounted less steeply
• Appears only when amounts differ by an order of
  magnitude or more
• Only in humans
• Not when both amounts are large
• Conclude admixture of prudence norm

71
The Similarity Heuristic

• U.S. and Israeli students
• Imaginary money, amount vs delay
• 467 now v 467.39 in 1 467 in 1 v 607 in 365
• 1000 x 4 mos v 997 1000 4 mos v 997 in 3mos
• 960 in 60 v 958 in 58 1080 in 1 v 1020 in
  60
• Similarity dominates hyperb. curve (Rubinstein,
  2003)

72
Temporal Construal

• U.S. Students
• Now Good radio, poor clock gt poor radio, good
  clk
• 1yr Good radio, poor clock gtgt poor radio, good
  clk
• Now Funny task, boring fillgtboring task, funny
  fill
• 6 wksFunny task, boring fillgtgtboring task,funny
  fill
• Value of high level construal ?with delay
  (Trope Liberman, 2000)

73
Framing Effects

• Prudence has a big effect, but we knew that
• Magnitude effect is probably due to invoking
  prudence
• Subadditive discounting, similarity heuristic,
  temporal construal are small, do not explain
  temporary preference

74
Objections to Hyperbolic Discounting

• Does not seem to predict craving for
  non-immediate rewards
• fMRI shows two or more motivational centers
• Several findings said to be inconsistent with
  hyperbolae

75
Conclusions

• Sudden craving in the absence of new information
  is due to recursive self-prediction is
  evidence for reward-responsiveness of
  appetites/emotions
• Neuroimaging data is agnostic
• Framing phenomena are marginal, and do not
  predict temporary preference

76
(No Transcript)
77
(No Transcript)
78
(No Transcript)
79
(No Transcript)
80
(No Transcript)
81
Outline

• Hyperbolic discounting is well established
• It predicts temporary preferences
• Brief temporary preferences seduce attention
• -- negative emotions
• -- premature satiation
• Premature satiation is controlled only by surprise