Dr' Bruce J' West

1
UNCLASSIFIED / FOR OFFICIAL USE ONLY
Decision Making and Stochastic Delay at Workshop
on Social Computing, Behavior Modeling and
Prediction 1 April 2008
Dr. Bruce J. West Chief Scientist Mathematical
Information Science Directorate Army Research
Office Bruce.j.west_at_us.army.mil 919-549-4257
UNCLASSIFIED / FOR OFFICIAL USE ONLY
2
Decision Making and Delay
Outline of talk

• Discounted Utility Model intertemporal choice
• Anomalies from discounted utility theory
• irrationality
• hyperbolic discounting
• Objective and subjective time
• entropy and the direction of time
• time as a stochastic variable
• Individuality and paternalism
• some experiments
• fit of theory to data
• Conclusions

3
Discounted Utility Model (DUM)
Decision Making and Delay

• Discount factor d compresses many mechanisms
• mortality, uncertainty, time compression,
• Accepted as both normative (how things should
  be) and
• descriptive (how things are)..but was initially
  arbitrary
• Samuelson (1937).
• Exponential form implies time consistency
  (rationality)

4
Anomalies from DUM
Decision Making and Delay

• Time inconsistency
• empirical discount factor is not constant
• over time
• across type of intertemporal choices
• Delay effect (hyperbolic discounting)
• Interval effect (non-stationarity)
• Sign effect (gains vs. loses)
• Magnitude effect (small vs. large)
• Direction effect
• Sequence effects (ordered set vs. single)

5
Model comparison
Decision Making and Delay
hyperbolic

• Exponential delay model
• monotonic decrease in value
• with objective time
• constant rate results in time
• consistency
• rationality
• Hyperbolic delay model
• decreasing rate results in time
• inconsistency
• irrationality (preference reversal)

exponential
exponential
hyperbolic
hyperbolic
exponential
6
Objective vs. subjective time
Decision Making and Delay

• Hyperbolic models
• Objective time
• clockwork universe
• entropy and the direction of time
• Subjective time
• unidirectional
• probability and statistics
• Motivate decision-making using
• ensemble distributions
• subjective time
• stochastic delays

7
Delay and uncertainty
Decision Making and Delay

• Decision-making models of intertemporal choice
  can be extended to incorporate probabilistic
  choice where p is
  the probability of reward at time t and F is an
  unspecified function.
• discrete continuous
• No reward before delay time t
• Delay-time probability density

8
Decision Making and Delay
Stochastic rate

• Deterministic discount rate
  is replaced with a conditional probability per
  unit time
• The ratio of the delay time distribution
  function to the survival probability density,
  integrates to
• The utility function in terms of subjective time
  is therefore

9
Example rate
Decision Making and Delay

• Rate of reward production suggested by hyperbolic
  distribution
• Probability of no reward before time t is
• so that the utility function is inverse
  power law
• T measures response time and a measures
  irrationality

10
Experimental data
Decision Making and Delay

• Students (20) asked to make decisions about
  hypothetical money to be received immediately or
  at a later time, concerning the subjects
  themselves or another person not known to them.
  Takahashi, Physica A (2007).

self
other
T1.85 a0.11
T31 a0.28
11
Implications from experiments
Decision Making and Delay

• The response times could describe paternalistic
  policy making government officials, where
  irrationality is enhanced.
• Irrationality is nowhere more significant than
  in the military where choices may determine
  whether others live or die.

12
Decision Making and Delay

• Nonlinear dynamic equation
  solved on the interval (0,1).
• define a delay-time distribution density
• assume a uniform distribution of initial
  conditions to obtain
• delay-time distribution density is non-Poisson,
  renewal and non-ergodic

13
Measured discount rates
Decision Making and Delay

• Higher discount rates compared with controls (
  smaller T and a in stochastic intertemporal
  model)
• smoking
• excessive alcohol consumption
• illicit drug use (cocaine, crack-cocaine and
  heroin)
• pathological gambling
• age
• cognitive ability (negative correlation with
  intellectual achievement)
• Consistent with neuroeconomic hypothesis that
  prefrontal cortex is essential for patient
  (forward looking) decision making.
• Chabris, Laibson Schuldt, The New Palgrave
  Dictionary of Economics (2007).

14
Brain Activity
Decision Making and Delay
ß network midbrain dopamine network reward
processing (ventral striatum V.Str. and medial
prefrontal cortex mPFC.) d network cognition
dorsolateral prefrontal cortex (diPFC) and right
posterior parletal cortex (R.Par.)
Sanfey, Loewenstein, McClure and Cohen, TRENDS in
Cognitive Science (2006).
15
More brain activity
Decision Making and Delay

• Two discounting slopes
• lt one year
• gt one year
• Different parts of the brain light
• up under fMRI
• short-term
• long-term

Wittmann Paulus (2007)
16
Decision Making and Delay

• Conclusions and Speculations
• decision-making is not always rational
• irrationality in intertemporal choice models take
  a hyperbolic form
• inverse power laws or hyperbolic utility
  functions can be generated by stochastic delay
  times
• different parts of the brain control decisions
  associated with long and short delay times
• the complexity of the brain produces the
  subjective nature of biological time