Effect of Conditional Feedback on Learning

1
Effect of Conditional Feedback on Learning

• Navid Ghaffarzadegan
• PhD Student, the State University of New York at
  Albany
• MIT-Albany-WPI System Dynamics Colloquium, Spring
  2008

2
Introduction

• barriers to learning from feedback in a dynamic
  decision making environment
• Complexity of the environment (Gonzalez 2005)
• Misperception of delays (Rahmandad et al. 2007,
  Rahmandad 2008)
• Feedback asymmetry (Denrell and March 2001),
• The existence of noise in feedback (Bereby-Meyer
  and Roth 2006),
• Problems of mental models (Senge 1996)
• People ignore and misperceive feedback (Sterman
  1989a, Sterman 1989b).

3
Introduction

• A common theme in formal studies on learning
• a decision maker makes a decision and receives a
  payoff
• the question is whether or not the decision maker
  is capable of learning from the information.
• Full Feedback

Decision
Perceiving payoff
Payoff
LEARNING
4
Introduction

• Little attention has been paid to the relevance
  of such an assumption.
• eg Police Officer, Admission Office, Human
  Resources Manager
• Conditional Feedback
• For positive decisions we perceive feedback much
  easier than for negative decisions

Decision
Perceiving payoff
Payoff
?
LEARNING
5
Research Problem

• What is the effect of conditional feedback on
  learning
• Or How relevant was the assumption of Full
  Feedback?
• Method
• 1- Simulation.
• 1- Build a differential equation model in Signal
  Detection Framework
• 2- Experiment with the model
• 2- Test with data
• Second hand data a published laboratory
  experiment

6
Framework

• Signal Detection Theory
• Signal vs. Noise
• e.g. Guilty vs. Innocent,
• e.g. capable vs. incapable candidates
• Decision makers try to differentiate signals from
  noise
• Judgment and Decision Making
• Evidence is often ambiguous, and there is
  uncertainty in the environment (Hammond 1996,
  Stewart 2000)

7
Framework

• Important concepts Base rate selection rate
  d threshold
• Payoff
• Threshold Learning

8
Framework

• Conditional Feedback
• Threshold Learning (Cue Learning)

9
Model I Full Feedback
Set threshold ? make decision ? Receive Payoff ?
Perceive Payoff ? correct threshold One stock
Threshold (experiment)
10
Model I Full Feedback
Learning Algorithm 1. Learning from payoff
shortfall Payoff shortfall maximum possible
payoff (Q) payoff (Q, d) maximum possible
payoff (Q) Vtn Q(Vtp-Vtn) 2. Anchoring and
adjustment assumption in correcting the threshold
(Tversky and Kahneman 1974, Epley and Gilovich
2001, Sterman 1989.b)
11
Model I Full Feedback
Inputs noise N(0,1) and signal N(d,1)
Base rate 0.5, values are symmetric, (To
make Base rate traceable) correct decisions are
more valued
12
Model I Full Feedback (learning from Payoff
Shortfall)
Payoff shortfall gt 0
Payoff shortfall0
13
Model I Full Feedback
14
Model I Full Feedback
15
Model I Full Feedback
noise N(0,1) and signal N(d,1) Base rate
0.5, values are symmetric, correct decisions are
more valued
16
Model I Full Feedback - Results

• In full feedback the model is able to learn from
  feedback

17
Model I Full Feedback - Results
Dynamics of selection rate for base rates of 0.3
and 0.7

• In full feedback the model is able to learn from
  feedback
• Looking at payoff shortfall in enough to learn
  threshold
• The speed of approaching depends on the time to
  change threshold

18
Model II Conditional Feedback
Our decision influence our payoff
perception. How do we judge our negative
decision's payoff. People can be different in
interpreting their negative decisions.
(Personality, second loop learning,..)
19
Model II Conditional Feedback
Constructivist strategy For negative decisions
perceived payoff payoff (p,0) p ratio of
signals to total decisions for negative
decisions p0 means assuming all of our negative
decisions are right. p1 means assuming all of
our negative decisions are wrong.
20
Model I Full Feedback
21
Model I Full Feedback
For negative decisions perceived payoff
payoff (p,0) pratio of signals to total
decisions for negative decisions p0 means
assuming all of our negative decisions are
right. p1 means assuming all of our negative
decisions are wrong.
22
Model I Full Feedback
For negative decisions perceived payoff
payoff (p,0) pratio of signals to total
decisions for negative decisions p0 means
assuming all of our negative decisions are
right. p1 means assuming all of our negative
decisions are wrong.
23
Model II Conditional Feedback - Results

• In conditional feedback learning depends on how
  people code their negative decisions
• Elwin et. al (2007) p0
• Stewart et. al (2007) people underestimate
  selection rate and overestimate threshold

24
Model I Full Feedback - Results
Dynamics of selection rate in last 50 trials (a)
and threshold (b) for base rate of 0.5

• Comparison of full feedback and conditional
  feedback in confident constructivist strategy

25
Model I Full Feedback - Results
Dynamics of selection rate for base rates of 0.3
and 0.7

• In conditional feedback and in confident
  constructivist strategy the model is not able to
  learn from feedback
• What is the real p? How do really people code
  their negative decisions?

26
Replications of an empirical investigation

• Data from Elwin et. al (2007)
• Comparison of Full Feedback and Conditional
  Feedback
• Sixty four subjects performed a computerized task
  of predicting economic outcomes for companies
• The experiment had two major phases
• training trials
• test phase
• In the training part, a group of subjects
  performed 120 trials of full feedback decision
  making, while the other group performed 240
  trials of conditional feedback.

27
Replications of an empirical investigation

• We use their published report in our model and
  test parameters that can replicate their
  findings.
• Main parameters d and p. (and time to adjust
  threshold)

28
Conclusion

• A new explanation for imperfectness of decision
  making in a series of tasks. (learning from clear
  shortfalls)
• Conditional feedback can result in bias and
  underestimation of the base rate.
• In respect to second loop learning People do not
  find the optimal threshold
• even if, in the real world, second loop learning
  exists, it works for a limited number of people
• Warning about overestimation of relevance of full
  feedback assumption in formal studies.

29
Conclusion

• Future works
• Effects of personality traits

Effect of Personality on Learning, e.g. using Big
Five
30
Conclusion

• Future works
• Making confidence endogenous.

Effect of Personality on Learning, e.g. using Big
Five Dynamics of confidence building
31
Conclusion

• Future works
• Two individuals communicating

32
Conclusion

• Future works
• Two individuals influencing each others
  performance

33

• Thanks
• FEEDBACK?