Decision-Making and Eye Movements

1
Decision-Making and Eye Movements

• April 2006
• Noel Schaeffer

2
Questions

• 1) How can eye movement data be used to provide
  information on decision making?
• 2) What do eye movements tell us specifically
  about the decision making process?

3
Abstract

• The major emphasis of this review is in three
  areas including
• 1) A brief history of eye movement studies.
• 2) The neurological foundations of eye movements
  as they are related to decision making.
• 3) Applied human factors study involving cancer
  detection using x-ray films.

4
Eye Movements and Decision Making A Brief
History

• Tikhomirov and Poznyanskaya (1966) studied eye
  fixations to better understand the heuristics
  used in problem solving.
• Building on the previous research of Yarbus
  (1961) the authors used eye movement data
  collected on film to gather information in order
  to better describe the intellectual functions
  of eye movements and their role in solving mental
  problems during a simulated chess game.
• Tikhomirov, O. K., Poznyanskaya, E. D. (1966).
  An investigation of visual search as a means of
  analyzing heuristics. Soviet Psychology, 5(2),
  3-15.

5
Eye Movements and Decision Making A Brief History

• They recorded eye movements during participants
  selection of moves in a chess game.
• The chess board was positioned in front of the
  participants while they rested their head on a
  chin rest to minimize movement.
• A small hole was made in the middle of the chess
  board to accommodate the filming camera.
• The eye movements were recorded using a filming
  method of 24 frames per second and were subject
  to a frame by frame analysis in order to collect
  fixation times and the pattern of eye movements.

6
Eye Movements and Decision Making A Brief History

• The most important finding described by the
  authors is that subjects activity in finding a
  move is constructed by means of repeated
  investigation of the same elements, carried out
  mainly by the movements of working out moves.
  (p. 9 Tikhomirov Poznyanskaya, 1966).

7
Eye Movements and Decision Making A Brief History

• A study by Russo and Rosen (1975) used eye
  fixation data to look at a multialternative
  choice process.
• The authors point out that up until this point in
  the literature there had been a focus only on two
  alternative choices and that this category of
  choices is not representative of naturalistic
  choice making behavior.
• Russo, J. E., Rosen, L. D. (1975). An eye
  fixation analysis of multialternative choice.
  Memory and cognition, 3(3), 267-276.

8
Eye Movements and Decision Making A Brief History

• The authors had participants view a monitor and
  press a button to begin the trial. A description
  of six used cars (make, year, and mileage) was
  displayed to them on a CRT monitor for less then
  a second one at a time, and then all together.
• The six used car descriptions were then available
  continuously in a 2 by 3 array until the end of
  the trial.

9
Eye Movements and Decision Making A Brief History

• The main use of the sequence of eye fixations was
  the inference of a pair comparison (binary
  processing) from alternations of 3 or more.
• For example, if a participant used a sequence of
  eye fixations of X-Y-X-Y then this was considered
  a pair comparison.

10
Eye Movements and Decision Making A Brief History

• The participants were instructed to choose the
  best used car from the six available.
• Eye movements were examined during the
  experiments using a photoelectric sensing
  apparatus.
• Eye positions were sensed 100 times per second
  and recorded by a PDP-12 computer (Digital
  Equipment Corporation).

11
Eye Movements and Decision Making A Brief History

• The authors discovered that participants do use a
  pair comparison (binary processing) structure
  during the multialternative choice process.
• The data showed that during a multialternative
  choice participants generally employed an eye
  fixation sequence of X-Y-X-Y which is considered
  a pair comparison computational strategy.

12
Eye Movements and Decision Making A Brief History

• A classic study by Rosen and Rosenkoetter (1976)
  examined eye fixations in order to inspect the
  judgment and decision making process.
• In this study the primary goal was to use the
  data collected from eye fixations to determine
  what heuristic strategies are used across
  different environments
• The authors divided the constructs of choice
  (decision, or dimensional strategy) and judgments
  (holistic evaluation) into two distinct areas.
• Rosen, L. D. Rosenkoetter, P. (1976). An eye
  fixation analysis of choice and judgment with
  multiattribute stumuli, Memory and Cognition,
  4(6), 747-752.

13
Eye Movements and Decision Making A Brief History

• For a dimensional strategy (decision) the eye
  fixation sequence should exhibit an alternating
  series of fixations from a single dimension of
  one stimulus alternative to the same dimension of
  the other stimulus (p. 748 Rosen Rosenkoetter,
  1976).
• Gamble A Gamble B
• Money to be won 4.29 2.85
• Probability of winning .44 .72
• Amount to be lost -1.29 -2.80

14
Eye Movements and Decision Making A Brief History

• For a dimensional strategy (decision) the eye
  fixation sequence should exhibit an alternating
  series of fixations from a single dimension of
  one stimulus alternative to the same dimension of
  the other stimulus (p. 748 Rosen Rosenkoetter,
  1976).
• Gamble A Gamble B
• Money to be won 4.29 2.85
• Probability of winning .44 .72
• Amount to be lost -1.29 -2.80

15
Eye Movements and Decision Making A Brief History

• Eye fixation patterns for those participants
  using a holistic evaluation (judgment) would
  follow a pattern consistent with moving from
  dimension to dimension within a single stimulus.
• Gamble A Gamble B
• Money to be won 4.29 2.85
• Probability of winning .44 .72
• Amount to be lost -1.29 -2.80

16
Eye Movements and Decision Making A Brief History

• Eye fixation patterns for those participants
  using a holistic evaluation (judgment) would
  follow a pattern consistent with moving from
  dimension to dimension within a single stimulus.
• Gamble A Gamble B
• Money to be won 4.29 2.85
• Probability of winning .44 .72
• Amount to be lost -1.29 -2.80

17
Eye Movements and Decision Making A Brief History

• The stimuli were presented to paid participants
  on a CRT monitor.
• Eye movements were recorded via a photoelectric
  eye position sensor, which was continuously
  displayed on another monitor in front of the
  experimenter who monitored the data for errors.
• The experimenter had all six participants use a
  bite bar to restrict their head movements and
  provide more accurate eye movement data.

18
Eye Movements and Decision Making A Brief History

• The authors found that the stimulus environment
  (i.e. the gamble, vacations, and gift packages
  environment) had a significant and systematic
  impact on the heuristics that were used by the
  participants.
• The gamble and vacation environments both showed
  a holistic evaluation (judgment).
• The gifts environment showed the use of a
  dimensional strategy (decision).

19
Eye Movements and Decision Making A Brief History

• Russo and Dosher (1983) in which they conducted a
  series of experiments that used eye fixation
  patterns along with verbal report to determine
  which decision making strategies (holistic or
  dimensional) would be used by participants.
• The paradigm under investigation was
  multiattribute binary choice.
• Russo, J. E., Dosher, B. A. (1983). Strategies
  for multiattribute binary choice. Journal of
  Experimental Psychology Learning, Memory, and
  Cognition, 9(4), 676-696.

20
Eye Movements and Decision Making A Brief History

• The task environment was hospitable (neutral) to
  both holistic and dimensional strategies in
  experiments 1 and 2.
• In experiment 3 the task was asymmetrically
  skewed to represent mostly holistic attributes
  (gambles).
• In all three experiments a dimensional strategy
  was predominant. Only in the third experiment in
  which the environment was manipulated did the
  authors see a small decrease in the use of a
  dimensional processing method.

21
Eye Movements and Decision Making A Brief History

• An explanation of the use of the dimensional
  strategy compared to the holistic strategy would
  be analogous to comparing apples to apples
  (dimensional) versus apples to oranges (holistic)
  (Russo Dosher, 1983).
• This analogy implies that it is somehow
  cognitively less effortful to compare these
  attributes with one another. The idea of
  reducing cognitive effort at the expense of a
  small increase in errors is congruent with the
  heuristics and biases approach posited by Tversky
  and Kahneman (1971, 1973, 1974).

22
Neural Foundations of Eye Movements and Decision
Making

• The examination of decision making can be done
  through the collection of eye movement data in
  combination with neurological data.
• Jeffrey D. Schalls Website
• Schall, J. D. (2001). Neural basis of deciding,
  choosing and acting. Nature Reviews
  Neuroscience, 2, 33-42.

23
Neural Foundations of Eye Movements and Decision
Making
24
Neural Foundations of Eye Movements and Decision
Making
25
Neural Foundations of Eye Movements and Decision
Making
26
Neural Foundations of Eye Movements and Decision
Making

• Research examining behavioral risk preference,
  which is of major interest in judgment and
  decision making, is exemplified by a study from
  McCoy and Platt (2005).
• The methodology of this study uses an interesting
  visual gambling task with macaque monkeys while
  recording neural output in the posterior
  cingulate cortex.
• McCoy, A. N., Platt, M. L. (2005).
  Risk-sensitive neurons in macaque posterior
  cingulate cortex. Nature Neuroscience, 8(9),
  1220-1227.

27
Neural Foundations of Eye Movements and Decision
Making
Reward
T1
Move
Fixation Off
T1
T2
Delay
T2
Time
Target On
Fixation On
28
Neural Foundations of Eye Movements and Decision
Making

• The visual gambling task included a fixation
  point (yellow LED) and two targets (yellow LEDs)
  displayed on a monitor. The targets were
  classified as certain and risky because of
  the amount of juice reward that was allowed
  access by the researchers.
• For example, an eye gaze to the certain target
  allowed the monkey 150ms access to the juice
  reward everytime. Eye movements to the risky
  target resulted in the random access to the juice
  reward for less then 150ms for half of the trials
  and more than 150ms for the other half of the
  trials.
• The mean access to the juice reward for the risky
  target was 150ms, which was equal to that of the
  certain target over time.
• There were 4,040 trials between two monkeys.

29
Neural Foundations of Eye Movements and Decision
Making

• The results of this study found that monkeys
  prefer the risky target despite the random
  variation in juice reward described earlier.
• Note that the overall juice reward from choosing
  each target was the same over time.
• Neuronal activity also increased in the posterior
  cingulate cortex in relation to the risky choice
  made by the monkey.

30
Neural Foundations of Eye Movements and Decision
Making

• The decisions about where and when to look are
  important because they not only provide insight
  into the cognitive processes that occur within
  the brain, but also allow researchers to examine
  the physiological events that take place at a
  mechanistic level.
• A review by Glimcher (2001) considers this point.
• Glimcher, P. W. (2001). Making choices The
  neurophysiology of visual-saccadic decision
  making. Trends in Neurosciences, 24(11), 654-659.
• Glimcher, P. W. (2003a). The neurobiology of
  visual-saccadic decision making. Annual Review
  of Neuroscience, 26, 133-179.

31
Neural Foundations of Eye Movements and Decision
Making

• The modern field of neurophysiology with respect
  to the study of decision making borrows heavily
  from the social sciences including economic
  theories of decision making in order to build on
  the evolving architecture called neuroeconomics
  that is quickly being established.
• Traditionally the neurosciences in general have
  been extremely data rich but have lacked the
  underlying theory necessary to build a proper
  architecture that would allow it to transcend
  physiology and perfuse into the social sciences.

32
Developments in Applied Eye Tracking and Decision
Making Research

• A review by Krupinsky (2000) provides a unique
  view of the importance of eye tracking research
  applied to the detection of cancer nodes in x-ray
  images.
• This importance is due primarily because of the
  imperfect ability of viewers to visually detect
  lesions or other malignant tumors.
• The error rates associated with miss diagnosed
  x-ray films directly impacts patient care.
• Krupiniski, E. A. (2000). The Importance of
  perception research in medical imaging.
  Radiation Medicine, 18(6), 329-334.

33
Developments in Applied Eye Tracking and Decision
Making Research

• The author classifies false negatives into three
  categories including
• Search Errors
• Missed lesions no foveal vision.
• Recognition Errors
• Some lesions are looked at but not long enough.
• Decision Errors
• Some lesions are looked at for long periods of
  time and still missed.
• The author argues that decision errors can be
  reduced by using perceptually-based feedback.

34
Developments in Applied Eye Tracking and Decision
Making Research
35
Developments in Applied Eye Tracking and Decision
Making Research

• It was hypothesized that the physical circle
  around the long-dwell areas served to focus
  attention on that area.
• This increased attentional focus allowed the high
  resolution fovea to fall on the suspected area
  more frequently and with greater accuracy
  (Krupiniski, 2000).
• This technique may be used in newer non-film
  (digital) applications to draw radiologists
  foveal attention to areas of suspicion.

36
Conclusion

• 1) How can eye movement data be used to provide
  information on decision making?
• Eye movement data can be used to
• examine decision strategies in multialternative
  choices.
• The combination of neuronal, behavioral and eye
  movement data can be used cooperatively to
  examine decision making.
• It can be used to assess decision making errors.

37
Conclusion

• 2) What do eye movements tell us specifically
  about the decision making process?
• Eye movements reflect cognitive programs that
  humans use to make decisions.
• Different environments select for different
  evaluation strategies and thus eye fixation
  programs, some of which may be error prone.
• Eye movements can help researchers evaluate the
  decision making process by providing additional
  information into the cognitive mechanisms (even
  at the neurological level) that produce them.