Towards a stochastic theory of saccadic eye movement

1
Towards a stochastic theory of saccadic eye
movement

• GURU Oct 1, 2001
• Jonathan Nelson

2
Thanks especially to

• Wendy Ark
• Gary Cottrel
• Carrie Joyce
• Javier Movellan
• Marty Sereno
• Ruth Williams

3
Outline

• Function of eye movement in vision
• Traditional analyses of eye movement
• Three papers using stochastic processes to model
  eye movement
• Research towards a better generative model

4
Why model eye movement?

• for cognitive science and economics how do a
  persons knowledge and goals influence their
  information-gathering in perception?
  (applications also to entertainment and interface
  design)
• for medical diagnosis how does a
  physician/technician read an x-ray? Can we
  automate or assist the physician?
• for understanding and treating pathological
  conditions autism, williams syndrome. Can
  training appropriate eye movement help autistics
  in social situations?
• for artificial perception are the visual data
  that humans acquire also informative for active,
  sequential artificial vision systems?

5
Saccadic eye movement

• Eccentricity and image resolution
• Saccades and fixations
• Relationship to task, applications

6
Traditional analyses

• Large number of papers
• Inspired by behavioral statistics
• lose dynamic character, or
• dichotomize/polychotomize time
• Yet unexplained qualitative results
• highly individual scanpaths
• high individual consistency from trial to trial

7
Towards a stochastic theory

• Regions of interest in an image
• Timescale

8
Regions of interest
9
Regions of interest
10
Regions of interest
11
Regions of interest
12
Timescale

• Discrete or continuous?
• Every point sampled by the eye tracker?
• Every nth millisecond?
• Every fixation?

13
Timescale

• Nearby fixations grouped together
• Saccade points rejected

14
What kind of SP?

• Discrete state, usually with regions of interest
  that contain current point of gaze as state
• Discrete time, with points of meaningful
  fixations forming the time step
• Time-homogenous Markov chain

15
White et al. mammograms

• How do experts read mammograms?
• Can experts patterns in reading mammograms
  suggest means of computer aided diagnosis?

16

• from http//sprojects.mmi.mcgill.ca/mammography/im
  ageanalysis.htm

17
White et al.s modeling

• One model per subject, per image
• A sequential clustering algorithm to determine
  regions of interest
• Exclude transitions to same state, and fixations
  outside of regions of interest
• Not enough data

18
Joyce face perception

• Goal augment or corroborate eyewitness
  testimony, using
• eye movement
• EEG
• GSR
• Task
• view a face
• recognize it as not novel, after a delay

19

• Task Study this face

20

• TaskHave you seenthis face before?

21
Joyces regions of interest

• 10 ROI
• hair
• right eye
• nose
• etc.
• Normalized faces
• Nearest neighbor

22
Joyces model

• Markov
• States each of 10 regions of interest
• Time from entering region till leaving region is
  one discrete step

23
Joyces results

• More entropy in saccade sequences when viewing a
  novel face
• Matches qualitative findings

24
Eye-typing Salvucci

• Goal have users type a pre-selected word, by
  looking at an on-screen keyboard
• Models used
• saccade or fixation HMM
• letter HMM
• grammar

25
Saccade-fixation HMM

• If previous time is fixation, current time is
  most likely fixation
• If low velocity, current time most likely
  fixation
• Standard HMM parameter estimation (Rabiner,
  1989)
• Viterbi to optimize

26
Fixations and centroids
27
Centroid submodel HMM

• States
• peaked bivariate distribution for (x, y)
• diffuse bivariate distribution for (x, y)

28
Grammar for words

• Which word gives the highest likelihood of the
  centroid data?

29
Results

• 92 accuracy with 1000 word vocabulary
• too computationally intensive for realistic-size
  (e.g. 50,000 words) vocabulary, for next few
  years
• a nice proof of concept
• (previous systems required gt750ms between
  fixations, and 4 degrees between targets)
• (for simpler multiple-letter models, use a
  string-edit distance algorithm to find the
  nearest vocabulary word)

30
Future work

• More temporal dependence try 3rd order model
• Can a sophisticated string-edit distance
  algorithm correct for bias in limited sampling?

31
Future work

• Systematic exploration of time index
• Improvement of saccade-fixation HMM
• Distribution of fixations within regions
• What is the nature of individual differences?
• Relationship of knowledge and goals to task

32
Sources Cited

• White, KP Hutson, TL Hutchinson, TE (1997).
  Modeling human eye behavior during mammographic
  scanning preliminary results. IEEE Transactions
  on Systems, Man and Cybernetics, A, 27, 494-505.
• Salvucci, DD (1999). Inferring intent in
  eye-based interfaces tracing eye movements with
  process models. Proceedings of the 1999 Computer
  Human Interaction Conference.
• Joyce, CA (2000). Saving Faces Using Eye
  Movement, ERP, and SCR Measures of Face
  Processing and Recognition to Investigate
  Eyewitness Identification. Ph.D. dissertation,
  University of California, San Diego.