Gaze-Augmented Think-Aloud as an Aid to Learning

1
Gaze-Augmented Think-Aloud as an Aid to Learning

• Sarah A. Vitak
• Scripps College

John E. Ingram Sewanee University of the South
Andrew T. Duchowski, Steve Ellis, Anand K.
Gramopadhye Clemson University
CHI 2012 5-10 May, Austin, TX
2
Motivation training

• Visual search
• well-defined strategies have been developed by
  experts
• e.g., top-down cognitive strategies based on
  experience
• Chest X-Ray (CXR) inspection
• Airway, Bones, Cardiac silhouette, Diaphrams,
  External tissue, ...

Expert (left) and novice (right) scanpaths over
abnormal CXR.
3
Objectives histology

• Develop histological search strategy training
• Test effectiveness of experts gaze atop video
• scanpath needs to demonstrate experts search
  strategy
• Task identify BrdU marked cells in epithelial
  layer

Example stimuli immuno-gold (BrdU) stained
cross-sections of bovine mammary tissue.
4
Previous work tracking experts

• Ericsson et al. (2006) surveyed experts gaze
• experts search strategies are task dependent
• experts shorter dwell times thought to reflect
  expertise
• experts make better use of peripheral information
• experts patterns of visual analysis develop with
  training
• experts use a larger area around fixation
• experts make better use of extra-foveal
  information

Expert football player (Ronaldo, gaze captured
with Dikablis tracker), expert pilots (Weibel et
al., 2012 ETRA), expert laparoscopic surgeons
(courtesy Stella Atkins and Bin Zheng)
5
Previous work scanpath training

• Fertile research area (see paper for review)
• Selected references related to training
• static, stylized scanpaths (VR)
• Sadasivan et al. (2005)
• dynamic scanpaths (still images)
• Nalangula et al. (2006)
• static scanpaths (still images)
• Litchfield et al. (2010)
• dynamic scanpaths (video)
• Jarodzka et al. (2010)

Sadasivan et al. (2005).
Jarodzka (2010).
6
Contributions GATA

• Gaze-Augmented Think-Aloud (GATA) builds on
• Feed-forward training
• Sadasivan et al. (2005)
• hierarchical task analysis scanpath
  visualization
• Stimulated Retrospective Think-Aloud
• Guan et al. (2006)
• verbalizing while watching scanpath visualization
• Eye Movement Modeling Examples (EMME)
• Jarodzka et al. (2010, 2009, 2010)
• task analysis highlighting of salient regions
  (e.g., foveation)

7
Gaze-Augmented Think-Aloud

• GATA
• recorded scanpath verbalization (audio track)
• a specific visual search strategy is required
• for histology images, devised STAMP
• Staining, Tissue, Artifacts, Magnification, Plane
  of section

Training video used in study.
8
Evaluation

• Experiment
• 2 x 2 mixed factorial design
• (presence or absence of video) x (experienced or
  naive participant)
• video between-subjects, 8 stimuli images
  within-subjects
• task identify cells marked with BrdU in
  epithelial layer
• procedure both groups saw training slides w/out
  scanpath

Training slides the first highlights marked
cells, second differentiates tissue type, third
shows epithelia with no marked cells, fourth
again shows marked cells in the epithelia, fifth
shows no marked cells same verbalization was
heard as in GATA video.
9
Evaluation

• Experiment
• apparatus Tobii ET-1750 (see paper for specs)
• participants 32 (aged 19-57,
  median 22)
• 15 experienced, 15 naive
• 2 participants dropped (no data)
• dependent measures
• speed (time to task completion)
• accuracy (hits misses)
• fixation counts
• fixation durations
• outliers beyond 3 SD
• one removed (from AVS)

Example of participant apparatus half the
participants (experienced) recruited from Animal
Veterinary Science (AVS) class who were
familiar with tissue images.
10
Dependent measures

• Speed
• time to task completion
• tabulated per each image, then averaged
• Accuracy
• hits (true positives)
• misses (false positives)
• Fixation counts
• Fixation durations
• longer durations suggest cognitive load (e.g.,
  difficulty)

Example of two captured scanpaths blue is from
experimental group, red is from control group.
11
Results speed

• Time to completion
• two-way ANOVA revealed significant effect of
  video
• F(1,6)9.25, plt0.05)
• but not of population group
• F(1,6)0.01, p0.92, n.s.)
• those who saw video
  performed significantly
  faster in either
  group

12
Results fixations

• Number of fixations
• two-way ANOVA revealed significant effect of
  video
• F(1,6)8.93, plt0.05)
• but not of population group
• F(1,6)0.02, p0.90, n.s.)
• numbers of fixation
  closely mirror
  
  time to completion
• not surprising as both are
  often correlated

13
Results fixation durations

• Fixation durations
• two-way ANOVA revealed no significant effect of
  video
• F(1,6)0.00, p0.56, n.s.)
• nor of population group
• F(1,6)0.00, p0.98, n.s.)
• durations not expected
  to differ between groups
• within-group analysis
  shows significant
  
  difference by naive group
  but not
  experienced group

14
Results accuracy

• Considering only effect of video on accuracy
• one-way ANOVA revealed significant effect on
  error rate
• F(1,27)10.15, plt0.01)
• but not on correct responses
• F(1,28)1.21, p0.28, n.s.)

15
Discussion

• Results suggest fairly clear explanation
• those who viewed scanpath were faster with fewer
  errors
• effect more pronounced in experienced group
• corroborates to a certain extent by Litchfield et
  al. (2010)
• Eye movements provide evidence of performance
• numbers of fixation correlate with time
• might not be obvious had stopwatch been used
• fixation duration may indicate expertise
• shorter fixations suggest faster recognition
  (familiarity with task)

16
Conclusion

• Gaze-Augmented Think-Aloud
• simpler than more elaborate visual codifications
• e.g., Sadasivan et al. (2005)
• scanpath indicates what to look for and what to
  avoid
• cost-benefit ratio is increased through
  simplicity
• likely to be effective for CXR training

Current training at local hospital from left to
right, intern, resident, radiologist.
17
Acknowledgments

• This work was supported by the US NSF
• Research Experience for Undergraduates grant
• Thank you!
• Questions?