Getting a Measure of Satisfaction from Eyetracking in Practice Workshop 24

1
Getting a Measure of Satisfaction from
Eyetracking in PracticeWorkshop 24

• Organisers
• Tony Renshaw, Leeds Metropolitan University, UK
• Natalie Webb, Amberlight Partners, London
• Janet Finlay, Leeds Metropolitan University, UK

2
Aims

• Initially
• Define best practice in eye tracking
• Suggest solutions to problems
• Highlight unanswered questions
• Explore both scientific and commercial use
• Explore how to measure satisfaction with
  eyetracking
• Adjusted through the day!

3
Format of Workshop

• Presentations 3 minutes per person identifying
  3 key issues from position papers
• Participants note issues and questions of
  interest from presentations on post-its
• Participants organise their questions under
  themes for discussion and choose groups
• Break out groups 3 AM, 3 PM
• Presentations from break out groups
• Areas of agreement
• Unresolved issues
• Future Work
• Summary key take away points

4
Breakout Sessions

• 6 themes
• When should we use/not use eyetracking?
• What are the best eye tracking methodologies?
• How do we make eye tracking data analysis
  manageable?
• How can we analyse gaze paths?
• How can we tie eye tracking metrics into
  measuring user experience?
• How do we deal with dynamic stimuli?

5
Participants Representing 6 countries and 3
continents Representing both research and
commercial users

• Sune Alstrup
• Duncan Brumby
• Edward Cutrell
• Andrew Duchowski
• Laura Granka
• Ying-Hua Guan
• John Hansen
• Keith Karn

• Craig Lindley
• Janet Read
• Jens Riegelsberger
• Alain Robillard-Bastien
• Kerry Rodden
• Anthony Santella
• Charlotte Sennersten
• Katerina Tzanidou

6
1. When should we use/not use eye tracking?

• Areas of agreement
• Use when appropriate but not always!
• Context and purpose of project is important and
  very different between commercial and research
• Should always consider complementary techniques
• Should recognise and work within the limitations
  of the technologies
• Unresolved issues
• How can we best combine eye tracking with other
  methods
• How much influence and interference?
• Replay gaze trails ? post hoc rationalisation?
• Concurrent think aloud ? alter eye movement
  behaviour?
• Dealing with limitations from mind-eye hypothesis
• Common reporting standards needed e.g. providing
  full information about calibration and people who
  could not be tracked
• Future work
• Investigate individual and population differences
  in eye gaze
• Better understand the influence of sampling on
  eye tracking studies
• Find ways to combine physiological and eye
  tracking data

7
2. What are the best eye tracking methodologies?

• Areas of agreement
• Must have a list of methods in order to provide
  best practices
• Cannot make sense of data external to intentions
• Very different intentions for different research
  goals (usability vs cognitive models)
• Unresolved issues/Future work
• Refining eye tracking methodologies
• Further detailing the best usage of retrospective
  reporting vs. concurrent think aloud

8
3. How do we make eye tracking data analysis
manageable?

• Areas of agreement
• Plan carefully with a clear view of purpose,
  problem being addressed and tasks
• Choose appropriate numbers of participants, task
  sizes and session lengths
• Select appropriate fixation parameters (duration
  and area)
• Unresolved Issues
• Analysis software provided with eye tracking
  hardware not adequate need better facilities
  for
• Filtering
• Summation
• Vizualization
• Improving ways to combine data with other
  protocols e.g. verbal
• Studies involving different applications have
  different analysis requirements e.g. games,
  learning systems
• Future Work
• Can borrow analysis algorithms from other areas
  e.g. bioinformatics, machine learning
• Need Open Source analysis tools
• Need more publications focusing on methodology
  and analysis techniques

9
4. How can we analyse gaze paths?

• Areas of interest
• Emerging standard
• E.g. String Edit Distance
• Unresolved issues
• Dealing with
• Strings of varying lengths
• Comparison of multiple strings
• Visual presentation of analysis
• Very long sequences
• Integration of data from other sources
• Annotation of edited strings with comments
• Future Work
• All of the above!

10
5. How can we tie eye tracking metrics into
measuring user experience?

• Areas of agreement/Unresolved Issues
• Definition of satisfaction is problematic
  depends on domain engagement or immersion may
  be more useful
• Currently lack of models connecting eye tracking
  metrics and higher level measures of user
  experience
• Would like to be able to correlate particular
  behaviours with measures
• Might be more helpful to approach this from
  behavioural rather than cognitive perspective
• Future work
• Need more models linking eye behaviour to user
  experience e.g.
• Novice vs. expert behaviour
• Certain scan paths indicating a certain depth of
  engagement (e.g. in games)
• Using eye movement behaviour to inform design
  through design patterns

11
6. How do we deal with dynamic stimuli?

• Areas of agreement
• Very little known!
• Unresolved issues
• Large variety of stimuli formats e.g. games, web
  applications, java, flash
• Problems of tracking moving AOI (areas of
  interest), definition of dynamic AOI
• Labelling of data is labour intensive
• What granularity of information is required?
• How can scan paths be determined in this type of
  stimuli?
• Further work
• Formulate appropriate cognitive and visual
  influence theories
• Explore role of peripheral vision in dynamic
  environments
• Explore influence of auditory information
• Develop suitable strategies for determining AOI
  online or post hoc
• Extract stimuli information directly from the
  application and integrate with eye tracking data

12
(No Transcript)
13
Outcomes

• Aim 1 Best practice is premature we first
  need a better understanding of possibilities
• Aim 2 Satisfaction seen as a problematic
  construct we need to develop more meaningful
  ways of interpreting eye movement data within
  specific contexts e.g. identifying common
  patterns of behaviour for specific scenarios

14
Wish list

• High level models and metrics
• Better analysis algorithms
• More flexible and powerful analysis tools
• Methodologies for combining eye tracking with
  other usability approaches
• Better understanding of constraints and
  parameters for eye tracking studies in specific
  contexts
• More research!

15

• If you are interested in eye tracking either as a
  researcher or for commercial use, we would like
  to talk to you!
• Please leave us your card in the tray provided
  and/or take one of ours.
• Thanks for your interest.