The Benefits of Using Eye Tracking in Usability Testing

1
The Benefits of Using Eye Tracking in Usability
Testing

• Jennifer C. Romano
• Usability Laboratory
• Statistical Research Division
• U.S. Census Bureau

2
Overview

• What is usability?
• Introduction to eye tracking
• Using eye tracking in usability testing

3
What is usability?

• Defined as the effectiveness, efficiency, and
  satisfaction with which a specified set of users
  can achieve a specified set of tasks in a
  particular environment (ISO/TR 169822002)

4
What is usability testing?

• Usability testing is concerned with assessing how
  usable a product is.
• Can users interact with the product in an
  effective, efficient, and satisfying way?
• Evaluation at any stage of development
• Iterative testing

5
Why do usability testing?

• Design often does not take the users mental
  models into account.
• Users have their own cognitive limitations and
  capabilities.
• Test the product on the end user.

6
Usability Evaluation Benefits

• Satisfied users
• Improved data quality
• Less measurement error
• Reduced training time

7
Usability Testing Measures

• Effectiveness
• Accuracy
• Efficiency
• Time to complete a task/survey
• Satisfaction
• Self-rated satisfaction

8
Introduction to Eye Tracking
Photo from www.smith.umd.edu/behaviorlab/eyetrack
er.aspx
9
Introduction to Eye Tracking

• Captures where people look as they navigate
  through a Web site
• Shows an individuals pathway
• Accumulates data and shows common pathways

10
Eye-Tracking Measures

• Fixations
• Time to first look at an area of interest (AOI)
• How often people look at AOI
• Hot spots
• Areas that get most attention
• Gaze plots
• Pattern of search

11
Benefits of Eye Tracking

• Identify areas that attract users attention or
  areas that are difficult to understand
• Identify how long it takes people to look at
  something
• Identify where people look
• Where people look
• Where people never look

12
Questions Eye Tracking Can Answer

• Why does it take users a long time to complete
  the task/survey?
• Why do users have low accuracy?
• Why arent the users satisfied?
• Are people looking atsomething?
• Terminology orsomething else?

13
Usability Study
14
Task 1

• Task You want to use data from two different
  data sources, but are unsure if they are
  comparable. Find out if this information exists
  on the site.
• Solution Get Data or User Guide

15
Task 1 Usability Findings

• Users seemed unclear on where to find the
  information. Five participants (56) selected
  Get Data from the center of the screen, and three
  (33) selected User Guide from the top
  navigation.
• Of the five participants who selected Get Data,
  two successfully completed the task. Of the three
  participants who selected User Guide, only one
  successfully completed the task.
• Accuracy 33
• Efficiency (correct only) 3m12s

16
(No Transcript)
17
Task 1 Hot Spot Findings
18
Task 1 Hot Spot Findings
19
Task 1 Hot Spot Findings
20
Task 2

• Task You are unsure of the difference between
  Places and Metropolitan Areas. Where would
  you find this information on this site?
• Solution Definition ? Geography

21
Task 2 Usability Findings

• Most participants initially scrolled through the
  definitions list, expecting to find the meanings
  of Metropolitan Areas and Places (which were
  not in the listing but rather located on the
  Geography page).
• Two participants never used the Geography link
  and instead navigated to the Site Map to find the
  target information.

22
Task 2 Usability Findings

• Users said that they expected the format to be
  the same as the format of the Definitions page.
• Accuracy 78
• Efficiency (correct only) 4m32s

23
(No Transcript)
24
Task 2 Gaze Plot Findings
25
Task 2 Gaze Plot Findings
26
Fixation Findings
Areas of Interest (AOIs) Areas of Interest (AOIs) Areas of Interest (AOIs) Areas of Interest (AOIs)
Participant Top nav Left nav Center Right nav
1 49s 5s 3s 7s
2 2s 3s 2s 4s
3 45s 34s 6s 20s
4 7m48s, task 3 27s 2s 27s
5 2m2s, task 2 3s 2s 1m20s
6 18s, task 2 15s, task 2 2s 16s, task 2
7 2m2s 20s 2s 2m23s
8 49s 1m50s 8s 9s
Mean 1m43s 24s 3s 34s
27
Conclusion

• Obtain measures that are not obtainable from
  traditional usability measures.
• What people (do not) look at
• How often they look there
• Time into the study to look there
• Quantitative data supports usability findings

28
Conclusion

• Without obtaining this type of data, it is
  impossible to determine where people look on the
  screen and how often they look there.
• Its simply not possible to get data on where
  people looked, how quickly they looked, and how
  often they looked using only conventional
  experimental methods.

29
Conclusion

• Future usability testing should aim to include
  eye tracking as a method of collecting data.

30
Thank you!

• Jennifer C. Romano
• Usability Laboratory
• Statistical Research Division
• U.S. Census Bureau
• [phone]
• Email
• Jennifer.romano_at_census.gov
• LinkedIn http//www.linkedin.com/in/jenniferroman
  o