Usability Studies

1

• Usability Studies
• at the University of Windsor
• Web Services Group
• IT Services

2
Usability Studies

• Environmental scan ...
• typical university Website
• static pages, dynamic pages, forms
• home-grown CMS
• partners outside of IT Services - PAC, CFL
• most pages maintained by IPers
• concern about performance
• look to us for leadership direction

3
Usability What is it?
What is Usability? Usability is a quality
attribute that assesses how easy user interfaces
are to use. The word usability also refers to
methods for improving ease-of-use during the
design process. Jakob Nielsen
4
Usability What is it?

• Learnability
• Can users accomplish basic tasks the first time?

How is Usability defined? Five quality
attributes
5
Usability What is it?

• Learnability Can users accomplish basic tasks
  the first time?
• Efficiency
• How quickly can users perform the tasks?

How is Usability defined? Five quality
attributes
6
Usability What is it?

• Learnability Can users accomplish basic tasks
  the first time?
• Efficiency How quickly can users perform the
  tasks?
• Memorability
• How easily is proficiency re-established upon
  return?

How is Usability defined? Five quality
attributes
7
Usability What is it?

• Learnability Can users accomplish basic tasks
  the first time?
• Efficiency How quickly can users perform the
  tasks?
• Memorability How easily is proficiency
  re-established upon return?
• Errors
• How many errors do users make? Are they severe?

How is Usability defined? Five quality
attributes
8
Usability What is it?

• Learnability Can users accomplish basic tasks
  the first time?
• Efficiency How quickly can users perform the
  tasks?
• Memorability How easily is proficiency
  re-established upon return?
• Errors How many errors do users make? Are they
  severe?
• Satisfaction
• How pleasant is the design to use?

How is Usability defined? Five quality
attributes
9
Usability Studies What are they?

• What is a Usability Study?
• experimental evaluation
• interaction between people and products
• a process to find problems and suggest
  improvements
• What information do we collect?
• Verbal assessment / feedback
• count of clicks required to find information
  or complete a task
• records reactions to designs and layouts (video
  and audio)

10
Usability Studies Why?
Why do a Usability Study? 1. measure current
Website performance2. find ways to improve
performance3. silence the critics (have some
data to argue with)
11
Usability What resources?
What resources did we use?

• Co-op student
• Equipment
• laptop with spy-ware
• (keystroke monitoring program - 75.00)
• tape recorder
• Volunteers / subjects
• prospective students, parents (HeadStart)
• bribes

12
Comparisons Usability Study

• What did we do?
• compared our Website to two other universities
• (No, we wont tell you who they were.)
• measured nine predefined (common) tasks

13
Graph it Usability Study
14
Recommendations Usability Study

• Example - two recommended changes ...
• changed Campus Directory to Find a Person
• added Lancer Athletics to Quick Links

15
Recommendations Usability Study
16
Searching Usability Study

• Find a Person
• 60.8 seconds before vs 58.9 seconds after
• Before
• 5 different methods used to search for a person
• After
• 88 used Find a Person in Quick Links

17
Go Lancers! Usability Study
Lancer Athletics to Quick Links resulted in
dramatic improvements to success rates.
Time, in seconds, required to find Lancer
Athletics (Task 5) Original
study Follow-up


Mean 52 24 Median 42 18 Mode 120 15 M
in 18 9 Max 120 66 SD 35.342 16.852 n
12 10
18
Learning Usability Study
What did we learn?

• Equipment was cumbersome
• laptop computer / poster board / release forms,
  etc.
• tape recorder (audio too quiet, couldnt share)
• keystroke monitor program (spy-ware)
• Spy-ware not designed for this
• times and urls were sometimes missed
• url didn't change between pages
• page not recorded if the title of the page
  didn't change
• anchor links were used on the page
• page was not loaded fully
• non-supported browser
• difficult to extract the time of each task in
  seconds
• difficult to find starting and ending url

19
Still Learning Usability Study

• What did we like?
• going to the volunteer
• opportunity for feedback from other visitors
• methodology for a comparison study
• getting numbers

20
Changes Usability Studies

• What did we change?
• defining goals for a Website
• testing the degree goals are achieved
• volunteers now come to our office
• we didnt use bribes this time
• acquired Morae from TechSmith Corporation
• (replaced the spy-ware) - approx. cost 1500
• - includes camera, microphone, advanced
  documentation, upgrade plan, i.e. the deluxe
  package

21
Morae - Components

• 3 Components
• Recorder
• Remote Viewer
• Manager

Testing Post Test
Records audio, video, keystokes, mouse clicks and
screen captures in a digital record
22
Define a Process Usability Testing

• What is the process?
• Step 1 identify (with client) the goals of the
  Website
• Step 2 find approximately 12000 volunteers
• Step 3 perform the usability study
• Step 4 crunch data to identify strengths
  weaknesses
• Step 5 suggest / make any changes
• Step 6 follow-up testing
• Step 7 further changes

23
Questions - Usability Testing
24
Questions - Usability Testing

• What questions do you ask?
• Please find this Website.
• What is your first impression?
• Why?
• Where does your eye go first?
• What info do you think can be found here?
• Please find a, b, c, d

25
Questions - Usability Testing
Sample Display Morae Video Clip
26
Usability Testing - Sample
27
Usability Studies Lessons Learned

• What did we learn about Morae?
• Good
• no need to be near volunteer
• recording process easy
• produce a video
• creates chartable data
• Bad
• harder to count clicks
• subjective to moderators opinion

28
Usability Studies Lessons Learned
Sample Display Morae Manager Screen
29
Samples - Usability Testing
30
Usability Studies Lessons Learned

• What did we learn about testing?
• test, test, test for a great site
• iterative process
• 1 volunteer is 100 better than 0
• 1 volunteer early on is better than 50 near the
  end
• The point of testing is not to prove or disprove
  something. Its to inform your judgment.

31
Usability Studies Lessons Learned

• What other lessons did we learn?
• the importance of recruiting representative users
  is overrated (Krug, 2000)
• little changes can make big differences
• nothing beats a live audience reaction
• keep the report to the client simple
• - use graphs, not stats

32
Usability Studies Lessons Learned

• What are the problems issues?
• defining goals - engage owners, visitors?
• recruiting volunteers bribes, participant
  pools, news, announcements what works best?
• process takes time - anything quicker or cheaper?
  
• frustrated developers / administrators have to
  wait
• setting priorities testing order
• centralized / decentralized testing
• other software / equipment that works better

33
Thank you

• Questions?
• http//www.uwindsor.ca/usability