Redesigning an OWL: Usability Testing for Writing Centers

1
Redesigning an OWL Usability Testing for
Writing Centers

• CCCC 2007, Session E.02
• New York, NY

2
Panel Overview

• General info about Purdue OWL redesign
• Methodologies and issues
• User-centered theory for OWLs
• Preliminary findings and gendered patterns of
  searching
• Writing center administrators perspective and
  how to prioritize user feedback
• Webmaster as respondent

3
Kaizen as Guiding Principle

• Barnes, Tony. Kaizen Strategies for Successful
  Leadership. Financial Times/Prentice Hall, 1995.
  As presented by Ryan Weber.

4
Brief History of the OWL

• 1994 gopher retrieval of handouts
• 1995 first website launched
• 2000 website redesign
• Text-heavy and print handout-based
• Hypertext workshops and PowerPoints
• Content for multiple audiences mixed
• 2004 new OWL family of Sites and planning for
  usability testing begins

5
OWL Users

• Students and instructors at Purdue
• Students and instructors elsewhere
• English language learners from more than 125
  countries
• Community-at-large

6
Timeline of Usability Testing
7
OWL Usability Testing Methodologies
Issues Dana Driscoll Ph.D. Candidate, Rhetoric
Composition OWL Coordinator Purdue University
8
OWL Usability Testing Methodologies
Issues Dana Driscoll Ph.D. Candidate, Rhetoric
Composition OWL Coordinator Purdue University
9
Usability Methods Issues

• Why usability testing?
• What resources are available?
• What do you want to test?
• What do you hope to learn?
• What will be done with the information once you
  learn it?

10
Usability Testing Resources

• Purdue OWL coordinator/technical coordinator
• Purdue Writing Lab administrators
• Professional Writing faculty and courses
• Graduate Rhetoric and Composition students and
  courses
• Computer labs for testing
• Participant compensation
• Undergraduate Professional Writing students

11
Deciding What How to Test

• Navigation?
• Content?
• Visuals?
• Ease of use?
• Comparison of different groups?
• User-centered design?

12
Focus of Testing Stages

• Generation 1 testing
• User preferences
• User design
• Site testing with live OWL
• Feedback on site

• Generation 2 testing
• Generation 1 testing procedures
• Site testing based on Live OWL and with
  user-centered prototypes

13
Demographics

• Gender, age, language
• College level, major
• Frequency of visits to the OWL, Old OWL, new OWL
• Comfort with technology and writing
• Searching styles

14
Paper Prototyping

• Solicited responses from users in three steps
• Included a number of tested variables inclusion
  of additional resources, placement on pages,
  icons, and search bar

15
Build-Your-Own

• Participants constructed pages from a number of
  choices.
• Research team questioned participants about their
  designs.

16
Onsite Task-Based Testing
17
Data Analysis

• Overwhelming amount of information
• Qualitative vs. Quantitative results
• Deciding what to quantify
• Deciding what groups to compare

18
Using the Data

• Design decisions
• Problem areas identified - but how to fix?
• Results led to further testing
• G2 with user-based tasks
• Remote testing
• Additional usability testing in the future with
  diverse populations

19
What Weve Learned

• Be clear on what you want to know.
• Use a mix of activities and collect both
  qualitative and quantitative data.
• Realize that usability testing, like other types
  of research, is an ongoing, cyclical process.

20
User-Designers User-Centered Theory for OWLs H.
Allen Brizee Ph.D. Candidate, Rhetoric
Composition Purdue University
21
Overview of User-Centered Theory

• Emphasizes people, not technology
• Empowers users, the audiences of technology
• Helps users find key information quickly
• States ideas and main points at the beginning
  rather than middle or end of communications
• Uses headings, concise prose, and lists to guide
  users to the information they need

22
Stakeholders in User-Centered Theory
Interface
Users(s) Situation
Artifact/ System
Users(s)
Designer(s)/ Artisan(s) Image
23
System-Centered Vs. User-Centered

• How do we determine users needs?
• Conduct usability tests before, during, and after
  development

System-Centered Design
User-Centered Design
Designer(s)/Artisan(s) Image
Artifact/ System
Artifact/ System
User(s) Situation
Interface
Designer(s)/Artisan(s) Image
Interface
User(s)
User(s)
24
Pilot Testing

• English 515 course (grads and undergrads)
• Findings
• Participants didnt understand new OWL resources
• Participants were sometimes confused using the
  OWL
• Conclusions and Recommendations
• Further tests needed to refine data
• Further tests needed to refine testing

25
G1 Testing

• Eighteen randomly gathered undergrads, grads,
  faculty, staff at Purdue.
• Demographic information helped us learn about
  users, form categories for analysis

26
G1 Findings by Task

• Task 1a, Choose a Paper Prototype
• Task 1b, Create a Paper Prototype
• Majority of participants preferred navigation bar
  on left
• Majority of participants wanted a search function
• Majority of participants preferred designs
  incorporating logos associated with separate
  areas of OWL Family of Sites

27
G1 Findings by Task

• Task 2 Site Usability Test (times)
• Many tasks took participants one minute or less
  to complete
• Alarming number of tasks took participants one
  minute or more to complete
• Ten tasks took five or more minutes to complete
• Four tasks could not be completed at all
• The shortest period necessary for participants to
  complete a task was thirty seconds
• The longest period required for participants to
  complete a task was ten minutes

28
G1 Findings by Task

• Task 2 Site Usability Test (mouse clicks)
• Many tasks required three (industry standard) or
  fewer clicks to complete
• Alarming number of tasks took four or more clicks
  to complete
• Fourteen tasks took ten or more clicks to
  complete
• The lowest number of clicks was two
• The highest number of clicks was 29

29
G1 Findings by Task

• Task 2 Site Usability Test (feedback survey)
• Participants liked and found useful content and
  design elements of the redesigned OWL
• However, participants did not respond in the
  positive manner expected with the new OWL
• Neutral to neutral-easy responses when asked
  about the ease of finding information on the OWL
• Easy responses when asked about navigation
• Neutral responses when asked about where they
  were while using the OWL
• Neutral-comfortable response when asked if they
  were confused while using the OWL

30
G2 Testing

• Fourteen randomly gathered undergrads, grads,
  faculty, staff at Purdue.
• Demographic information helped us learn about
  users, form categories for analysis

31
G2 Findings by Task

• Task 1a, Choose a Paper Prototype
• Task 1b, Create a Paper Prototype
• All participants preferred navigation bar on left
  (vs. a majority from G1)
• All participants wanted a search function (vs. a
• majority from G1)
• Half of participants preferred designs
  incorporating logos associated with separate
  areas of OWL Family of Sites (vs. a majority from
  G1)
• All participants wanted a most popular resource
  box
• Most participants wanted resources broken down by
  audience (supports our taxonomy recommendations)

32
G2 Findings by Task

• Task 2 Site Usability Test (times)
• Results were essentially the same as G1
• Task 2 Site Usability Test (mouse clicks)
• Results were essentially the same as G1
• Task 2 Site Usability Test (feedback survey)
• Results were essentially the same as G1

33
G2 Findings by Task

• User-Centered OWL Prototype Results
• Time Participants took less time to complete
  tasks with the user-centered OWL than with the
  redesigned OWL site
• 97 second vs. 105.5 seconds
• Mouse clicks participants had a lower average
  number of mouse clicks using the user-centered
  OWL than the redesigned OWL
• 4.89 vs. 6.09

34
G2 Findings by Task

• User-Centered OWL Prototype Results
• Feedback survey
• Participants rated experiences using the
  user-centered prototype more positively as a
  whole
• Organization (which the user-centered OWL
  prototype targeted) was significantly rated
  higher by participants when it was user-based

35
G1 G2 Recommendations

• Design links, pages for the types of visitors
  using the OWL (user-based taxonomy)
• Move the navigation bar from the right side to
  the left side of the OWL
• Add a search function
• Incorporate logos in the OWL Family of Sites
  homepage
• Continue testing to measure usability and to
  generate new ideas for design and content

36
G1 G2 Outlines
37
G1 G2 Screenshots
38
G1 G2 Screenshots
39
Preliminary Findings Patterns of Searching and
Learning Related to Gender and Prior
Usage Morgan Sousa Ph.D. Candidate, Rhetoric
Composition Purdue University
40
Participants Gender Breakdown

• First Generation Usability Testing (G1)
• 13 males
• 5 females
• Second Generation Usability Testing (G2)
• 6 males
• 8 females

41
Gender-Based Searching Styles

• G1 results suggested gender-based differences in
  searching styles
• G1 and G2 revealed additional information
• Clicks
• Time
• Perceptions

42
Overall Conclusions about Gender

• Female users
• required more clicks and time to complete tasks
• felt significantly less comfortable than males
  using the site
• reported higher levels of difficulty when
  navigating the site
• Differences originate in usage patterns
• More research is needed

43
Participants Prior OWL Usage

• G1 Participant OWL Usage
• 11 previous users
• 7 new users
• G2 Participant OWL Usage
• 11 previous users
• 2 new users
• 1 user did not respond

44
Overall Conclusions about Usage

• New OWL users reported having neutral opinions
  toward the site
• Previous OWL users reported that the redesigned
  OWL is stronger and more successful than the
  original
• Differences in usage will be a prime focus in
  future generations of testing

45
Grouping, Alignment Wasted Space Prioritizing
Users Feedback Tammy S. Conard-Salvo Associate
Director, Writing Lab Purdue University
46
WC Administrators Goals

• Develop a next-generation OWL with resources that
  are
• Information-rich
• Technology-rich
• Accessible
• Meeting multiple users needs and expectations

47
Administrators Role in Testing

• Acquiring usability expertise through
  collaboration
• Participation in testing with multiple roles
• Involved client
• Co-Principle Investigator
• Testing administrator
• Observe results first-hand

48
Usability Recommendations

• Develop user-based taxonomy and navigation
• Shift navigation to left side
• Add a search function
• Incorporate graphical logos on splash page
• Continue testing to measure usability and to
  generate new ideas for design and content

49
Prioritizing Users Feedback

• Which revisions are most important?
• Organization and navigation
• Appearance
• How do changes reflect the mission and identity
  of the OWL?
• Intra-program and user-based collaboration
• Stuart Blythe revisited
• OWL and the Writing Lab as research site

50
Webmaster Responds Karl Stolley Ph.D. Candidate,
Rhetoric Composition OWL Webmaster Purdue
University
51
QA

• Michael Salvo salvo_at_purdue.edu
• Dana Driscoll driscodl_at_purdue.edu
• H. Allen Brizee hbrizee_at_purdue.edu
• Morgan Sousa msousa_at_purdue.edu
• Tammy Conard-Salvo tcsalvo_at_purdue.edu
• Karl Stolley stolley_at_purdue.edu