Usability testing

1
Usability testing

• Bernard and Belindas route to information

2
Background

• Personal philosophy of continuous improvement
• Previous experience - set sights on website

3
History

• Pre-2002 - website developed organically
• October 2002 - Subject pages. Launched Christmas
  2002

4
(No Transcript)
5
(No Transcript)
6
History

• Pre-2002 - website developed organically
• Christmas 2002 - Subject pages.
• April 2003 - total re-design prompted by
  navigation difficulty (structure)

7
Process of 2003 re-design

• Map all content on current site
• Gather ideas from library staff for additional
  content (BRIO)
• Information architecture
• Best practice in design (literature and existing
  UK library websites)
• Ease of use - intuitive

8
Results of research

• New narrow deep structure
• Consistency from templates
• One page per concept and one concept per page
• Branding
• 3 clicks from homepage
• Direct access for important pages

9
Process of creation

• Present research findings and recommendations to
  WDG and SLLs
• Design work by Print Room
• Set up website (empty content)
• Allocate pages to SLLs (training)
• SLLs write content (230 pages)
• Launch October 2003

10
We now had

• Structure
• Layout
• Navigation
• Design and branding
• Consistency
• Manual
• Devolved responsibility

11
(No Transcript)
12

• but the best laid plans

13
University and FOI

• Central control on externally presented
  information
• Split external and internal websites
• External - equivalent of prospectus PR
• External use Content Management System
  (internally developed)
• Intranet - University members only

14
The good

• Member of 4 person University project group
• University realised size of Library website
• University accept Library website functions as a
  portal
• Long lasting links with people involved

15
The bad

• Project started November 2003 launched Easter
  2004
• Beta version of CMS
• Guinea pigs for CMS
• Lots and lots of work

16
(No Transcript)
17
Consequences

• Internal site mostly the same
• URL change - marketing group
• Search tool and link checker no longer work
• Vast number of complaints
• 90 complaints to University The library has
  disappeared
• Higher prominence (quick links internal and
  external webpages)

18
(No Transcript)
19
All these improvements

• Pre-2002 - website developed organically
• Christmas 2002 - Subject pages.
• October 2003 - total re-design
• Easter 2004 - Creation of external site URL
  change
• Plus the day job.
• And still not happy!
• So we decided to undertake usability testing.

20
Why usability testing?

• Continuous improvement
• Not actually asked the people who matter - the
  users of the website
• Marketing Group - On campus and online

21
Usability testing method choice

• Inspired by SCONUL / LIRG Impact project
• Structured observation
• Quick and easy
• Interesting and unexpected results
• Engage with users

22
Structured observation methodology

• Ask participants to undertake a particular task
• Observe how undertake task - route used
  hesitation where click how quick ask to
  vocalise decisions made
• Ease of recording - pre-specified tick sheet for
  most common route etc
• Note comments

23
Pre-testing nerves

• Unsure about methodology
• Still not happy with website - knew it could be
  better still
• Worried
• Davina had an idea

24
Personas

• Represent key sectors of the university
• Names and personal details are given to them to
  make them realistic and authentic to real users
• Using this persona we would identify the
  information they would be looking for and see
  whether it is available and easily found

25
Timeline

• 12th February 2004 attended a training session
  held by UKOLUG at CILIP headquarters
• 20th May 2004 pitched the idea to the Web
  Development Group
• 12th August 2004 personas created and library
  staff attached
• 21st February 2005 arranged a meeting for all
  staff involved
• Spring/Summer 2005 testing
• July 2005 summary of results circulated and
  action points begun
• November 2005 Most action points completed. 2
  personas were abandoned

26
Heuristic evaluation

• Producing a list of points that all pages should
  conform to
• Examples
• All pdfs have an image next to them
• All titles are same size font
• This was done in parallel to the persona testing

27
Categories of users

• Current Undergraduate Full time campus based
  (Bertha)
• Current Postgraduate Full time campus based
  (Bartholomew)
• Current Academic Full time campus based (Bert)
• Current Undergraduate Full time campus based
  (just returning to education) (Basim)
• Current Postgraduate Full time campus based (just
  returning to education) (Becky)
• Current Postgraduate Distance learner (Bernard)
• Current Undergraduate Part time (Bethany)
• Current Undergraduate Placement (Binti)
• Current Postgraduate Part time (Belinda)

28
(No Transcript)
29
Current Postgraduate Distance learner Name Bern
ard Age 28 Subject studied Building
services Level Postgraduate Home Leeds Educa
tion Undergraduate degree in Iceland Library
Knowledge Never used a UK library but used them
in Iceland Frequent sources of information
Internet and the books he orders from
Amazon Knowledge of sources of information Books
and technical reports Use of campus Never Job
Building firm full time
30

• Web usage Daily
• Web competency High
• Hobbies Ice hockey
• Children Two children under 5 years old
• Problems that need overcoming at university
• Finding it difficult to find the technical
  information when
• searching google
• Frustrated that he needs to buy all his books
• Only able to work on his assignments in the late
  evening
• Would like to receive more help with his
  searching as he
• thinks he is missing out on things

31
Current Postgraduate Part time Name Belinda Age
25 Subject studied Education Level Postgr
aduate Home Twickenham Education Undergraduat
e degree at Edinburgh Library
Knowledge Attended a library induction and used
the Edinburgh library Frequent sources of
information Books and electronic
journals Knowledge of sources of
information Aware books and journals Use of
campus Occasionally Job Primary school
care assistant
32

• Web usage Every other day
• Web competency High
• Hobbies Photography
• Children No
• Problems that need overcoming at university
• She is originally from Italy and finds it
  difficult to read information quickly
• She is about to start researching about
  educational psychology and she doesnt
• know what journals to use
• She hasnt got a lot of spare time
• She often uses the Library after the enquiry desk
  is closed

33
Factors we had to consider

• Choice of staff New to Library
• Student on work experience
• No Subject Librarians
• Different organisational strategies
• How to record the route taken and observations
• Choice of questions
• Characteristics of the personas
• Keep a balance between marketing function of the
  website and teaching function

34
Involvement of the Library

• Web development group
• Marketing group
• Formed Jan 2004 and immediately realised the link
  between
• the web and marketing. First action point to set
  up a dummy
• web address which was easy to market
• Subject team

35
Practicalities

• Created documentation
• Arranged a briefing meeting
• Completed a mock example to show people what we
  were looking for
• Set them on their merry way

36
Example

• Assumptions made Not confident with the website,
  time poor, working
• from home, technically poor
• Reads homepage
• Looks through navigation bar and click A-Z
• Reads top paragraph
• Finds historical abstracts
• See no off-campus access so abandons
• .
• Time taken 15 mins

37
Questions

• Access a relevant database for your subject
• Discover the opening hours of the library
• Find a guide on how to use the catalogue
• Each persona then had an individual question to
  complete

38

• Bernard
• Discover whether it is possible to use Leeds
  university library
• Belinda
• Find a way of gaining help without coming to the
  Library
• Bartholomew
• Discover whether he can use a laptop in the
  Library
• Binti
• Find a way of getting material the library
  doesnt have

39
Documentation

• After each task is performed the observer asked
  the persona
• Did they find that easy to find?
• What problems were discovered?
• What assumptions have been made when the page was
  constructed that didnt make sense to your
  persona?
• Was each of the next clicks obvious?
• Any general comments

40
Issues

• Two personas were never observed (Becky Binti)
• People didnt prioritise it as a task
• Not all staff attended the briefing meeting (19
  staff involved)
• Not all observers recorded information in the
  same way
• Time consuming as 2 people involved per persona

41
Issues with method

• People found it hard to forget their previous
  knowledge and competency
• Not always clear that the results really reflect
  the persona involved

42
Results

• Initial report produced July 2005
• Not all evaluation had been completed but wanted
  to make
• significant changes over the summer
• Results analysed for broad changes and individual
  issues for
• each persona

43
Results

• No links between subject databases and A - Z page
• No interaction between catalogue and website
• Information Skill Training and BRIO as headings
  not understood
• Pages for specific groups of users (e.g. Distance
  learners) not appropriately tailored
• Contact Us and Opening Hours difficult to find
• All sorts of problems with finding help

44
Changes

• Information skills section and Enquiry
  section amalgamated into new top level Help
  section
• Services pages subsumed under About the
  Library
• Opening Hours and Contact Us links on homepage
• How to guides on catalogue
• Link between subject pages and A-Z

45
Changes to homepage

• Driven by marketing group
• Remove pictures
• News section
• Did you know unlock content
• Quick links
• Promote Subject pages Research and Help
  sections

46
(No Transcript)
47
Still not happy

• Still not tested views of actual users!
• Always intended to be rolling programme (personas
  a trial run)
• No more nerves
• Method worked v well
• Knew could get good student response (LibQual)
• Difficulties solved by using real people

48
Timeline

• November 2005 WDG decided to continue with the
  testing
• December 2005 Involved marketing group
• February 2006 Involved subject librarians
• March 2006 Testing takes place
• July 2006 Report produced

49
Rolling programme

• Decided to use undergraduates in the first year
  of testing
• Largest cohort of users
• Running focus groups for Research students/staff
• Running a survey for Distance Learners
• Easily identifiable
• Diverse in subject area and level

50
Considerations

• Balance of subjects
• Balance of levels
• Multiple campuses
• Trained / untrained

51
Practicalities

• E-mail sent to them from SITS database
• Library office collated responses
• Standard messages sent to responders
• Back up of reserves
• Timetable of observers
• Choice of room
• Money
• Choice of day

52
Questions

• Rechecked some questions to make sure changes had
  been
• effective.
• Decided to use the same questions for each user
• Access a relevant electronic database for your
  subject
• You are on this page and you need some help, what
  do you do?
• Discover the opening hours for Uxbridge Library
• Renew your books

53

• Asked them initial questions about themselves
• School
• Age range
• Year of study
• Have you had previous training on the intranet
  from the ED or a session run by a librarian?
• Would you rate your internet competency as High,
  Medium or Low?
• How often do you use the Librarys intranet?
• How often do you use the physical library?

54
Lessons learnt about method

• Very quick (too quick)
• Very easy students and observers
• Great response from students
• Admin support vital (over 200 responses)
• SITS sent personal e-mails
• Natural balance of subjects / levels
• E-mail reminder 2 days before slot as Week 1 had
  poor showing
• Uxbridge-centric (MS vouchers in a hurry) Doh!

55
Results

• Renew your books quick and easy - different
  methods
• Quick links well used
• A-Z not helpful in narrowing down choice of
  database
• Untrained not use Subject pages Trained did
  (hurrah!)
• Informational loops (e.g. ILL -gt FAQs -gt ILL)
• Use physical and virtual library equally
• Still problems with Help content
• No search option

56
Changes

• Subjects detail on Databases A-Z list
• Further help at bottom of every information
  page (e.g. ILL)
• Response time on Online Enquiry Form
• Merge FAQ pages (online campus)
• Search software - purchased

57
Accessibility

• Totally re-create site to use CSS for layout,
  design and decoration
• Accessible to all users
• No obvious changes in standard browser

58
(No Transcript)
59
(No Transcript)
60
Rolling onwards

• Library committed to continue with the programme
• Continue to check that the changes we make are
  successful
• Allow more free text comments about the whole
  site
• Publicise across University
• Take confidence to begin other evidence-based
  projects