Why evaluate?

1
On a scale of 1 to 9. . .
Have you ever . . .
Choose the answer that most closely resembles. . .
How would you rate. . .
What could the library. . .
How strongly do you agree with the statement . . .
2
On a scale of 1 to 9. . .
Getting Using Feedback from ILL Users What
Have We Learned? Presentation by ILL
Subcommittee members Stuart Frazer Old
Dominion University Bud Bennett Radford
University Kevin Unrath Hollins University
Have you ever . . .
Choose the answer that most closely resembles. . .
How would you rate. . .
What could the library. . .
How strongly do you agree with the statement . . .
3
Why evaluate?
On a scale of 1 to 9. . .
Have you ever . . .

• Measure user satisfaction.
• Discover barriers to improving service.
• Learn more about your users.
• See
• Littlejohn, Nancy and Barbara Wales. Assessment
  Plan for Interlibrary Loan Departments at
  Academic Libraries. Journal of Interlibrary
  Loan, Document Delivery Information Supply. v7
  n2, 1996, pgs 3-18.

Choose the answer that most closely resembles. . .
How would you rate. . .
What could the library. . .
4
ILL Surveying the context
On a scale of 1 to 9. . .
Have you ever . . .

• Stand alone survey, or be part of a larger
  library survey.
• Survey complements other evaluation methods
• Analysis of statistics.
• Informal feedback.
• See
• Perrault, Anna H. and Marjo Arseneau. User
  Satisfaction and Interlibrary Loan A Study at
  Louisiana State University. RQ. v35 n1, Fall
  1995, pgs 90-100.

Choose the answer that most closely resembles. . .
How would you rate. . .
What could the library. . .
5
Pitfalls of surveying
On a scale of 1 to 9. . .
Have you ever . . .

• False positives.
• Asking the wrong question.
• No follow-up.
• See
• Adamson, Colin. How to Waste Money Measuring
  Customer Satisfaction. Managing Service
  Quality. v4 n5, 1994, pgs. 9-12.
• Applegate, Rachel. Models of User Satisfaction
  Understanding false positives. RQ. v32 n 4,
  Summer 1993, pgs. 525-539.

Choose the answer that most closely resembles. . .
How would you rate. . .
What could the library. . .
6
Guidelines for questions
On a scale of 1 to 9. . .
Have you ever . . .

• Keep questions to twenty words or less.
• Each question should only ask about one thing.
• Participants wont be sure how to answer
  questions that contain double negatives.
• Be alert to the potential for ambiguity.
• Adapted from
• Stein, Joan. Designing user satisfaction surveys
  for interlibrary loan services. Performance
  Measurement and Metrics. v1 n1, August 1999,
  pgs. 45-61. Available online at
  http//www.aslib.com/pmm/1999/aug/03.pdf

Choose the answer that most closely resembles. . .
How would you rate. . .
What could the library. . .
My librarys interlibrary loan service should
deliver photocopies of articles to me quickly and
electronically
How many times have you used the Librarys
interlibrary loan service in the past month?
My interlibrary loan requests should not take
longer than two weeks to arrive Agree
Disagree
7
Guidelines - continued
On a scale of 1 to 9. . .
Have you ever . . .

• Offer enough response alternatives with
  closed-ended questions.
• Dont overtax your participants memories.
• Avoid hypothetical questions.

Choose the answer that most closely resembles. . .
How would you rate. . .
What could the library. . .
I Most often request the follow type of material
through ILL ____Books ____Articles
______________Other
How many times have you used ILL in the past
year?
8
Guidelines - continued
On a scale of 1 to 9. . .
Have you ever . . .

• Avoid using jargon and acronyms.
• Look out for words with alternative uses or
  meanings.
• Avoid using leading questions.
• Be careful of loaded words or phrases.

Choose the answer that most closely resembles. . .
How would you rate. . .
What could the library. . .
When did you last use the librarys interlibrary
loan request form on the web?
Interlibrary loan service should be free under
any circumstances. Agree Disagree
9
Surveys at VIVA Institutions
On a scale of 1 to 9. . .
Have you ever . . .

• Some participate in LibQUAL
• Most others include interlibrary loan as a part
  of their overall customer service survey.
• Only a few have given separate interlibrary loan
  only surveys to their users.

Choose the answer that most closely resembles. . .
How would you rate. . .
What could the library. . .
10
Overall satisfaction
On a scale of 1 to 9. . .
Have you ever . . .

• Users at VIVA libraries indicated a high level of
  overall satisfaction with interlibrary loan.
• Statewide, 90 of respondents rated ILL services
  as good to excellent.

Choose the answer that most closely resembles. . .
How would you rate. . .
What could the library. . .
11
Statewide Statistics
On a scale of 1 to 9. . .
Have you ever . . .
Choose the answer that most closely resembles. . .
How would you rate. . .
What could the library. . .
12
User comments
On a scale of 1 to 9. . .
Have you ever . . .

• Most useful part of the survey.
• Helped motivate staff.
• Suggested needed changes.

Choose the answer that most closely resembles. . .
How would you rate. . .
What could the library. . .
13
Thank you
On a scale of 1 to 9. . .
Have you ever . . .

• Thanks to the institutions who responded to our
  call for survey data
• Radford University
• Old Dominion University
• University of Virginia
• College of William Mary
• Hollins University
• Virginia Tech
• Southside Virginia Community College
• Northern Virginia Community College
• Hampton-Sydney College

Choose the answer that most closely resembles. . .
How would you rate. . .
What could the library. . .