4th International JISCCNI Conference

1
Service Quality Assessment in a
Digital Library Environment
?

• 4th International JISC/CNI Conference

Edinburgh, Scotland June 27, 2002
Joseph Boykin, Fred Heath, Duane Webster
2
Overview of Discussions

• New Models for Understanding and Describing
  Library Success
• ARLs New Measures Initiative
• LibQUAL Project Development
• Experience with LibQUAL
• Conclusions and Next Steps

3
The Association of Research Libraries
Mission Shaping and influencing forces affecting
the future of research libraries in the
process of scholarly communication. Members 1
23 major research libraries in North
America. Ratios 4 of the higher education
institutions providing 40 of the information
resources. Users 3 million students and
faculty served. Expenditures 2.35 billion
annually, 727 million for acquisitions of
which 9 is invested in access to electronic
resources.
www.arl.org
ASSOCIATION OF RESEARCH LIBRARIES
4
The Problem of Assessment in Research Libraries

• ARL Membership Criteria Index variables emphasize
  inputs, primarily expenditures
• To rise in the ARL Index it is only necessary to
  spend more
• No demonstrable relationship between expenditures
  and service quality
• The lack of metrics describing performance

5
(No Transcript)
6
ARL New Measures Initiative

• Collaboration among member leaders with strong
  interest in this area
• Specific projects developed with different models
  for exploration
• Intent to make resulting tools and methodologies
  available to full membership and wider community

7
ARL New Measures Projects

• Demonstration project for service effectiveness
  measures (LibQUAL?)
• Project to define usage measures for electronic
  information resources (includes institutional
  outcomes)
• Investigation of role libraries play in support
  of the research process
• Investigation of role libraries can play in
  campus learning outcomes activities
• Identification of cost-drivers and development of
  cost-benefit studies

8
Assessment
The difficulty lies in trying to find a single
model or set of simple indicators that can be
used by different institutions, and that will
compare something across large groups that is by
definition only locally applicable i.e., how
well a library meets the needs of its
institution. Librarians have either made do with
oversimplified national data or have undertaken
customized local evaluations of effectiveness,
but there has not been devised an effective way
to link the two Sarah Pritchard, Library Trends,
1996
9
LibQUAL Description

• LibQUALTM is a research and development project
  undertaken to define and measure library service
  quality across institutions and to create useful
  quality-assessment tools for local planning.

10
The Purpose of Our Research

• To fill a knowledge void in modeling the
  dimensions of library service quality from a user
  perspective
• Based upon the model, to develop a web-delivered,
  effective total market survey instrument
  equivalent for service quality assessment in
  academic libraries
• Using the derived instrument to recommend a
  process for an ongoing program of comparative
  outcome measurement for academic libraries

11
LibQUAL Project Goals

• Development of web-based tools for assessing
  library service quality
• Development of mechanisms and protocols for
  evaluating libraries
• Identification of best practices in providing
  library service
• Establishment of a library service quality
  assessment program at ARL

12
Project Resources

• LibQUALTM is an ARL/Texas AM University joint
  effort. The project is supported in part by a
  3-year grant from the U.S. Department of
  Educations Fund for the Improvement of
  Post-Secondary Education (FIPSE).

13
LibQUALTM Team

• ARL
• Duane Webster
• Martha Kyrillidou
• Kaylyn Hipps
• Julia Blixrud
• Jonathan Sousa
• Consuella Waller

• TAMU
• Fred Heath
• Colleen Cook
• Bruce Thompson
• Yvonna Lincoln
• Trey Thompson
• Julie Guidry

14
LibQUALTM Project History
Jan2000
Sept. 1999
July 2000
Sept. 2000
Oct. 2000
Jan. 2001
June 2001
9/99 - ARL launches New Measures Initiative
which includes the study of service effectiveness
known as SERVQUAL spearheaded by Texas AM
University.
1/00 - Initial 12 institutions begin the
SERVQUAL study led by the Texas AM team.
7/00 - LibQUAL as a distinct library-based
assessment tool is presented.
9/00 - ARL and Texas AM awarded a FIPSE grant
to fund further development of the LibQUAL
project.
10/00 - The ARL symposium, New Culture of
Assessment in Academic Libraries Measuring
Service Quality attracts a group of 170 people.
1/01 - Representatives from 43 research and
university libraries participating in the Spring
2001 Implementation meet in Washington, DC
during ALA midwinter. 6/01 - National Science
Foundation awards grant to ARL and Texas AM to
adapt LibQUAL for NSDL

15
LibQUAL Contribution

• Grounded questions yield data of sufficient
  granularity to be of local use
• Normative data across cohort group at first and
  higher order levels
• Surfaces Best Practices
• Web approach makes little demand of local
  resources while compiling robust dataset

16
Relationships perceptions, service quality and
satisfaction
.only customers judge quality all other
judgments are essentially irrelevant Zeithaml,
Parasuraman, Berry. (1999). Delivering quality
service. NY The Free Press.
17
LibQUALTM Participants
Year 3
Year 2
170 Participants
43 Participants
Year 1
12 Participants
Spring 2001
Spring 2000
Spring 2002
For More Information about Participants
Visit the LibQUAL web site.
18
SURVEY INSTRUMENT
19
Dimensions of Library Service Quality
20
LibQUAL Core Questions Y1
__________________________________________________
___________________________
Factor_ ______ _ No. I II
III IV Item Core
_________________________________________________
____________________________ 32 .84947
.12848 .24465 .13335 1 Willingness to help
users 33 .80847 .13662 .25348 .14147 1 Giving
users individual attention 7 .80757 .17881
.12781 .21125 1 Employees deal with users
caring fashion 50 .79273 .19288 .18847 .12497
1 Employees who are consistently
courteous 31 .77262 .16358 .26461 .20061 1
Employees have knowledge answer questions 5
.74072 .14754 .18453 .29624 1 Employees
understand needs of users 3 .74052 .15102
.17296 .20793 1 Readiness to respond to
users' questions 18 .71718 .19757 .18289
.26766 1 Employees who instill confidence in
users 43 .62487 .22402 .29970 .28256 0
Dependability handling service problems   20 .1655
6 .87679 .11430 .16236 2 A haven for quiet
and solitude 2 .17739 .83172 .08498 .13901
2 A meditative place 19 .22362 .83147 .14705
.22566 2 A contemplative environment 25 .16013
.80492 .18894 .16628 2 Space that
facilitates quiet study 41 .20398 .80204
.17599 .20255 2 A place for reflection and
creativity   37 .22528 .12353 .78405 .15466
website enabling me locate info on my
own 28 .19602 .09611 .75780 .13173 elec
resources accessible home or office 14 .33339
.16156 .60389 .31109 access tools allow
me find on my own 45 .30467 .23784 .59090
.28919 3 Modern equip me easily access info I
need 17 .35390 .18467 .55690 .41864 info
easily accessible for independent use 29 .30136
.21018 .55341 .38474 4 Convenient access to
library collections   11 .13494 .23183 .18868
.73636 3 Comprehensive print
collections 39 .14894 .23743 .29367 .60350
3 Complete runs of journal titles 16 .29445
.19831 .22384 .60107 3 Interdisciplinary
library needs addressed 9 .27782 .05333
.16331 .57866 4 Timely document
delivery/interlibrary loan 8 .22850 .18484
.13137 .56343 0 Convenient business
hours ____________________________________________
____________________

21
Affect of Service

• Emerged as the dominant factor early in our work
• Absorbed several of the original SERVQUAL
  questions measuring Responsiveness, Assurance and
  Empathy
• In the current analysis also includes Reliability
• All in all the Human Dimension of Service
  Quality

22
Library as Place

• Transcends the SERVQUAL dimension of Tangibles to
  include the idea of the library as the campus
  center of intellectual activity
• As long as physical facilities are adequate,
  library as place may not be an issue

23
Personal Control

• How users want to interact with the modern
  library
• Personal control of the information universe in
  general and web navigation in particular

24
Access to Information

• Ubiquity of access information delivered in the
  format, location and time of choice
• Comprehensive collections

25
Project Deliverables

• Print and web-based results include
• Aggregate Summaries
• Demographics by Library
• Item Summaries
• Dimension Summaries
• A copy of the survey instrument
• Dimensions measured for survey implementation

26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
(No Transcript)
31
(No Transcript)
32

33
(No Transcript)
34
Two Interpretation Frameworks

• Score Norms
• Zone of Tolerance

35
Score Norms

• Norm Conversion Tables facilitate the
  interpretation of observed scores using norms
  created for a large and representative sample.
• LibQUALTM norms have been created at both the
  individual and institutional level

36
C
37
C
Zone of Tolerance

• The area between minimally acceptable and desired
  service quality ratings
• Perception ratings ideally fall within the Zone
  of Tolerance

38
ZONE OF TOLERANCE DIMENSIONS
39
(No Transcript)
40
LibQUAL Fundamental Contributions to the
Measurement of Effective Delivery of Library
Services

• Determine the degree to which information derived
  from local data can be generalized, providing
  much needed best practices information
• Demonstrate the efficacy of large-scale
  administration of user-centered assessment
  transparently across the web

• Shift the focus of assessment from mechanical
  expenditure-driven metrics to user-centered
  measures of quality
• Re-ground gap theory for the library sector,
  especially academic libraries

41
LibQual 2000 and 2001 Results
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
NSF Grant

• Assess service quality in digital libraries
• 3 year period
• Adopt LibQUAL instrument for use in the Science,
  Math, Engineering and Technology Education
  Digital Library community (NSDL)

47
NDSL LibQUAL Goals

• Define dimensions of digital library service
  quality from the users perspectives
• Develop tool for measuring user perceptions and
  expectations of digital library service quality
  across NSDL digital library contexts
• Identify digital library best practices that
  permit generalizations across operations and
  development platforms

48
NDSL LibQUAL Activities

• 120-200 qualitative interviews to contribute to
  identifying dimensions of digital library service
  quality
• Test and refine dimensions of digital library
  service quality and self-sufficiency through
  development of total market survey
• Implement survey across variety of organizational
  and digital library implementations

49
LibQUALTM Related Documents

• LibQUALTM Web Site http//www.arl.org/libqual/
• LibQUALTM Bibliography
• http//www.coe.tamu.edu/bthompson/servqbib
• Survey Participants Procedures Manual
• http//www.arl.org/libqual/procedure/lqmanua
  l2.pdf

50
LibQUAL? Related Documents

• LibQUAL? Web Site http//www.arl.org/libqual/
• LibQUAL? Bibliography
• http//www.coe.tamu.edu/bthompson/servqbib

woof