Preparing to Work Effectively with LibQUAL

1
Preparing to Work Effectively withLibQUAL
Survey Results
Raynna Bowlby Martha Kyrillidou Association of
Research Libraries
ALA Annual Conference Washington, DC June 26, 2007
old.libqual.org
2
Preparing to Work Effectively with LibQUAL
Survey Results

• Prepared
• properly expectant, organized, or equipped
  ready
• Ready
• completely prepared or in condition for
  immediate action or use or progress
• Dictionary.com Unabridged (v 1.1)

3
Desired OutcomesAs a result of our work together
today, you will be prepared to

• Perform some simple analyses of the LibQUAL
  survey results data
• Today well be working mainly with the 22 core
  LibQUAL questions and Comments
• Organize your library colleagues and groups /
  teams to work with LibQUAL results
• Present the results to different stakeholders
• Utilize the data to target areas for improvement
• Begin a process of continuous assessment
• Thank you for helping us beta test this
  workshop we welcome your feedback!

4
After the LibQUAL Survey -- Work Plan

• Understanding the Data and Results
• Interpreting representativeness, zone of
  tolerance, peer comparisons, longitudinal
  analysis, analytics and SPSS
• Developing the Organization
• Confirming survey objectives, clarifying roles
  responsibilities, presenting results, engaging
  others in utilizing data, discussing results
  implications
• Using LibQUAL Effectively
• Targeting incremental improvements, applying data
  in decision-making, implementing a cycle of
  continuous assessment and improvement

5
After the LibQUAL Survey -- Work Plan

• A list of steps describing work that can be
  accomplished after receiving the survey,
  including quantitative and qualitative data
  analysis and organizational development
• Think about
• Which steps might your library undertake?
• Will the assignment be individual or a group /
  team?
• What KSAs might best support the accomplishment
  of this work?
• Who will be assigned?
• (Who will you consult with to make these
  decisions)?

6
Understanding the Data Results
7
Results Notebooks

• Sections for Overall, Undergraduates, Graduates,
  Faculty, Staff, Library Staff include
• Demographic Summary
• Core Questions Summary
• Dimensions Summary
• Local Questions
• General Satisfaction Questions
• Information Literacy Outcomes Questions
• Library Use Summary
• Appendix describing changes in the dimensions and
  the questions included in each dimension.

8
Results

• Results Notebooks (PDF)
• http//old.libqual.org/Manage/Results/index.cfm
• Group Notebook (PDF)
• Unified Notebook (PDF)
• User Comments
• Excel Data File
• SPSS data files (if requested at registration)
  will be e-mailed to the primary contact two to
  three months after the session has closed.

9
DO YOU UNDERSTAND YOUR RESULTS?
10
Representativeness - Standard Disciplines An
Academic Library
Respondents Population
11
Representativeness - Customized Disciplines An
Academic Library
Respondents Population
12
Radar Chart Basics
13
Key to Radar Charts
14
Understanding a Radar Chart
15
What Do the Colors Mean?
Green And Blue
16
What Do the Colors Mean?
Little Yellow
17
What Do the Colors Mean?
A lot of Yellow
18
What Do the Colors Mean?
Red
19
Dimension Summary
20
Understanding Your Individual Results
Thermometer Charts
Key Term Zone of Tolerance
21
Dimension Summary
Key Term Zone of Tolerance
22
Peer Comparisons
Source Jim Self, University of Virginia,
Presented at Performance Measurement in Academic
Libraries Workshop, EBLIP4, Durham, North
Carolina, May 11, 2007
23
Peer Comparisons
Source Jim Self, University of Virginia,
Presented at Performance Measurement in Academic
Libraries Workshop, EBLIP4, Durham, North
Carolina, May 11, 2007
24
Peer Comparisons
Source Fred Heath, LibQUAL Results Meeting,
ALA Annual Conference, Seattle, WA, January 22,
2007
25
Longitudinal Analysis
Source Fred Heath, LibQUAL Results Meeting,
ALA Annual Conference, Seattle, WA, January 22,
2007
26
NORMS

• Identify your score
• Compare it to a relevant norms table
• Year
• Subgroup
• Dimension
• Norms are stable and are not calculated on an
  annual basis anymore (see article)

27
ANALYTICS

• Institution Explorer
• User group and Discipline analysis
• Representativeness graphs
• Radar chart
• Library Use chart
• Thermometer chart
• Cumulative percentile distribution
• Longitudinal

28
SPSS

• What is SPSS?
• Statistical Package for the Social Sciences
• How do I know that my data are trustworthy?
• Reliability and Validity Analysis
• Are there differences between different groups?
• Compare mean scores for subgroups

29
Quantitative Analysis Excel Data

• Excel data files available on the LibQUAL Web
  site
• http//old.libqual.org/Manage/Results/index.cfm
• Use customized radar chart template to create
  custom analyses
• http//old.libqual.org/Manage/Resources/SampleRada
  rChart/index.cfm

30
Qualitative Analysis User Comments

• About one-half of users include comments on their
  surveys
• User Comments available on the LibQUAL Web site
• Download comments in Excel or text file
• Skim the comments
• Conduct ATLAS.ti analysis

31
Identifying What is Actionable

• Conclusions and recommendations are grounded in
  survey data
• Identifying What is Actionable Worksheets
• Priorities, Weaknesses, Strengths

32
Developing the Organization
33
Confirming Survey Objectives/Desired Outcomes

• Conducting LibQUAL is not an end in itself
• Articulate WHY the library wants feedback from
  users be specific
• Indicate potential targets of interest, for
  example
• Customer input for library Strategic Planning
• Assessment for institutional accreditation
• Intention to become more user-centered
• Focus on specific institutional or library
  strategic initiatives such as space/facilities
  issues, collection(s) changes, web re/design,
  customer service, etc.

34
Identifying Clarifying Roles Responsibilities

• Develop an understanding of the work that your
  library may want to undertake after the LibQUAL
  survey (see the Work Plan)
• Decide what work your library will/will not do
• Determine if your library will assign the
  responsibility to a single individual or will
  assemble a group / team
• Identify the knowledge, skills, abilities (KSAs)
  needed to successfully perform the desired work
• Assign the work to match the KSAs and/or acquire
  the needed KSAs

35
Knowledge, Skills, Abilities (KSAs)

• Commitment to focus on user-centered outcomes
  to apply user-centered thinking
• Dedication to achieving continuous process
  improvement
• Ability to think act system-wide, to see the
  big picture
• Effective communication skills
• Comfortable w/statistics data
• Critical thinking / analytical skills

36
Knowledge, Skills, Abilities (KSAs)Useful to
have or to develop (individually or collectively)

• Statistical methods quantitative data analysis
  (e.g., use of SPSS)
• Qualitative data analysis (e.g., Atlas-TI)
• Automate data collection tasks (e.g., write
  scripts)
• Design charts graphs (e.g., Excel)
• Writing (e.g., Word)
• Presentation (e.g., Powerpoint)

37
Implementing a LibQUAL Group or Team

• Consider the purpose / scope of the groups work
• Broad, all assessment vs. specific, LibQUAL
  only
• On-going vs. situational?
• Task-oriented, e.g. administer, analyze?
• Facilitative, e.g. communicate, present, assist
  others in understanding/utilizing data/results?
• Leadership-focused, e.g. synthesize, recommend,
  direct others to implement changes?

38
Components of the Chargefor a Group/Team

• Purpose and/or Scope
• Products / Deliverables
• Assumptions and Parameters
• the extent of the groups autonomy / authority
• approval process(es)
• w/whom and how often the group will communicate
  w/colleagues
• suggested approaches for methods of operation and
  how the group will function
• available resources (budget, people, etc.)
• Timeline
• Member KSAs

39
Assessment Group Resources

• Coming soon ARL SPEC Kit on Assessment
• Stephanie Wright (U. Washington) and Lynda White
  (U. Virginia)
• Boston College http//www.bc.edu/libraries/about/
  assessment/
• Ohio State University http//library.osu.edu/site
  s/staff/assess/
• U. Colorado (Boulder) http//ucblibraries.colorad
  o.edu/internal/assessment/index.htm
• U. Connecticut http//www.lib.uconn.edu/about/adm
  inistration/surveys/
• U. Georgia http//dataserv.libs.uga.edu/assessmen
  t/index.html
• U. Kansas http//www.informationservices.ku.edu/a
  ssessment/charge.shtml
• U. Maryland http//www.lib.umd.edu/PASD/MIS/larc/
  
• U. Mass http//www.library.umass.edu/assessment/i
  ndex.html
• U. Washington http//www.lib.washington.edu/asses
  sment/
• Washington State U. http//www.wsulibs.wsu.edu/ge
  neral/WG/AWG.html

40
Presenting Results to Stakeholders

• Identify all of the stakeholders or constituents
  who want and need to know about the survey
  results
• Consider the stake of each of the above what
  specific aspect of LibQUAL will be of most
  interest / concern
• Determine how to communicate with each identified
  stakeholder

41
Brown University Library Stakeholder
Accountability Matrix http//www.arl.org/arldocs/s
tats/statsevents/laconf/2006/BowbyOMahony.ppt
42
Assumption College LibQUAL Marketing
Planold.libqual.org/documents/admin/ThistleLibQU
ALIntroSeattle.ppt
43
Communicate with your Customers(faculty,
students, other)

• Particularly those whom you asked to participate
  in the survey
• Announce incentive award winners
• Inform users of highlights of survey results
• More importantly, what the library intends to do
• You asked for it

44
Engaging Others in Understanding Utilizing
Data Results

• Spread knowledge about LibQUAL
• Develop a culture of accountability for listening
  to customers and acting on user feedback

45
Developing a Culture of AssessmentToolkit
http//personal.anderson.ucla.edu/amos.lakos/CUtoo
lkit.html

46
Spreading Knowledge about LibQUAL throughout
the Library Staff

• Post Report on library web
• Make staff aware of LQ tutorial
• http//old.libqual.org/Information/Tools/index.cfm
  
• All-staff presentation
• Disseminate Comments to depts, units, groups,
  branches, librarians, etc.
• Work w/department leaders, managers
• Identifying Library Departments with Contribution
  or Impact on LibQUAL Questions
• Other?

47
Corroborating LibQUALResults w/Other Data

• Consider if your LibQUAL data confirms your
  prior expectations and results from other
  assessment and measurement, and, vice versa
• For example
• Undergrads have increasing needs desires re
  Library as Place and show increases in library
  gate counts/attendance
• Faculty grad students have access from home or
  office as highest priority and library web logs
  demonstrate increasing use from campus depts.

48
Discussing Results Implications

• Go beyond informing
• Lead efforts to analyze, synthesize, and
  interpret findings

49
Brown University LibraryUsing the Mini-Survey
to Explain LQ to Stakeholders
50
The Library SummitClemson University U. of
Texas at Austinhttp//old.libqual.org/documents/a
dmin/LibQUAL_Summit.pdf
51
Why Hold a Library Summit?http//www.lib.utexas.e
du/vprovost/assessment/libsummit/pdf/info_pkt.pdf
52
Identify 2 or 3 LibQUAL questions for which
you/your department has primary responsibility
Being User-CenteredGuiding Library Staff in
Understanding Using LibQUAL Results
Do these cluster in a specific dimension?
53
Overcoming Resistance BarriersChanging
Attitudes Behaviors

• We know whats best
• They are wrong
• We dont have the resources to

• ...only customers judge quality all other
  judgments are essentially irrelevant.
• Zeithaml, Parasuraman, Berry.(1999). Delivering
  Quality Service. NY The Free Press

54
Individual ExerciseAfter the LibQUAL Survey
-- Work Plan

• Review the list of steps describing work that can
  be accomplished after receiving the survey,
  including quantitative and qualitative data
  analysis and organizational development
• Which steps might your library undertake?
• Will the assignment be individual or a group /
  team?
• What KSAs might best support the accomplishment
  of this work?
• Who will be assigned?
• (Who will you consult with to make these
  decisions)?

55
Using LibQUAL Effectively
56
Targeting Incremental Improvements

• From all of the data, determine what can and
  should be addressed
• Prioritize some action items
• Align with mission, vision and goals of parent
  organization
• Address users top priorities, by user group
• Improve areas of strong user dissatisfaction
• Build on strengths, if they are truly user needs
  and priorities
• Identify work that can be de-emphasized and
  resources that can be reallocated

57
S.M.A.R.T. Goals

• S Specific
• the desired outcome or result is clearly defined
• M Measurable
• accomplishment can be charted and/or observed
• A Attainable
• achievable, goal is challenging but realistic
• R Relevant
• results-oriented, in line with institutional
  goals and library vision
• T Timely
• deadlines are set for accomplishment

58
SMART Goals -- Examples

• 75 of materials acquired from other libraries
  are received by users within 10 days of request
  by 12/07
• 50 of new books are on shelf within 5 days after
  library receipt by 10/07
• 60 courses utilizing WebCT include links to
  library research materials by 3/08
• Lower the unit cost of each service desk
  transaction by 10 from FY08 to FY08
• Versus general goals
• Improve ILL turn-around time
• Make new materials accessible to users more
  quickly
• Decrease staffing budget

59
(No Transcript)
60
Applying Data in Decision-Making

• Assessment and LibQUAL data and results are
  easily accessible and present when, where, and
  with whom decisions are made in your library
• Goals, priorities, and resource allocation are
  influenced by customer needs and wants

61
Archiving Data for Access
62
Archiving Data for Access
63
Cycle of Planning and Assessment
www.mindtools.com
64
Identifying Using Best Practices

• Seeking out and learning from colleague
  institutions
• Participate in LibQUAL as part of a consortial,
  regional, or interest-group implementation
• Identify peer institutions who may agree to do
  LibQUAL during the same cycle
• Implementing improvements based upon best
  practices

65
Integrating LibQUAL w/ Other Assessments

• Confirming your conclusions
• Obtaining user specifications for initiatives
• Monitoring progress
• Some options
• Suggestions boxes
• Focus groups
• Exit surveys

66
Planning Next StepsContinuous Assessment
Improvement
67
In development .

• Customized Consulting Services
• Longitudinal data analysis, across multi years
• Tailored peer comparisons, libraries you choose
• Assistance w/ content analysis for Comments
• Facilitation of LibQUAL staff retreat or summit
• Guidance in preparing communication plan
• Customized training in assessment skills
• Launch an assessment group or team
• Assistance in identifying key customer issues for
  future action

68
Workshop Evaluation
martha_at_arl.org raynna.bowlby_at_charter.net
69
LibQUAL Resources

• LibQUAL Web site
• http//old.libqual.org
• Publications
• http//old.libqual.org/publications
• Events and Training
• http//old.libqual.org/events
• Gap Theory/Radar Graph Introduction
• http//old.libqual.org/Information/Tools/libqualp
  resentation.cfm
• LibQUAL Procedures Manual
• http//old.libqual.org/Publications/index.cfm

70
LibQUAL Team

• MaShana Davis
• Technical Communications Liaison
• mashana_at_arl.org
• Yolanda Glass
• Administrative Assistant
• yolanda_at_arl.org
• Kristina Justh
• Customer Relations Coordinator
• kristina_at_arl.org
• Martha Kyrillidou
• Director, Statistics and Service Quality
  Programs
• martha_at_arl.org
• Khyati Nayak
• Senior Applications Developer
• khyati_at_arl.org