Title: Does your information literacy work make a difference How can you tell
1Does your information literacy work make a
difference? How can you tell?
- Sharon Markless
- Kings College London and IMA
2What do we mean by IL work making a difference?
- Getting people to
- Use appropriate search strategies rigorously
evaluate e-resources manage research information
effectively - Value publications/research datasets/ICT
resources/ IL interventions - Be more confident in using the full range of
information/databases - Access a different range of resources assess
students IL collaborate/alter pedagogy - Know more about where to find appropriate
resources, which portals to use, where to
publish
3What longer term/higher order impact might IL
work achieve?
- Support widening participation in HE
(recruitment, graduation rates) - Influence patient outcomes e.g. choice of drugs,
appropriate investigations - Save clinical time
- Increase in publications
- More/larger research grants
- Increased professional or academic success????
4But beware of taking a step too far
- Whose impact? Identifying the significant
contribution of the library/ information service?
(attribution studies factorial design,
controlling variables - scale required e.g.
Colorado etc) - Important not to make library a hostage to
fortune What can you really affect? - Use the research when considering your possible
value and impact
5Research evidence shows health libraries can
influence patient outcomes by supporting
-
- rapid diagnosis
- appropriate investigation
- choice of drugs
- reduced length of stay in hospitals
- avoiding hospital-acquired infection
- avoiding additional outpatient visits
- Weightman and Williamson (2005)
- reduced waiting list due to new treatment
- Single site case study (2007)
6Research evidence shows academic library impact
- Local level (case studies)
- can enhance student retention rates
- help academic research publications output
- may meet specific student and researcher IL
training needs - changes in assessment practices
- high ROI in electronic content, in grant money
- National level
- enhance knowledge and use of e-resources
- enhance information literacy
7Step one in evaluating impact focus
- Make decisions about your focus
- areas of impact for your IL work your impact
objectives (what changes are you looking for?)
All linked to the purposes and aspirations of the
organisation - Precision matters impact objectives drive the
whole process of evaluation - Be realistic with available resources 3-5 years
8Examples of impact objectives from real libraries
- To secure the library as a main source of
information in conducting research in the field - To enable students/doctors/nurses to operate
effectively in an e-environment - To develop clinicians ability to find and use
appropriate information - Targeted staff making more effective use of
evidence-based information in clinical practice - Competent independent use of electronic resources
- Healthcare staff find the evidence to develop
local care pathways.
9Challenges librarians encountered when
formulating impact objectives
- using support and provide in objectives was
automatic but they will draw you down the
process road! - Often unclear where the impact/process line is
drawn (integrate IL into the curriculum lecturer
attitudes) - Its hard work and means reviewing plans,
policies, strategy but no-one else can do it for
you!
10Step two in evaluating impact impact indicators
- Indicator a statement around which you can
collect evidence on a regular basis to show a
trend - Strong surrogates for impact
- Choose what matters
- Limit numbers
- Direct link to relevant impact objective
11Examples of impact indicators from real libraries
- Levels of remote use of resources
- Proportion of targeted staff using the evidence
base - Do students cite eresources in bibliographies?
- Proportion of users needing intermediary
assistance for directional and basic search
enquiries - Nurses competence in appraising evidence
- Percentage of searches focused in the wrong area
- Do clinicians follow up and ask for articles?
- Percentage of clinical time spent on searching
-
12Step three in evaluating impact collecting
evidence
- What counts as evidence?
- Statistics (relationships between)
- Case studies, stories, critical incidents
- Analysis of student bibliographies and
assignments/projects - Results of self-assessment or diagnostic tests
- Document analysis
- Systematic observation data
- Systematic, rigorous, mainly qualitative
13How can we collect evidence?
- Observation
- Asking questions
- Inferring
- Time consuming
- New skills for librarians?
- Appropriateness different tools for different
groups and contexts
14Rich mixture of methods adopted by teams in
LIRG/SCONUL initiative and health libraries work
- Diagnostic tests (especially for distance
learners) - Questionnaires (ideal type) embedded questions
- Focus groups (structured)
- Interviews during searches
- Student progress files/reflective diaries
- Analysis of assignments and bibliographies
- Teaching audit
- Critical incident interviews
15Some issues in gathering evidence reported by
librarians
- What standard of evidence do you need?
- Necessity of collecting baseline data
- Short-, medium- or long-term? Impact may take
time, not fit annual planning cycles - Low response rates
- Capturing evidence in busy libraries
16Some issues about evaluating impact reported by
librarians
- Collaboration/cooperation is critical,
particularly in data collection - Important to focus on one aspect of provision in
depth - Sustaining the work what might be needed for
institutionalisation? - Challenging and stressful nature of engaging with
impact
17Some issues about evaluating impact reported by
librarians
- Importance of a coherent framework and structure
to support the process - Use of an impact model systematic approach
- 3 x workshops take detailed work back to own
library troubleshooting - Materials, especially examples of indicators etc,
approaches to data collection, research tools - Teams within each participating library
18Some final words from librarians who have been
doing it
- Keep a balance you also need to maintain the
infrastructure - Choose what matters
- Keep it manageable
- Influencing academics and getting change at
Academic Boards was harder to do than the
evaluation.
19- Weightman, A. L. and Williamson, J. (2005) The
value and impact of information provided through
library services for patient care a systematic
review in Health Information and Libraries
Journal 22 (1) march 4-25 - For research tools and approaches used in
evaluating impact http//vamp.diglib.shrivenham.c
ranfield.ac.uk/impact/impact-initiative - Return On Investment case study Elsevier
website Library Connect