MINES for Libraries

1
MINES for Libraries
Presented by Martha Kyrillidou Director of the
ARL Statistics and Service Quality
Programs Association of Research
Libraries at Rutgers University Library June 1
2007 New Brunswick, NJ
www.arl.org/stats/
2
ARL Overall
3
Libraries Remain a Credible Resource in 21st
Century
98 agree with statement, My library contains
information from credible and known sources.
Note. Digital Library Federation and Council on
Library and Information Resources. (2002).
Dimensions and Use of the Scholarly Information
Environment.
4
Changing Behaviors
Only 15.7 agreed with the statement The
Internet has not changed the way I use the
library.
Note. Digital Library Federation and Council on
Library and Information Resources. (2002).
Dimensions and Use of the Scholarly Information
Environment.
5
ARL Toolkit

• StatsQUAL
• ARL Statistics
• LibQUAL
• E-Metrics
• DigiQUAL
• MINES for Libraries

6
What is MINES?

• Action research
• Historically rooted in indirect cost studies
• Set of recommendations for research design
• Set of recommendations for web survey
  presentation
• Set of recommendations for information
  architecture in libraries
• Plan for continual assessment of networked
  electronic resources
• An opportunity to benchmark across libraries

7
MINES for LibrariesTM

• MINES is a transaction-based research methodology
  consisting of a web-based survey form and a
  random moments sampling plan
• MINES typically measures who is using electronic
  resources, where users are located at the time of
  use, and their purpose of use in the least
  obtrusive way
• MINES was adopted by the Association of Research
  Libraries (ARL) as part of the New Measures
  toolkit in May, 2003.
• MINES is different from other electronic resource
  usage measures that quantify total usage (e.g.,
  Project COUNTER, E-Metrics) or measure how well a
  library makes electronic resources accessible
  (LibQUAL).

8
Questions Addressed By MINES for Libraries for
the OCUL Scholars Portal

• How extensively do sponsored researchers use
  OCULs Scholars Portal? How much usage is for
  non-funded research, instruction/education,
  student research papers, and course work?
• Are researchers more likely to use the Scholars
  Portal from inside or outside the library? What
  about other classifications of users?

• Are there differences in Scholars Portal based on
  the users location (e.g., in the library
  on-campus, but not in the library or
  off-campus)?
• Could MINES, combined with usage counts, provide
  an infrastructure to make Scholars Portal usage
  studies routine, robust, and easily integrated
  into OCULs administrative decision-making
  process for assessing networked electronic
  resources?

9
MINES for LibrariesTM Survey Form Five
Questions and a Comment Box
10
Methodological considerationsExperience with the
MINES Survey

• Terry Plum
• Assistant Dean
• Simmons GSLIS
• Rutgers University
• June 1 , 2007

11
Issues with web surveys

• Research design
• Coverage error
• Unequal access to the Internet
• Internet users are different than non-users
• Response rate
• Response representativeness
• Random sampling and inference
• Non-respondents
• Data security

12
MINES strategy

• A representative sampling plan, including sample
  size, is determined at the outset. Typically,
  there are 48 hours of surveying over 12 months at
  a medical library and 24 hours a year at a main
  library.
• Random moment/web-based surveys are employed at
  each site.
• Participation is usually mandatory, negating
  non-respondent bias, and is based on actual use
  in real-time.
• IRB waiver or approval
• Libraries with database-to-web gateways or proxy
  re-writers offer a comprehensive networking
  solution for surveying all networked services
  users during survey periods.

13
MINES strategy

• Placement
• Point of use
• Not remembered, predicted or critical incident
• Usage rather than user
• What about multiple usages
• Time out ?
• Cookie or other mechanism with auto-population or
  more recently counting invisibly with a time out.
  
• Distinguish patron association with libraries.
• For example, medical library v. main library.
• But what if the resources are purchased across
  campus for all. Then how to get patron
  affiliation?

14
Web Survey Design Guidelines

• Web survey design guidelines that MINES followed
  
• Presentation
• Simple text for different browsers no graphics
• Different browsers render web pages differently
• Few questions per screen or simply few questions
• Easy to navigate
• Short and plain
• No scrolling
• Clear and encouraging error or warning messages
• Every question answered in a similar way -
  consistent
• Radio buttons, drop downs
• ADA compliant
• Introduction page or paragraph
• Easy to read
• Must see definitions of sponsored research.
• Can present questions in response to answers
  for example if sponsored research was chosen,
  could present another survey

15
Quality Checks

• Target population is the population frame
  surveyed the patrons who were supposed to be
  surveyed - except in libraries with outstanding
  open digital collections.
• Check usage against IP. In this case, big
  numbers may not be good. May be seeing the
  survey too often.
• Alter order of questions and answers,
  particularly sponsored and instruction.
• Spot check IP against self-identified location
• Spot check undergraduates choosing sponsored
  research measurement error
• Check self-identified grant information against
  actual grants
• Content validity discussed with librarians and
  pre-tested.
• Turn-aways number who elected not to fill out
  the survey
• Library information architecture -- Gateway v.
  HTML pages there is a substantial difference in
  results.

16
(No Transcript)
17
Documenting the Purpose and Use of Electronic
ResourcesExperience with the MINES Survey

• Brinley Franklin
• Vice Provost for University Libraries
• University of Connecticut
• Rutgers University
• June 1 , 2007

18
It is useless to tell the acquisitions librarian
that half the monographs ordered will never be
used,unless we can specify which 50 to avoid
buying. (Galvin and Kent, 1977)
19
Reliance on Vendor Statistics
Vendor statistics, while more reliable than
in the past, are still maturing.
20
Measuring Digital Content Use

• The most popular current method of measuring
  usage of electronic resources by libraries is not
  through web-based usage surveys, but through
  vendor supplied data of library patron usage or
  transaction usage.
• Web-based usage surveys are increasingly relevant
  in the collection of usage data to make
  collection development and service decisions, to
  document evidence of usage by certain patron
  populations, and to collect and analyze
  performance outputs.
• Brinley Franklin and Terry Plum, Successful Web
  Survey Methodologies for Measuring the Impact of
  Networked Electronic Services (MINES for
  LibrariesTM) IFLA Journal 32 (1) March, 2006

21

• A web-based transactional survey that collects
  data on users demographics and their purpose of
  use. It is administered in real time over the
  course of at least a year using a random moments
  sampling plan.
• MINES for LibrariesTM has been administered at 40
  North American universities in the last four
  years. More than 100,000 North American
  networked services users have been surveyed using
  a standard protocol.

22
(No Transcript)
23
Library User Survey Patron Status
24
Library User Survey Affiliation
25
Library User Survey Location
26
Library User Survey Purpose
27
Sample Survey Data File Generated

• Other UConnInstruction/Education/Departmental
  (Non-Funded) Researchhttp//newfirstsearch.oclc.or
  g/donerefererdbnameWorldCatautho100122319FSI
  P12365012/3/2004Off CampusUConn
  Faculty12.101.104.84
• Family StudiesInstruction/Education/Departmental
  (Non-Funded) Researchhttp//www.jstor.org/cgi-bin/
  jstor/gensearch12374312/3/2004Off CampusUConn
  Undergraduate Student12.101.104.84Non-UConnOther
  Activitieshttp//homerweb.lib.uconn.edu/cgi-bin/Pw
  ebrecon.cgi?DBlocalPAGEFirst13084112/3/2004Of
  f CampusNon-UConn12.18.36.40
• Non-UConnInstruction/Education/Departmental
  (Non-Funded) Researchhttp//magic.lib.uconn.edu/in
  dex_real.html13312912/3/2004Off
  CampusNon-UConn12.76.131.246
• Non-UConnOther Activitieshttp//magic.lib.uconn.ed
  u/index_real.html12110612/3/2004Off
  CampusNon-UConn130.132.86.156Agriculture
  Natural ResourcesInstruction/Education/Departmenta
  l (Non-Funded) Researchhttp//magic.lib.uconn.edu/
  index_real.html12335712/3/2004
• Off CampusNon-UConn131.128.89.54EducationInstructi
  on/Education/Departmental (Non-Funded)
  Researchhttp//www.euromonitor.com/womdas/125744
  12/3/2004
• Off CampusNon-UConn134.241.135.70Non-UConnInstruct
  ion/Education/Departmental (Non-Funded)
  Researchhttp//homerweb.lib.uconn.edu/cgi-bin/Pweb
  recon.cgi?DBlocalPAGEFirst13285212/3/2004
• Off CampusNon-UConn134.74.42.82Business
  AdministrationOther Activitieshttp//homerweb.lib.
  uconn.edu/cgi-bin/Pwebrecon.cgi?DBlocalPAGEFirs
  t12564612/3/2004In the LibraryUConn
  Faculty137.99.1.122
• Liberal Arts SciencesOther Activitieshttp//www.
  siam.org/journals/simax/simax.htm12521712/3/2004
  On Campus - StorrsUConn Graduate
  Student137.99.1.122
• EngineeringInstruction/Education/Departmental
  (Non-Funded) Researchhttp//homerweb.lib.uconn.edu
  /cgi-bin/Pwebrecon.cgi?DBlocalPAGEFirst120431
  12/3/2004On Campus - StorrsUConn Graduate
  Student137.99.1.219
• Business AdministrationInstruction/Education/Depar
  tmental (Non-Funded) Researchhttp//proquest.umi.c
  om/pqdweb?RQT31812163312/3/2004On Campus -
  StorrsUConn Graduate Student137.99.100.105
• Business AdministrationInstruction/Education/Depar
  tmental (Non-Funded) Researchhttp//www.jstor.org/
  journals/07322399.html12165212/3/2004On Campus
  - StorrsUConn Graduate Student137.99.100.105
• Business AdministrationInstruction/Education/Depar
  tmental (Non-Funded) Researchhttp//homerweb.lib.u
  conn.edu/cgi-bin/Pwebrecon.cgi?DBlocalPAGEFirst
  12295312/3/2004On Campus - StorrsUConn Graduate
  Student137.99.100.175
• Business AdministrationInstruction/Education/Depar
  tmental (Non-Funded) Researchhttp//homerweb.lib.u
  conn.edu/cgi-bin/Pwebrecon.cgi?DBlocalPAGEFirst
  12484112/3/2004On Campus - StorrsUConn Graduate
  Student137.99.100.175
• Business AdministrationInstruction/Education/Depar
  tmental (Non-Funded) Researchhttp//proquest.umi.c
  om/login?COPTSU5UPTAmVkVSPTImREJTPTE3MjErMysxNkJD
  clientId4899612042312/3/2004On Campus -
  StorrsUConn Graduate Student137.99.100.175

28
Demographics by Location of UserU.S. Main
Libraries
MINES for Libraries
On Campus, Not in the Libraryn 6,391
Inside the Libraryn 9,172
Off-Campusn 4,953
29
Demographics by Location of User Ontario Council
of University Libraries
On Campus, Not in the Library n 7,090
Off-Campus n 9,163
Inside the Library n 4,047
30
Purpose of Use
Are users engaged in coursework, funded (or
unfunded) research, public service,
patient care, or other activities?
31
Purpose of Use By Location U.S. Main Campus
Libraries 2003 2005
On-Campus, Not in the Library n 9,460
In the Library n 9,733
66
Overall Use n 26,983
Off-Campus n 7,790
72 of sponsored research usage of electronic
resources occurred outside the library 83 took
place on campus.
32
OCUL Scholars Portal Users Purpose of Use
In a sample of 20,300 electronic resources uses
at OCUL libraries, there were four uses outside
the library for each use in the library.
33
Questions?

• Learn more about LibQUALTM, DigiQUALTM, MINES
  for LibrariesTM at
• www.statsqual.org

34
Analysis

• Web deliverables
• Crosstabulations in html for all OCUL data
• Interactive crosstabs for all OCUL and
  institutions
• Print deliverables
• summary tables for OCUL
• summary tables for each institution
• Final report

35
(No Transcript)
36
OCUL Scholars Portal UsageAffiliation
37
(No Transcript)
38
(No Transcript)
39
(No Transcript)
40
Affiliation by Purpose of Use
41
User Status by Purpose of Use
42
Location by Purpose of Use
43
Reason for Use
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
Library Assessment Conference
August 4-6, 2008 Seattle, WA, USA