The Importance of Network Measures and Statistics

1
ARL New Measures Initiative The E-Metrics Project
The Importance of Network Measures and Statistics
2
The ARL New Measures Initiative

• Service Effectiveness Measures (LibQUAL)
• Higher Education Outcomes Research Review
• Investigation of Cost Drivers (i.e. Technical
  Services)
• Self-study Program Based on ILL/DD Study
• E-Metrics (Measures for Electronic Resources)
• ARL New Measures Initiative http//www.arl.org/sta
  ts/newmeas/newmeas.html

3
Working Definition of Networked Services

• Those electronic information resources and/or
  services that users access electronically via a
  computer network
• From on-site in the library
• Remote to the library but from a campus
  facility
• Remote from the library campus

4
Networked Information Resources

• Locally Licensed Databases
• Regional or Statewide Consortia Licensed
  Databases
• Aggregated Databases
• Publishers Databases
• Publicly Available (Web) Resources

5
Networked Information Services

• Access to text numerical databases, electronic
  journals, electronic books, listservs, e-mail
• Instruction, Training Workshops
• Reference Information Services
• Virtual Reference
• Interlibrary Loans, Document Delivery
• IT Infrastructure
• Institutional Personal Portals

6
The Changed Library Information Environment
7
Percentages of Acquisitions Dollars Devoted to
Electronic Resources
1999-2000 several libraries reported over 20
spent for electronic resources. Source ARL
Statistics and Supplementary Statistics.
8
Comparison of Annual Increases in Electronic
Resources Total Materials Expenditures
ARL Libraries Comparison of Yearly Increases in
Electronic Resources and Total Materials
Expenditures Average Counts

• 446.9 increase in dollars spent for Electronic
  Resources between 1992 and 2000
• 49.4 increase in dollars spent for Total Library
  Materials between 1992 and 2000
• Source ARL Statistics and Supplementary
  Statistics

9
Why You Need Networked Data Statistics
Funding

• Financial Support
• To justify - make a case for continued current
  support for digital collections
• To make a case for additional support for
  technology infrastructure

10
Why You Need Networked Data Statistics
Infrastructure

• Better Internal Processes
• To measure track changes in internal processes
• To enable better decision-making in allocating
  prioritizing resources needs
• To enable assessment of service quality in a
  networked environment

11
Why You Need Networked Data Statistics
For Comparisons

• Institutional Comparisons
• For benchmarking digital services
• To enable competition for resources with other
  departments on campus

12
Why do You Need Networked Data Statistics
Vendor Negotiation

• Need for accurate reporting of network use
• Need for accurate estimates of per client use
• Ability to compare overlapping coverage
• Need the ability to pressure vendors to price
  according to the librarys real need

13
Does this Describe your Library?

• We do not currently have this data.
• We are seeking ways to find measures.
• We are looking for formal mechanisms for
  dissemination and evaluation.
• We are discussing how to support analysis of the
  data.

Source February 2000 Scottsdale E-Metrics
Planning Workshop
14
Changes Needed to Implement the E-metrics in
Research Libraries

• Need an Organizational Culture Change
• With Focus on Customers
• With Focus on Networked Services
• Local, State, Consortia, etc.
• Need to Create a Culture of Assessment

15
Culture of Assessment - definition

• Basic Value - customer learning focus
• A Culture of Assessment is an organizational
  environment in which
• decisions are based on facts, research and
  analysis, and where
• services are planned and delivered in ways that
• maximize positive outcomes and impacts for
  library clients.
• A Culture of Assessment exists in organizations
  where
• staff care to know what results they produce and
  
• how those results relate to customer
  expectations.
• Organizational mission, values, structures, and
  systems support behavior that is performance and
  learning focused.

16
Culture of Assessment We have a GAP
17
Bridging the Gap

• Think Systemically (SIPOC)
• Anticipate the Future
• Take Risks innovate experiment
• Focus on Customers
• Create Sustain a Culture of Assessment

18
Bridging the Gap

• Build Organizational Structures that Support Work
  based on Data Analysis - MIS Resource
• Invest in Staff Resources Learning
• Invest in IT Infrastructure, Services Support
• Build Collaborative Environments with Vendors
  Other Libraries

19
Additional Approaches to Evaluation

• Balanced Scorecard - http//www.bscol.com/
• LibQual Project - http//www.arl.org/libqual
• Activity Based Costing - http//www.mamag.com/wint
  er01/w01tati.htm
• Learning Outcomes Assessment - http//www.arl.org/
  stats/newmeas/outcomes/heo.html
• Measuring Library Service Quality ARL Online
  Lyceum - http//www.arl.org/training/quality5Fold
  .html
• Measuring Performance of ILL/DD -
  http//www.arl.org/stats/newmeas/ill-dd.html

20
Projects Related to E-Metrics

• EQUINOX Project (EU-UK)
• Publishing and Library Solutions Committee (PALS)
  - (UK)
• International Coalition of Library Consortia
  (ICOLC)
• National Commission of Libraries and Information
  Science (NCLIS)

21
Projects Related to E-Metrics

• Institute of Museum and Library Services (IMLS)
• Council on Library and Information Resources
  (CLIR)
• NISO Standard on Library Statistics.
• Round-up of Other E-metrics Developments ARL

22
Keep in Mind

• The Evolving Nature of the IT Environment
• The Evolving Nature of Digital Offerings New
  Vendor Configurations
• New Partnership Arrangements
• New Collaborative Service Options - influence on
  e-metrics
• New Usage Payment Models

23
Forces that Shape the Network Services
Environment in Research Libraries
Internal

• Increasing Demand for Digital Content
• IT Infrastructure Networked
• Competencies Expert Staff (HR)
• Accountability Assessment
• Organizational Structure
• Institutional Mission

24
Forces that Shape the Network Services
Environment in Research Libraries
External

• Increasing Offering of Digital Content
• Inter-institutional Competition
• Increasing Ability for Library Users to Bypass
  the Library for Scholarly Information
• Ever-changing Information Technology Environment

25
Take ActionNow!!!

• PLAN PRIORITIZE
• You Dont Have to Collect ALL the Measures
• Collect the MOST IMPORTANT Measures for Your
  Library
• EXPERIMENTATION is OK!
• Take ACTION on What You Measure!!!

26
Developing measures and evaluation techniques for
networked services will take time, effort, and
on-going learning on everyones part but we
must begin now.
(Carla Stoffle, University of Arizona)
We not only need to measure things in new ways
but we also need to measure new things.
(Sherrie Schmidt, Arizona State University)