Usage Statistics

1

• Usage Statistics
• Information Behaviors
• Understanding User Behavior with Quantitative
  Indicators
• John McDonald
• Assistant Director for User Services Technology
  Innovation
• The Libraries of the Claremont Colleges
• November 2, 2007
• NISO Usage Data Forum

2
Correlation Boba Fett and Ladybugs
3
We have the data, now what do we do?

• What we have done
• Cancel journals
• Inform purchase decisions
• What we should do
• Understand usage behaviors
• Guide our decision making processes
• Understand our impact on our patrons

4
Information Usage Behaviors

• Starting
• Browsing
• Accessing
• Chaining
• Differentiating
• Extracting

• Verifying
• Networking
• Monitoring
• Managing
• Manipulating
• Teaching
• Ending

• Ellis (1993), Ellis Haugan (1997) Meho
  Tibbo (2003), McDonald (2007)

5
Accessing
How Do We Observe Measure these Behaviors?
Chaining Differentiating
Managing Ending
Accessing Browsing
6
How do we observe measure?

• Pose a Question
• How will a new service affect our users?
• Develop a Theory
• Explain what you think happened.
• Test the Theory
• Develop metrics, collect data, analyze.

7
Example 1 Starting Accessing

• Question How will a new service affect our
  users?
• Theory If we improve the users ability to
  identify relevant material (starting) and
  retrieve it (accessing), we either save them time
  or effort and allow them to access more
  material.
• Test There will be a significant increase in
  the usage of material.

8
Starting Accessing Use Before After OpenURL
Publisher Use 2000 Publisher Use 2000 Publisher Use 2001 Publisher Use 2001 Publisher Use 2002 Publisher Use 2002 Wilcoxon Signed-rank Test Wilcoxon Signed-rank Test
Subjects Journals Mean SD Mean SD Mean SD z Pgtz
Astronomy 1 347 0 813 0 1408 0 -1.00 0.32
Biology 104 638 1625 847 2079 957 2351 -5.88 0.00
Chemistry 42 1388 3248 1553 3889 2542 7294 -4.85 0.00
Comp. Sci. 14 197 429 224 490 175 239 -1.63 0.10
Engineering 20 92 200 164 310 174 312 -2.41 0.02
Gen. Sci. 3 16243 15571 20938 20345 26553 26506 -1.39 0.17
Geology 22 46 183 44 143 144 374 -3.10 0.00
Mathematics 29 59 155 80 153 121 182 -3.68 0.00
Physics 28 198 313 1081 2107 1526 2933 -4.00 0.00
Total 263 701 2730 975 3527 1301 4953 -10.39 0.00
significant at .05 level significant at
.01 level
9
Example 2 Differentiating

• Question Do our choices affect our users
  ability to differentiate between resources?
• Theory If we group resources together, we allow
  users to identify relevant resources and provide
  efficient methods to differentiate between
  resources.
• Test There is significant increase in searches
  across common resource groupings.

10
Differentiating Federated Search Statistics
Database Searches
Web of Science 3823
OPAC 3314
WorldCat 3267
PubMed 238
INSPEC 233
MathSciNet 183
Faculty of 1000 Biology 176
Compendex 132

11
Differentiating OPAC Searches (2005 v. 2006)
12
Differentiating WorldCat Searches
13
Example 3 Chaining

• Question Do our users move from one information
  resource to another?
• Theory If users are moving from resource to
  resource, usage of resources in the same
  environment (one provider) and results of that
  usage (citations) will increase.
• Test There will be a significant increase in
  the usage and/or results of usage of a resources
  material.

14
Chaining JSTOR Citations (2000 v. 2004)
15
Example 4 Managing, Teaching

• Question Are our users managing or utilizing
  content differently?
• Theory A stable online archive allows users to
  re-access or re-use content more efficiently
  (utility usage or virtual vertical file), or
  utilize it for instructional purposes in
  different ways (virtual syllabus).
• Test There will be a significant increase in
  the systematic re-use of current, locally
  produced content.

16
Managing, Teaching Use of local content
17
Example 5 Service Effects

• Question How do our choices in libraries affect
  user behavior?
• Theory When we change the display options (e.g.
  cataloging) for journals, did that affect either
  publisher usage or SFX usage?
• Test Changing cataloging results in decreased
  local journal usage as measured by the publisher
  and SFX.

18
Service Effects Usage of Journals (2005 v. 2006)
19
Service Effects SFX Clickthrough Rate (Local v.
Shared)
20
Example 5 Services Related Behaviors

• What else do users want or need?
• Are there services related behaviors that we
  can observe? Providing content is one option,
  but how are researchers using associated
  information services?
• If we provide them the article they want in
  fulltext, we see that sometimes they ask for
  other types of things.
• Can we match these things to those user
  behaviors?

21
Services Related Behaviors
Information Service Requests
Order Article via Document Delivery 951
See References for this Article 790
Search Library Catalog 580
Read Abstract 283
Search Article Title on the Web 170
Send Feedback to Library 15
See Articles citing this Article 11
22
What else could we be studying?

• Monitoring
• Many information providers have e-alerts, repeat
  saved searches, etc.
• Networking
• Users may want to email a citation to a colleague
  or another student.
• Extracting
• Passing the bibliographic information to another
  database to search.
• Analyzing
• Including user behavior information in the
  statistical measurement tools.

23
Questions?
John McDonald November 2, 2007