Usage Factor and PIRUS A move towards more sophisticated, granular, and comprehensive usage metrics

1
Usage Factor and PIRUSA move towards more
sophisticated, granular, and comprehensive usage
metrics? Richard Gedye, Research Director,
Oxford Journals

• APE 2009
• Berlin, 20 January 2009

2

• Usage Factor

3
The challenge

• ISI's Impact Factor compensates for the fact that
  larger journals will tend to be cited more than
  smaller ones
• Can we do something similar for usage?
• In other words, should we seek to develop a
  Usage Factor as an additional measure of
  journal quality/value?

4
For example..

• Usage Factor
• Total usage over period x of articles published
  during period y
• Total articles published during period y

5
Usage factor advantages

• A useful counterweight to Impact Factors
• Especially helpful for journals and fields not
  covered by ISI
• Especially helpful for journals with high
  undergraduate or practitioner use
• Especially helpful for journals publishing
  relatively few articles
• Data available potentially sooner than with
  Impact Factors

6
Usage factor advantages

• Authors select journals that will give their
  articles prestige and reach. Impact Factor is a
  widely used surrogate for the former, while
  perceived circulation and readership reflect the
  latter. But usage is becoming more important as a
  measure of reach
• Carol Tenopir

7
Usage factor advantages

• UF is a simple metric that would make usage more
  understandable to editors and authors as a
  measure of value. There is currently much talk of
  usage and a lot of data, which the non-librarians
  find confusing.
• Publisher

8
Usage Factor

• Research

9
Authors support for a new, usage based measure
10
Some previous evidence
From New journal publishing models an
international survey of senior researchers Ian
Rowlands and Dave Nicholas, A CIBER report for
the Publishers Association and the International
Association of STM Publishers, 22 September 2005
11
Librarians new journal decisions
12
Librarians renewal decisions
I would view Usage Factor as an aid for
collection rather than cancellation decisions.
Usage per se is a more suitable tool for us when
considering cancellation.
13
Publishers Issues to address

• Detecting and deterring gaming
• Differences between disciplines and journal types
• What about print usage
• How to integrate usage data when journal content
  hosted on multiple sites

14
Publishers Issues to address

• in order to fully assess the value of a Usage
  Factor, I would need more information on the
  calculation, its origins and maybe some worked
  examples where sample data is applied to the
  equation.

15
Usage Factor

• Development

16
Development first steps

• Original Working Group expands to become Project
  Steering Group
• 6 publishers
• 1 aggregator
• 1 hosting service
• 2 libraries
• 1 consortium
• All interested in the insights to be gained from
  contributing data to the project

17
Development Phase 1

• Journal usage data from multiple publishers to be
  ingested and analysed by expert third party

18
Development Phase 1

• RFP seeking expert third party shortly to be
  published now that most outstanding issues
  addressed
• For example-
• Measuring number of qualifying items published
• Assigning a correct publication year for each
  item
• Excluding spiders, crawlers, etc
• Selecting a third party schema for classifying
  all journals by subject
• to agree final detailed format of standard report
• Aim to ensure data consistency, integrity, and
  fitness for purpose

19
Phase 1 Deliverables

• A report which will-
• Outline the various metrics assessed
• Recommend which of them prove consistent and
  robust enough to be adopted for scaled up onward
  monitoring
• Suggest any ways in which data providers might
  amend the way they capture, structure, label, and
  maintain their data which would make the
  measurement of Usage Factors-
• Easier
• More reliable
• Propose ways to audit Usage Factors for accuracy

20
Reality check..

• Currently journal publishers are under a lot of
  pressure to demonstrate the value they provide.
  By participating in this process, publishers will
  influence it, helping to develop useful measures
  in which they can have confidence.
• This is going to happen in any event, so it is
  best that UF is developed and implemented by a
  trusted organization in which publishers are
  represented.

21

• PIRUS

22
PIRUS Publisher and Institutional Repository
Usage Statistics

• A COUNTER Project
• Sponsored by JISC through the
• PALS Metadata and Interoperability programme
  (phase 3)

23

• Currently the COUNTER code does not cover
  reporting of usage below the individual journal
  level.
• Demand for usage statistics at the individual
  article level from users has hitherto been low.

24
Increasing interest in article level usage

• More journal articles hosted by Institutional and
  other Repositories
• Publishers want a more complete picture of
  article use
• Authors want a more complete picture of article
  use
• Institutions and funders want a more complete
  picture of article use

25
Increasing interest in article level usage

• Online usage becoming an alternative, accepted
  measure of article and journal value
• Knowledge Exchange report recommends developing
  standards for usage reporting at the individual
  article level
• JISC Usage Statistics Review Project goal
  item-level usage statistics
• UK Research Excellence Framework Usage-based
  metrics being considered as a tool for assessing
  research

26
Article usage metrics now more practical

• Implementation by COUNTER of XML-based usage
  reports makes more granular reporting of usage a
  practical proposition
• Implementation by COUNTER of the SUSHI protocol
  facilitates the automated consolidation of usage
  data from different sources.

27
The challenge

• An article may be available from-
• The main journal web site
• Ovid
• ProQuest
• PubMed Central
• Authors local Institutional Repositories
• If we want to assess article impact by counting
  usage, how can we maximise the actual usage that
  we capture?

28
PIRUS Project Mission

• To develop a global standard to enable the
  recording, reporting and consolidation of online
  usage statistics for individual journal articles
  hosted by Publishers, Institutional Repositories,
  and other entities

29
PIRUS Project AchievementsAugust 2008 January
2009

• Researched current practices for storing and
  labeling individual articles in both publisher
  and repository environments
• Use of DOI for item mapping is critical
• Successfully tested a mechanism for automatically
  gathering usage data from Institutional
  Repositories
• Proposed a simple XML-based report for displaying
  article usage statistics that can be implemented
  by any entity that hosts and provides online
  access to articles.

30
PIRUS Report and Recommendations

• Initial report recommending best practice and
  proposals for how it might be implemented
  submitted 16 January 2009

31
Article identification

• Only items with a DOI will qualify to be counted
• Resource type needs to be clearly identified so
  that we can exclude any agreed exceptions to the
  above
• We should encourage journal article to be used
  as a resource type descriptor

32
3 Usage collection strategies

• Publishers, IRs, and aggregators issue COUNTER
  defined reports for all articles hosted
• Embed or associate tracker code into relevant
  items so that an identifying message is sent to a
  central server for every successful request
• Expose raw usage data in such a way that it can
  be easily harvested by any organisation with the
  resources to use it

33
Usage factor and PIRUS Project

• Any Questions?

34
Usage factor and PIRUS Project

• Richard Gedye
• richard.gedye_at_oup.com
• USAGE FACTOR
• http//www.uksg.org/usagefactors
• PIRUS
• http//www.projectcounter.org/PIRUS_Project_plan.d
  oc