Selection and appraisal for scientific

1
Selection and appraisal for scientific
scholarly repositories

• Ross Harvey
• Practices and Challenges in Preservation and
  Access for Scientific and Scholarly Digital
  Repositories
• DCC/DPE/DRIVER/Nestor Joint Workshop, 27-28
  November 2007, Berlin

2
Outline

• Introduction
• Definitions
• The appraisal process
• Appraising digital objects
• Is appraisal dead?
• If we dont appraise
• Reasons for appraising
• Some issues from practice
• Different institutional contexts, different
  approaches
• Examples epidemiological data sets, social
  science data
• Appraisal tools
• Automating appraisal
• Summary

3
Introduction

• Focus the importance of selection and appraisal
  for long-term curation of content in scientific
  and scholarly digital repositories
• Key themes
• Why appraise?
• What is different about appraising digital
  objects?
• Tools to assist appraisal
• Automating appraisal
• Increasing interest in appraisal
• Little mention in 2005 DCC workshop on long-term
  curation within digital repositories
  (http//www.dcc.ac.uk/events/dr-2005/)
• DELOS conference devoted to appraisal, 15-16
  November 2007 (http//www.dpc.delos.info/rethinkin
  g_appraisal/)

4
Definitions

• Appraisal the process of evaluating records to
  determine which are to be retained as archives,
  which are to be kept for specified periods and
  which are to be destroyed (Ellis 1993, p.461)
• JISC Records appraisal is the process of
  determining the archival value and ultimate
  disposition of records. Appraisal decisions are
  based on a number of criteria including the
  historical, legal, administrative, and financial
  value of the records. Identifying permanently
  valuable records through appraisal is one of the
  basic aims of records management. The management
  and appraisal of electronic records therefore
  contributes to digital preservation.
• Selection the process of deciding what will be
  added to a librarys collection

5
The appraisal process

• Appraisal is an evil necessity (Terry Eastwood)
• Selection involves bias acknowledge it
• Criteria applied
• Policy does the institution have a selection
  policy? Does the material fit into it?
• Legal and intellectual property issues have, and
  can, acceptable rights be negotiated?
• Technical issues can the file format be handled,
  currently and in future? Is transfer to a more
  manageable format possible?
• The existence of documentation, ancillary data
  and metadata is there sufficient data?

6
Appraising digital objects

• Different appraisal approaches are needed for
  different kinds of digital materials discussion
  about appraisal has assumed one size fits all
• Appraisal of digital objects is different
  emphasises
• Metadata (documentation that explains the
  character of the data)
• Analysis of the technical feasibility of
  preservation (e.g. file formats, viability of
  maintaining/preserving them)
• Analysis of the economic feasibility of
  preservation (can we afford to do it?)
• Appraisal happens earlier in the life-cycle
• Print (pre-digital) paradigm Benign neglect,
  appraisal traditionally carried out well after
  the records have been created
• Digital paradigm digital materials need
  attention from the time they are created
  (digital mortgage)

7
Is appraisal dead?

• With our increasing ability to store digital
  information and access it, is appraisal
  necessary?
• Appraisal is costly
• Even if we have the technical ability to store
  and preserve everything, we probably dont want
  to
• It is worth examining the idea of 'selection' in
  the context of large amounts of digital material
  being available. Indeed, it needs scrutiny. Do we
  think for a second as the world information grows
  exponentially - and the number of curators does
  not - that we're capable of selecting even the
  thinnest cream of the crop? There is clearly a
  challenge here - as more material is either born
  digital, or digitised, does the pre-selection
  model still apply, or is post-selection going to
  become the norm? If the latter, what does it mean
  for repositories? (posting on ngc4lib list)

8
If we dont appraise

• What are the consequences of not doing some
  appraisal, of not making some selection?
• Balance material's 'worth' against the likely
  success/cost of long-term preservation and
  provision of access
• Likelihood, cost of economical long-term
  viability
• Can we afford to maintain large quantities? (NB
  costs of digital preservation still unclear)
• Post-selection approach shifts costs elsewhere
  eg digital archaeology approaches and untried
  solutions
• Despite reducing storage costs and increasing
  processing power, information retrieval
  capabilities are not sufficiently developed to
  ensure retrieval

9
Reason for appraising

• Good management
• The quality of long-term management is directly
  related to the quantities managed
• It is as important to determine what we want to
  exclude from our collections as it is to decide
  what to include
• Trust
• Repositories should contain objects that are
  trusted by researchers
• Cost issues cant carry out routines,
  verification etc required to ensure trust on very
  large quantities
• Resource issues
• Long-term funding for repositories not (yet)
  guaranteed
• Resource limitations (e.g. funding, skills)
  maintaining smaller quantities helps manage
  resource limitations

10
Some issues from practice (PARADIGM Project)

• Multiple copies of the same document stored by
  different people
• Which copy is the master when similar documents
  appear on the computers of three or four members
  of staff? Should we retain the version held by
  the more senior member of staff?
• System files present in acquisitions of digital
  records
• The archivist will also want to identify and
  remove operating system, application and other
  software files
• Processing required prior to appraising digital
  records
• Records accessioned on older media and/or in
  older formats will need to be extracted in order
  to undertake an assessment of their content,
  context, structure and technical viability. May
  add significant costs to the processing of
  archives before appraisal even begins
• Establishing the authenticity of creation dates
• Some applications automatically generate dates -
  whenever the document is viewed the current date
  will be given
• Establishing the authenticity of authorship
• How accurate is the 'author' metadata associated
  with a file? e.g. where staff routinely share
  computers

11
Different institutional contexts, different
approaches

• Different contexts require different appraisal
  approaches
• Science datasets if we created it, it is worth
  keeping - emphasis is placed on re-use capacity
• Legal requirements to keep everything published
  in specified regions (e.g. national libraries)
• Reasons
• Accountability different domains have different
  requirements
• Discipline-specific requirements and practices
• Who pays? Who uses the results? What do they
  insist on? (Designated Community?)
• Data privacy relevant to some contexts, but not
  all
• Scientific data
• Growing interdisciplinary approaches to
  scientific data
• Large curated scientific databases a case for
  less appraisal

12
Example epidemiological data sets

• Possible retention criteria
• The nature of the questions being asked by the
  study
• Whether it addresses only one question or many
• Whether the question has been asked before
• The richness of the data set
• If it is a longitudinal study indicates an
  amber light
• Sample-related studies
• Stability of the measures used
• Possibility to go back to the population (e.g.
  for consent, ethical committee access)
• Uniqueness value for possible future comparisons
  (Lord Macdonald JISC E-science Curation Report
  2003, p.46)

13
Example social science data

• Data Preservation Alliance for the Social
  Sciences (DataPASS) appraisal guidelines
• Key questions addressed are
• How significant are the data for research?
• How significant is the source and scientific
  progress and society?
• Is the information unique?
• How usable are the data?
• What is the timeframe covered by the information?
  
• Are the data related to other data in the
  archives?
• What are the cost considerations for long-term
  maintenance of the data?
• What is the volume of data?
• http//www.icpsr.umich.edu/DATAPASS/pdf/appraisal
  .pdf

14
Appraisal tools

• Very few appraisal tools exist
• U.S. Geological Survey Records Appraisal Tool
  http//igskmncngs030.cr.usgs.gov/RAT/tool.php
  This tool assists the USGS in appraising record
  collections that are offered to, or sought by,
  the USGS.
• We should be expending our resources on the data
  we most value. Determining that value requires us
  to make judgments, but utilizing a repeatable and
  comprehensive scheme can allow us to judge data
  responsibly. Documenting those judgments is
  essential, because future generations will depend
  on the current scientists and records managers to
  preserve the data that will advance knowledge
  (J. Faundeen, USGS)
• PARADIGM Project notes the use of Directory Tools
  (e.g.DirPrinting) to assist appraisal

15
(No Transcript)
16
Automating appraisal

• Development work proceeds, but no workable tools
  yet
• ECHO Project (http//www.ndiipp.uiuc.edu/)
• PLANETS (http//www.planets-project.eu/)
• PRESERV (http//preserv.eprints.org/
• Appraisal in the Digital World conference 15-16
  Nov 2007 (http//www.dpc.delos.info/rethinking_app
  raisal/)

17
Summary

• Appraisal is necessary for good management of a
  TDR
• Appraisal needs redefinition in the digital
  environment
• e.g. it has to take much more account of
  technical characteristics, of disciplinary
  differences
• More work is required to develop and test
  different models of appraisal that take better
  account of domain differences, technical issues,
  cost-benefit consequences and issues

18
Further reading

• DELOS Appraisal Conference papers (when
  available) http//www.dpc.delos.info/rethinking_a
  ppraisal/
• ERPANET/CODATA Workshop (2003) The selection,
  appraisal and retention of digital scientific
  data final report, ERPANET/CODATA Workshop,
  Biblioteca Nacional, Lisbon, December 15-17,
  2003. http//www.erpanet.org/events/2003/lisbon/Li
  sbonReportFinal.pdf
• Ross Harvey (June 2006), Appraisal and
  Selection, DCC Digital Curation Manual, S. Ross,
  M. Day (eds), http//www.dcc.ac.uk/resource/curati
  on-manual/chapters/appraisal-and-selection
• PARADIGM Project
• http//www.paradigm.ac.uk/

19
Speaker contact information

• Ross Harvey
• Visiting Professor, HATII, University of Glasgow
• Professor, School of Information Studies,
  Charles Sturt University, NSW, Australia
• Email rossharvey_at_csu.edu.au