Digital Preservation at Cornell

1
Digital Preservation at Cornell

• Anne R. Kenney
• Association of Canadian Archivists
• May 25, 2002

2
5 Stages of Digital Preservation

• Acknowledging the problem
• Digital preservation projects are initiated
• Digital preservation projects segue into programs
• Digital preservation programs become
  comprehensive and coordinated
• Institutional programs embrace inter-institutional
  collaboration and dependency

3
Stage 1 Acknowledging the Problem

• Hello, my name is Cornell and Im a digital
  imaging junkie.
• More than a decade of imaging projects
• No inventory
• Uneven/inconsistent documentation
• Proprietary formats
• Obsolete hardware and software
• No ongoing funding stream

4
Stage 2 From Digital Imaging to Digital
Preservation

• Risk Management of Digital Information A File
  Format Investigation
• http//www.clir.org/pubs/abstract/pub93abst.html
• Project Prism
• http//www.library.cornell.edu/preservation/prism.
  html
• Project Harvest
• http//www.library.cornell.edu/harvest/
• RLG DigiNews
• http//www.rlg.org/preserv/diginews/

5
Stage 3 From Digital Preservation Projects to
Programs

• USDA Economics and Statistics System
• http//usda.mannlib.cornell.edu/
• Project Euclid
• http//projecteuclid.org/
• ArXiv
• http//arxiv.org/
• C-Theory

6
Digital Preservation Components
Planning and Policy (DPO-IRIS)
Technology D-LIT
Content and Use (dispersed)
7
Stage 3.5 Mainstreaming and Coordinating Programs

• Establishing a Central Depository
• Joint initiative of IRIS and D-LIT
• Preserving Cornell's Digital Image Collections
  Implementing an Archival Strategy, March 2001
  (IMLS grant)

8
Central Depository Part 1

• Establishing a Central Depository for Preserving
  Digital Image Collections - Part I
  Responsibilities of Transferee
• http//www.library.cornell.edu/imls/index.htm

9
CD Part 1 Responsibilities of Transferee

• 1. Selection and Content Considerations
• Scope, Content/Functional Criteria, Priorities
• 2. Legal Considerations
• 3. Technical Requirements for Conversion
• Source Material for Digitization
• Recommended and Minimal Requirements
• Quality Control

10
CD Part 1 (cont.)

• Pre-Depository Storage and Maintenance
• Metadata Requirements
• Descriptive
• Structural Metadata
• Preservation Metadata
• 6. Key Issues for CD 2 role and
  responsibilities of the Depository

11
Stage 4 From Central to Common Depository

• Monitoring and managing distributed resources and
  formats
• A focal point for digital preservation at
  Cornell
• Mapping emerging models and standards to
  institutional context

12
Foundation Documents

• Trusted Digital Repositories Attributes and
  Responsibilities (RLG/OCLC)
• http//www.rlg.org/longterm/repositories.pdf
• OAIS Reference Model (CCSDS)
• www.ccsds.org/documents/pdf/CCSDS-650.0-R-2.pdf
• SIP Transfer Issues
• Producer-Archive Interface Methodology Abstract
  Standard (CCSDS)
• http//ssdoo.gsfc.nasa.gov/nost/isoas/CCSDS-651.0-
  W-1.pdf

13
Foundation Documents

• AIP Components (OCLC/RLG PMWG)
• A Recommendation for Content Information
• A Recommendation for Preservation Description
  Information
• http//www.oclc.org/research/pmwg/
• Format Issues
• Draft Standard - Data Dictionary - Technical
  Metadata for Digital Still Images (NISO)
• http//www.niso.org/committees/committee_au.html

14
RLG-OCLC Attributes of a Trusted Repository

• Administrative Responsibility
• Organizational Viability
• Financial Sustainability
• Technological and Procedural Suitability
• System Security
• Procedural Accountability

15
Connecting the Dots at Cornell

• Mapping 16 requirements in Central Depository to
  the 6 components of RLG/OCLC Attributes
• Not one-to-one relationship
• Not all components of equal value
• Nested relationships

16
1. Administrative Responsibility
2. Organizational Viability
Digital Archives Border
3. Financial Sustainability
6. Accountability (Certification)
4. Technological Suitability
5. System Security
6. Procedural Accountability
17
OAIS

• Open Archival Information System

18
Overview of the OAIS Model
from Reference Model for an Open Archival
Information System 4
19
Mapping RLG/OCLC Attributes to OAIS Reference
Model

• Administrative Framework (RLG/OCLC)
• Process Framework (OAIS)

20
1. Administrative Responsibility
2. Organizational Viability
Digital Archives Border
3. Financial Sustainability
6. Accountability (Certification)
4. Technological Suitability
5. System Security
6. Procedural Accountability
21
Cornell Mapping to the OAIS reference model

• Mapping Producer-Archives Interface Standard to
  draft CD ingest process
• Reviewing SIPs for Euclid and ArXiv, etc.

22
Cornell Mapping to the OAIS reference model

• Mapping the AIP
• CUL Metadata Working Group assessing OCLC/RLG
  reports on content information and preservation
  description information
• DIP and Access ENCompass and Luna Insight

23
Connecting the Dots

• Linking SIP, AIP, and DIP
• Validating requirements
• Integrating preservation considerations and OAIS
  design elements in ENCompass development
• Linking archive implementations
• From Central Depository to Common Depository
  (Stage 4)

24
OAIS at Cornell
Research DPO
ENCompass Luna Insight
Project Euclid ArXiv
Central Depository
25
Stage 5 Into the Future

• Inter-institutional collaboration and dependency

26
From One to Many

• Pre and post-competitive collaboration
• Guns and butter
• Putting money behind the mouth
• Some emerging preservation models
• California Digital Library
• DLF/OCLC registry initiative
• LOCKSS

27
Vertical Archive Silos
Format-based
Institution-based
Publisher-based
28
Cutting Across
Subject-based
Format-based
Institution-based
Publisher-based
29
From LOCKSS to Gridlocks
Subject-based
Format-based
Function-based
Institution-based
Publisher-based
Government-controlled
30
Secure Integrated Digital Preservation Matrix
External Relations and Dependencies
Certification Program
Administrative Responsibility/Viability
Digital Archives Border
Value-Added Services
Financial Model
Program Components
31
Extending Responsibility Beyond the Institution

• Preserving Web Resources
• Building Consortial Digital Archives
• Understanding Interconnectedness
• Developing Methods and Tools for the Matrix
  (prevent, predict, detect, respond, repair)

32
Digital Preservation It just might work.

• Get Smart to Agent 99