DEVELOPING AN ISO REFERENCE MODEL FOR AN OPEN ARCHIVAL INFORMATION SYSTEM OAIS Presentation to Socie

1
Ingest Report October 14, 1999
2
Ingest Methodology Standard

• Mike Martin paper considered a starting point for
  developing Ingest Methodology Standard
• Many justifications for the effort
• Support development of training materials for
  data producers
• Encourage standardization of archive submission
  information
• Identifies the respective players and their roles
  to improve efficiency of interactions
• Increase awareness of existing standards for
  ingest process
• Identify resources that archives should provide
  to support the ingestion process
• Identify resources that data producers need to
  submit to an archive
• Facilitate prediction of usage patterns and
  improve archive planning of storage requirements
• Improve ability of archives to evaluate whether
  proposed submission is a good candidate for
  archiving
• Highlights the role that archivists need to be
  involved early in the data production process
• Allows the archive to be viewed more as an
  'editor' to promote maximally usable products
• Building a foundation for good archiving
  practices
• Provides a framework for supporting a
  certification process in the ingest area
• Allows the early assessment of possible trouble
  points, such as unacceptable formats
• Clarifies a division of responsibility between
  data producer and archive and channels of
  communication
• Maximize the quality of data arriving at the
  archive
• Improve the long term preservability of the data
  at the archive
• Provides some potential cost savings by
  increasing efficiency

3
Mikes paper The Archive Ingest Process

• Methodology for archive ingest process
• Identifies key OAIS ingest functions
• Primarily a data producer view
• Six steps for data producer interaction with
  archive
• Orientation - finding out what archive will
  expect
• Archive Planning - deciding what to archive, when
  and generally how
• Design - determining what needs to be included in
  the Submission Information Package
• Review - the final archive quality check
• Delivery - transferring the Submission
  Information Package to the archive

4
First 2 Steps - Examples

• Orientation
• Establishes contact with archive
• Provides general information to archive
• Obtains archive orientation materials
• Establishes technical contacts
• Archive Planning
• Prepare a Producer Data Management Plan (PDMP)
• Prepare a Submission Agreement (SA)
• Plan for updates to PDMP and SA
• Keep archive data engineer informed
• Participate in planning meetings
• Review/sign off archive interface plan

5
Middle 2 Steps - Examples

• Archive Design
• Review archive standards
• Design data products and representation
  information
• Design the data set or collection
• Design volumes and volume sets
• Design data production process
• Plan data validation process
• Prepare Preservation Description Information
• Data set assembly and validation
• Create data products
• Prepare Volume Components
• Execute data validation procedures
• Transfer data to final medium

6
Final 2 Steps - Examples

• Review
• Establish a review committee
• Prepare for the review
• Conduct review
• Correct/document review liens
• Delivery
• Coordinate generation of duplicate copies (in
  needed)
• Data classification (restriction) procedures
• Transfer volumes to archive
• Update data sets with corrections or enhancements

7
Ingest Methodology Standard Workpackage

• Developed Ingest Methodology Workpackage based on
  template provided
• Mikes Ingest Methodology paper seen as a base
  document for the effort
• Includes initial list of possible participants

8
Ingest Format Considerations

• Highlighted by Holdsworth and Reagan papers
• Undocumented / unavailable formats
• Dependency on external entities for access to
  data
• Conversion to open formats
• Open /documented formats
• May be some rights limitations
• Optimal case
• Conversion to new formats may be needed
• Encapsulation standards
• Supports adding attributes to manage and ensure
  adequate documentation

9
Conversions

• Conversions on Ingest , or on dissemination?
• Offers opportunity to reduce expense to have
  archival forms already in Consumer usable form
• May want original form as well as optimal form
  for Consumer (du jour format)
• Possible information loss - is it tolerable?
• Modified (irreversible)
• Modified but reversible (logically/adequate
  meaning preserved)
• Modified but reversible (bit for bit)
• Unmodified (Encapsulated)

10
Possible Archive Information Unit
(AIU) Implementation of OAIS Concept

• Containerization of attribute object and primary
  digital object
• Attribute Object contains
• Archival Id for Unit
• Id for attribute object representation
  (format,meaning)
• Standard and optional Archive attributes
• Optional Producer attributes
• Id for primary digital object representation
• Primary Digital Object contains
• Bit sequence of primary digital object

Attribute Object
Primary Digital Object