DEVELOPING AN ISO REFERENCE MODEL FOR AN OPEN ARCHIVAL INFORMATION SYSTEM OAIS Presentation to Socie

1
ISO Reference Model For anOpen Archival
InformationSystem (OAIS)Presentation to
ESDIS Don Sawyer /NASA/NSSDC July 26, 1999
2
Outline of Talk

• History
• Reference Model overview
• Digital Archive Directions (DADs) workshop
• Reference Model Status
• Follow-on Efforts

3
NASA Role

• Participation from multiple centers/disciplines
• NSSDC, ESDIS, PDS, Life Sciences
• Consultative Committee for Space Data Systems
• International group of space agencies
• Developed variety of science discipline-
  independent standards
• Became working body for an ISO TC 20/ SC 13 about
  1990
• TC20 Aircraft and Space Vehicles
• SC13 Space Data and Information Transfer Systems

4
Initial Archive Standards Proposal

• ISO suggested that SC 13 should develop archive
  standards
• Address data used in conjunction with space
  missions
• Address intermediate and indefinite long term
  storage of digital data

5
Response

• Response to Consultative Committee for Space Data
  Systems (CCSDS) and ISO TC 20/SC 13
• No framework widely recognized for developing
  specific digital archive standards
• Begin by developing a Reference Model to
  establish common terms and concepts
• Ensure broad participation, including traditional
  archives
• (Not restricted to space communities all
  participation is welcome!)
• Focus on data in electronic forms, but recognize
  that other forms exist in most archives
• Follow up with additional archive standards
  efforts as appropriate

6
Getting Started

• First open US workshop held October 1995
• Variety of government, academic, and industry
  participation, including National Archives
• Active US working group was formed
• US workgroup activities are fully open
• New participants always welcome
• Plans, minutes, drafts available from Web
• Broad international workshops also held
• Britain and France
• Issue resolution at CCSDS international workshops

7
Results

• Reference Model targeted to several categories of
  reader
• Archive designers
• Archive users
• Archive managers, to clarify digital preservation
  issues and assist in securing appropriate
  resources
• Standards developers
• Adopting terminology that crosses various
  disciplines
• Traditional archivists
• Scientific data centers
• Digital libraries
• Getting favorable comments wherever exposed

8
Reference Model for anOpen Archival Information
System
9
Open Archival Information System (OAIS)

• Open
• Reference Model standard(s) are developed using a
  public process and are freely available
• Information
• Any type of knowledge that can be exchanged
• Independent of the forms (i.e., physical or
  digital) used to represent the information
• Data are the representation forms of information
• Archival Information System
• Hardware, software, and people who are
  responsible for the acquisition, preservation and
  dissemination of the information
• Additional OAIS responsibilities are identified
  later and are more fully defined in the Reference
  Model document

10
Document Organization

• Introduction
• Purpose and Scope, Applicability, Rationale, Road
  Map for Future Work, Document Structure, and
  Definitions of Terms
• OAIS Concepts
• High level view of OAIS functionality and
  information models
• OAIS external environment
• Minimum responsibilities to become an OAIS
• Detailed Models
• Functional model descriptions and information
  model perspectives
• Preservation perspectives
• Media replacement, repackaging, format
  conversions, access preservation
• Archive Interoperability
• Criteria to distinguish types of cooperation
  among archives
• Annexes (not part of the standard)
• Scenarios of existing archives, relationships
  with other standards, UML guide, informative
  references, layered model, composite functional
  view

11
Purpose, Scope, and Applicability

• Framework for understanding and applying concepts
  needed for long-term digital information
  preservation
• Long-term is long enough to be concerned about
  changing technologies
• Starting point for model addressing non-digital
  information
• Provides set of minimal responsibilities to
  distinguish an OAIS from other uses of archive
• Framework for comparing architectures and
  operations of existing and future archives
• Basis for development of additional related
  standards
• Addresses a full range of archival functions
• Applicable to all long-term archives and those
  organizations and individuals dealing with
  information that may need long-term preservation
• Does NOT specify any implementation

12
Model View of an OAISs Environment

• Producer is the role played by those persons, or
  client systems, who provide the information to be
  preserved
• Management is the role played by those who set
  overall OAIS policy as one component in a broader
  policy domain
• Consumer is the role played by those persons, or
  client systems, who interact with OAIS services
  to find and acquire preserved information of
  interest

OAIS (archive)
Producer
Consumer
Management
13
OAIS Information Definition

• Information is defined as any type of knowledge
  that can be exchanged, and this information is
  always expressed (i.e., represented) by some type
  of data
• In general, it can be said that Data interpreted
  using its Representation Information yields
  Information
• In order for this Information Object to be
  successfully preserved, it is critical for an
  archive to clearly identify and understand the
  Data Object and its associated Representation
  Information

Interpreted Using its
Yields
Data Object
Representation Information
Information Object
14
Information Package Definition
Preservation Description Information
Content Information

• An Information Package is a conceptual container
  of two types of information called Content
  Information and Preservation Description
  Information (PDI)

15
External Data Flow Diagram
Producer
Submission Information Packages
OAIS
queries
result sets
orders
Dissemination Information Packages
Consumer
16
OAIS Responsibilities

• Negotiates and accepts Information Packages from
  information producers
• Obtains sufficient control to ensure long-term
  preservation
• Determines which community (designated) needs to
  be able to understand the information provided
• Ensures the information to be preserved is
  independently understandable to the Designated
  Community
• Follows documented policies and procedures which
  ensure the information is preserved against all
  reasonable contingencies
• Makes the preserved information available to the
  Designated Community in forms understandable to
  that community

17
Detailed Models

• Overview

18
OAIS Functional Entities
Data Management
C O N S U M E R
Queries, orders
P R O D U C E R
Ingest
Access
Result sets
SIP
DIP
Archival Storage
Administration
MANAGEMENT
SIP Submission Information Package AIP
Archival Information Package DIP Dissemination
Information Package
19
OAIS Archival Information Package
Archival Information Package (AIP)
delimited by
describes
Packaging Information
Package Descriptor
described by
identifies
e.g., How to find Content Information and PDI
on some medium
e.g., Information supporting customer searches
for AIP
Preservation Description Information (PDI)
Content Information
further described by
e.g., Hardcopy document Document as an
electronic file together with its format
description Scientific data set consisting
of images and text in three electronic files
together with format descriptions
e.g., How the Content Information came into
being, who has held it, how it relates to
other information, and how its integrity is
assured
20
Content Information
Content Information
Interpreted using

Interpreted using
Data Object
Representation Information
1
Physical Object
Digital Object
1
1..
Bit
21
Preservation Description Information

• Provenance Information
• Describes the source of Content Information, who
  has had custody of it, what is its history
• Context Information
• Describes how the Content Information relates to
  other information outside the Information Package
• Reference Information
• Provides one or more identifiers, or systems of
  identifiers, by which the Content Information may
  be uniquely identified
• Fixity Information
• Protects the Content Information from
  undocumented alteration

22
Information Preservation

• Digital Migration is defined to be the transfer
  of digital information, while intending to
  preserve it, within the OAIS.
• Focus on preservation of the full information
  content
• New information implementation replaces the old
• OAIS has full control and responsibility over all
  aspects of the transfer
• Three major motivators are seen to drive Digital
  Migrations of Archival Information Packages
  within an OAIS
• Media Decay
• Increased Cost Effectiveness
• New Consumer Service Requirements

23
Digital Migration Approaches

• Four primary types of digital migration in
  response to motivators, ordered by increasing
  risk of information loss
• Refreshment
• Media replacement with no bit changes
• Replication
• No change to Packaging Information or Content
  Information/PDI bits
• Repackaging
• Some bit changes in Packaging Information
• Transformation
• Reversible Bit changes in Content
  Information/PDI are reversible by an algorithm
• Non-reversible Bit changes in Content
  Information/PDI are not reversible by an
  algorithm

24
Archive Interoperability Motivators

• Users of multiple OAIS archives have reasons to
  wish for some interoperability or cooperation
  among the OAISs.
• Consumers
• Common finding aids to aid in locating
  information over several OAIS archives
• Common Package Descriptor schema for access
• Common DIP schema for dissemination
• Single global access site.
• Producers
• Common SIP schema for submission to different
  archives
• Single depository for all their products.
• Managers
• Cost reduction through sharing of expensive
  hardware
• Increasing the uniformity and quality of user
  interactions across OAISs

25
Categories of Archive Interactions

• Independent
• No knowledge by one OAIS of standards
  implemented at another
• Cooperating
• Potentially common submission standards, and
  common dissemination standards, but no common
  access. One archive may make Event Based Order
  requests for key data at the cooperating archive
• Federated
• Access to all federated OAIS is provided through
  a common set of access aids that provide
  visibility into all participating OAISs. Global
  dissemination and ingest are options
• Shared resources
• An OAIS in which Management has entered into
  agreements with other OAISs to share resources.
  This requires various standards internal to the
  archive (such as ingest-storage and
  access-storage interface standards), but does not
  alter the communitys view of the archive

26
Federated Archives
Local
Consumer
Dissemination Information Package
(Optional)
Access
Access
Global
Ingest
Access
Consumer
OAIS 1
DM
Adm
Common Catalog
DM
Adm
OAIS 2
Access
Ingest
Access
Local
Dissemination Information Package
(Optional)
Consumer
27
Reference Model Summary

• Reference model is to be applicable to all
  digital archives, and their Producers and
  Consumers
• Identifies a minimum set of responsibilities for
  an archive to claim it is an OAIS
• Establishes common terms and concepts for
  comparing implementations, but does not specify
  an implementation
• Provides detailed models of both archival
  functions and archival information
• Discusses OAIS information migration, access
  preservation, and interoperability among OAISs

28
Reference Model Acceptance

• Reference model has been getting good reviews
• Society of American Archivists 1997 annual
  meeting
• NAGARA 1998 Annual Meeting
• Various international conferences
• Recently adopted by European deposit libraries
  project NEDLIB
• Major context for June 1998 Digital Archive
  Directions (DADs) workshop hosted by NARA
• Attended by representatives of international and
  US national archives, science data centers,
  digital libraries, and academic institutions
• Provided strong support for model and offered
  improvements
• http//ssdoo.gsfc.nasa.gov/nost/isoas/dads/

29
DADs Workshop Recommendations

• Working groups identified several best
  practices and standards desired, including
• Recommended data ingest methodology
• Best practices for digitizing analogue data
• Best practices for media selection, testing and
  usage
• Best practices for error control through the
  archive
• Archival Submission Information Package standard
• Consumer Archive Interface standard
• Unique Archival Information Package Identifier
  standard
• AIP Content Layered Model and Standard APIs
  report
• Plenary recommended a coordination function be
  established
• Promote the reference model
• Promote coordination of work on best practices
  and standards
• Promote development of an archive accreditation
  method

30
Reference Model Status

• Ultimate success of OAIS Reference Model effort
  depends on obtaining adequate review and comment
• Reference Model Red Book/ISO Draft International
  Standard (DIS) expected July 30, 1999
• Formal NASA/GSFC review will be announced in
  August
• Comments desired by September 30, 1999
• Send to donald.sawyer_at_gsfc.nasa.gov
• Currently available version is White Book 5.0,
  available under Reference Materials heading, at
• http//ssdoo.gsfc.nasa.gov/nost/isoas/overview.htm
  l

31
Follow-On Efforts

• Soliciting interest in developing/adopting new
  archiving standards
• Ingest methodology/standards
• Identification of Archival Information Packages
• Accreditation of archives
• Workshop proposed for October 13-14 at NARA
• Depends on level of interest
• E-mail solicitation notice is available