PeDALS Persistent Digital Archives

1
PeDALSPersistent Digital Archives Library
System

• Richard Pearce-Moses
• Deputy Director for Technology Information
  Resources
• Arizona State Library, Archives and Public Records

2
Curatorial Rationale

• Transformation of traditional, paper-based
  practices into the digital arena
• Open Archival Information System (OAIS)

• Acquisition
• Arrangement description
• Housing storage
• Reference and access
• Preservation

• Ingest
• Storage
• Data management
• Preservation
• Access

3
Data Flow
4
Middleware Microsoft BizTalk

• Automated business rules
• Transforming SIPs to AIPs and DIPs
• Mapping, generating metadata
• Connecting multiple databases (glue)
• Many OOOs
• One repository
• Allows communication between systems
• Validation

5
1. OOO Recordkeeping System

• For each series of records OOO and repository
• Negotiate metadata you will receive
• Negotiate format of the records (TIFF, PDF, XML)
• Negotiate format of the submission information
  package
• Negotiate frequency and manner of transfer
• OOO develops procedures to create SIPs
• Metadata, Record
• Shipping manifest with hash and file names

6
Submission Information Packages

• OOO Metadata
• Well number" , "Owner" , "Title" , "File name"
• "56-000001","CITY OF TUCSON","2003 annual
  report","56 files\56-000001_0000.pdf"
• "56-000001","CITY OF TUCSON","2004 annual
  report","56 files\56-000001_0000_E52B0.pdf"
• "56-000001","CITY OF TUCSON","2005 annual
  report","56 files\56-000001_0000_E8578.pdf"
• "56-000001","CITY OF TUCSON","2006 annual
  report","56 files\56-000001_0000_EC3F8.pdf"
• Records
• XML
• PDF
• Other formats

7
2. Ingest Transfer to Drop Box

• Transfer to a drop box in DMZ
• FTP
• Tape
• Disk
• Isolated for virus scanning
• Validation
• Were all records received without corruption?
• Were any false records received?

8
3. Data Management Metadata

• Generate core metadata
• Administrative (6 elements)
• Descriptive (28 elements)
• Preservation (12 elements)
• Stored in Accessions Register
• MS SQL Server

9
Administrative Metadata

• Information created by repository to track
  records in the system
• Accession Number
• Transfer Authority
• Acquisition Ingest Identifier
• Acquisition Date
• Unique Item Identifier
• Item Location

10
Discovery Metadata

• Information created by OOO or Repository to help
  retrieving records for a variety of purposes

• Office of Origin, Variant name
• Source
• Series Title, ID
• Series Dates
• Series Extent
• Series Description
• Arrangement
• Restrictions
• Series Subjects, Keywords
• Activity

• Item Title
• Originator ID
• Item Extent
• Item Date
• Item Description
• First1024
• Party and Role, Subjects, Location
• Item Keywords, Form/Genre
• Related Item
• Language
• Open Date

11
Preservation Metadata

• Information created by Repository to support to
  protect integrity, support readability over time

• Access Facilitators
• Operating System
• Access Inhibitors
• Hardware
• Exceptions
• Signature Information

• File Description
• Fixity
• Functionality
• Software
• Structural Type
• Technical Infrastructure

12
Mapping and Creating Metadata
13
Mapping and Creating Metadata
14
4. Storage

• Create AIP
• ltAIPgt ltHashgt lt/Hashgt ltCoreMetadatagt
  lt/CoreMetadatagt ltMetadatagt lt/Metadatagt ltRecordgt
  lt/Recordgtlt/AIPgt
• Deposit in Digital Stacks (LOCKSS)
• Generate manifest list to expose to LOCKSS
• LOCKSS harvests from manifest server

15
Why LOCKSS?

• Benefits
• Automatic integrity checking
• Automate error-correction
• Geographically dispersed copies
• Bitstream preservation
• Committed community of support
• Hardened operating system
• Concerns
• Maximum number of objects in a Unix file system
• Community of support is small

16
4. Access

• DIPs for public access
• No administrative, preservation metadata
• Formats supported by common browsers
• Website
• Records not confidential (by law)
• SQL query engine with discovery metadata
• Limited access website
• In repository, selected locations
• Record series with personally identifying
  information

17
5. Preservation

• Bitstream preservation
• Developing audit procedures
• Periodic validation of dark archives against
  accession register
• For future development
• Capturing minimum preservation metadata
• On-the-fly rendering tools
• Long-term format migration

18
Community of Shared Practice

• Personal Relationships
• Challenge of building relationships over the
  Internet
• Lack of rich, immediate feedback in communication
• Lack of spontaneity, serendipity, play
• Inter-Agency Relationships
• Different practices
• Laws and regulations
• Money

19
(No Transcript)
20
For more information

• http//rpm.lib.az.us/PeDALS/
• Principal Investigator
• Richard Pearce-Moses
• Project Coordinator
• Sara Muth
• State Partner Leads
• Florida Mark Flynn
• New York Bonnie Weddle
• South Carolina Bill Henry
• Wisconsin Helmut Knies