University of Illinois OAI Metadata Harvesting Service

1
University of IllinoisOAI Metadata Harvesting
Service

• Timothy W. Cole
• University of Illinois at UC
• 18 November 2001
• DLF Fall Forum
• http//oai.grainger.uiuc.edu/presentations.htm

2
Presentation Outline

• OAI Protocol for Metadata Harvesting
• Objectives of the Illinois Project
• Tools Under Development
• Initial Harvesting Activities
• Next Steps
• Search Interface Design
• Mapping EAD to Dublin Core
• Usability Evaluation

3
OAI Protocol for Metadata Harvesting

• Harvesting Approachto Interoperability
• Provider Services Harvesting Services
• Relies on HTTP XML
• Applicability BeyondE-Print Archives
• http//www.openarchives.org/

4
OAI PMH (continued)

• Records Header, Metadata, Conditions of Use
• Verbs Used in OAI Requests
• Identify Repository Collection-Level Metadata
• ListMetadataFormats Per Repository or Record
• ListSets Names and Labels of Repository Sets
• ListIdentifiers Limit by Set and/or Datestamp
• ListRecords Limit by Set and/or Datestamp
• GetRecord Specify Identifier Metadata Prefix

5
OAI PMH (continued)

• Protocol Technical Review (Current)

6
Illinois Project Objectives

• Investigate Applicability of OAI-PMH to Domain of
  Cultural Heritage Information Resources
• Create Make Available Tools for OAI Harvesting
• Implement Test Prototype Harvesting Service
• Investigate Advanced Interface Indexing
  Features in Context of OAI-PMH
• Identify Community "best practice" Covenants
  Conventions Needed to Complement OAI-PMH
• Research Other Critical Concerns Issues

7
Objectives (continued)

• Special Emphasis on
• CIC Resources EAD Finding Aids
• Normalization Across Controlled Vocabularies
• Effective Presentation of Results Links From
  Results to Related Resources
• Utility for Higher Ed K-12 Communities
• Coordinated With Univ. of Michigan Project
• Shared Harvesting Indexing Tools
• Jointly Sponsored Workshops

8
Tools Under Development

• OAI Harvester for Windows 2000
• 3 Components
• Crawlers Fetch Store Metadata
• Harvest Manager Initiates Manages Crawlers
• User Interface Sets Harvest Schedule Options
• Supports Concurrent Crawler Instances
• Web-Based User Interface (Java Applet)
• LINUX Version to Follow
• Harvester Demonstration

9
Tools (continued)

• 3 Sample OAI Provider Service Implementations

Descriptive MetadataStored
OAI Administrative Metadata (IDs, Dates)
XML
RelationalDBMS
RelationalDBMS
OAI Application (ASP / VBScript)
OAI Harvesters
HTTP (Microsoft IIS)
10
Initial Harvesting Activities
11
Initial Harvesting (continued)

• Total of 380,000 Records Harvested So Far
• Simple Search Interface
• Other Data (in locally created prototype OAI
  Provider Services)
• EAD Finding Aids Examples From
• Michigan State University
• Online Archive of California
• Bentley Historical Society (University of
  Michigan)
• University of Texas, TARO Project
• MARC Records from Illinois State Library (600,000)

12
Next Steps Search Interface Design

• Scenario Based Design (J. M. Carroll)
• Higher Education
• K-12 Teachers Students
• Web Established HCI Design Conventions(J.
  Nielsen, B. Schneiderman, etc.)
• Basic Advanced Search Screens
• Field-Specific, Keyword Phrase Searching
• Collection Sub-Domain Search Filters
• Stored Expert Searches
• Visual Search Interface Techniques (e.g., VisIT)
• Incorporate Value-Added Content at Indexing
• E.g., Mappings Between Controlled Vocabularies

13
Next Steps Mapping EAD to DC

• Strategies for Mapping EAD Finding Aids to Dublin
  Core
• Current Approach Map Only Data from EADHEADER
  ARCHDESC Elements Above DSC Element
• How to Export Metadata Contained in DSC
  (Description of Subordinate Components)?
• One-to-One vs. One-to-Many Mappings
• Determining Level of CO1, CO2, CO3, Elements
• Inconsistent Practices in Application of EAD
• Presentation of Search Results

14
Next Steps Usability Evaluation

• January, 2002 Initial Availability of Search
• Over-the-Shoulder Usability Testing
• Observe Participants Perform Assigned Tasks
• Analyze Participant Feedback, Search Logs
• Iterative Design, Repeat Testing
• Evaluative Comments Solicited From GSLIS Faculty
  Graduate Students
• Anonymous Transaction Logs Throughout

15
Further Information

• Tim Cole(t-cole3_at_uiuc.edu)
• Joanne Kaczmarek (jkaczmar_at_uiuc.edu)
• http//oai.grainger.uiuc.edu/