The NSDL Program

1
The NSDL Program Stephen Griffin National Science
Foundation
2

• The NSDL Program
• 1996 Vision articulated by NSF's Division of
  Undergraduate Education
• 1997 National Research Council workshop
• 1998 Preliminary grants through Digital
  Libraries Initiative 2
• 1998 SMETE-Lib workshop
• 1999 NSDL Solicitation
• 2000 6 Core Integration demonstration projects
  23 others funded
• 2001 1 large Core Integration System project
  funded
• 2002 More than 60 independent projects funded

3
NSF-funded Research Programs
New ideas
New ideas
Research
4
The NSDL Program
NSF's objective Build a comprehensive digital
library for all aspects of science
education NSF's approach Solicitation encouraged
wide diversity of proposals divided into general
categories Best 60 proposals funded -- more to
follow Grants allow projects flexibility Result
A splendid set of projects A challenge in
interoperability!
5
The NSDL The Challenge of Scale William Y.
Arms Cornell University
6
Core Integration Philosophy
Scientific and technical information
Materials used in education
Materials tailored to education
7
Core Integration Philosophy

• It is possible to build a very large digital
  library with a small staff.
• But ...
• Every aspect of the library must be planned with
  scalability in mind.
• Some compromises will be made.

8
How Big might the NSDL be?

• All branches of science, all levels of
  education, very broadly defined
• Five year targets
• 1,000,000 different users
• 10,000,000 digital objects
• 10,000 to 100,000 independent sites

9
Resources for Core Integration
Core Integration

Budget 4-6 million Staff 25 -
30 Management Diffuse

How can a small team, without direct management
control, create a very large-scale digital
library?
10
Collections the Basic Assumption The Core
Integration team will not manage any collections
11
The NSDL program funds only a fraction of the
relevant collections.
Collections
12
Every Collection is Different
13
The Core Integration Task ...
... to provide a coherent set of collections and
services across great diversity.
14
Interoperability
The Problem Conventional approaches to
interoperability require partners to support
agreements (technical, content, and business But
NSDL needs thousands of very different
partners ... most of whom are not directly part
of the NSDL program The Approach A spectrum of
interoperability
15
Levels of interoperability
Level Agreements Example Federation Strict use
of standards AACR, MARC (syntax, semantic, Z
39.50 and business) Harvesting Digital
libraries expose Open Archives metadata
simple metadata harvesting protocol and
registry Gathering Digital libraries do not
Web crawlers cooperate services
must and search engines seek out information
16
Searching
What to Index? When possible, full text indexing
is excellent, but full text indexing is not
possible for all materials (non-textual, no
access for indexing). Comprehensive metadata is
an alternative, but available for very few of the
materials. What Architecture to Use? Few
collections support an established search
protocol (e.g., Z39.50)
17
Broadcast Searching does not Scale
Collections
User interface server
User
18
The Metadata Repository
Services
The metadata repository is a resource for service
providers. It holds information about every
collection and item known to the NSDL.
Users
Metadata repository
Collections
19
Search Architecture
Metadata repository
Portal
OAI
SDLIP
Search andDiscoveryServices
Portal
http
Portal
Collections
James Allan, Bruce Croft (University of
Massachusetts, Amherst)
20
The Metadata Repository as a Resource
Support for Service Providers

• Records are exposed through Open Archives
  Initiative harvesting protocol.
• Core Integration team will provide some services
  based on the metadata repository.
• The architecture encourages others to build
  services.

21

• Metadata Strategy
• Metadata is expensive
• The NSDL cannot afford to create it manually

22

• Metadata Strategy Support eight standard
  formats
• Collect all existing metadata in these
  formats
• Provide crosswalks to Dublin Core
• Expose records in the metadata repository for
  others to harvest
• Concentrate on collection-level metadata
• Use automatic generation to augment
  item-level metadata

23
Collection-level Operations Material in the NSDL
is selected and managed as collections Alexandr
ia Digital Library Cornell course web
sites JISC Resource Discovery Network Joe's
web page
Human effort will be used to select and integrate
major collections. Automated methods (e.g., web
crawling) will be used to identify and integrate
additional collections.
24
Quality Control
The Problem Material in the NSDL should be
relevant. But we cannot select each item
individually. The Approach Most selection and
quality control decisions are made at a
collection level, not at an item level.
Information about quality will be maintained in
a collection-level metadata record, which is
stored in a central metadata repository. This
metadata is made available to NSDL service
providers. User interfaces can display quality
information.
25
User Interfaces The Problem Cannot handcraft
every web page Must be usable on a very wide
range of equipment and with a very diverse group
of users The Solution Data driven portals using a
channel architecture. Interfaces guide the user
to understand the library. One library, many
portals.
26
(No Transcript)
27
(No Transcript)
28
The Mortal behind the Portal
This space left intentionally blank.
29
Where is the Center of the Universe?
Alexandria
Library of Congress
Elsevier
NSDL
Joe's Pictures
Informedia
Math DL
30
Where is the Center of the Universe?
British Library
Internet Archive
Library of Congress
Elsevier
OCLC
Harvard
NSDL
31
Where is the Center of the Universe?
Google
email
Office
Course web sites
Bill Arms
Directories
News and weather
NSDL
Technical documentation
32
Acknowledgement and Disclaimer
The NSDL is a program of the National Science
Foundation's Directorate for Education and Human
Resources, Division of Undergraduate
Education. The NSDL Core Integration is a
collaboration between the University Center for
Atmospheric Research (Dave Fulker), Columbia
University (Kate Wittenberg) and Cornell
University (Bill Arms). The Technical Director
is Carl Lagoze (Cornell University).
33
The NSDL The Challenge of Scale William Y.
Arms Cornell University