An introduction to the NSDL

1
An introduction to the NSDL William Y.
Arms Cornell University
2
Acknowledgement and Disclaimer
The NSDL is a program of the National Science
Foundation's Directorate for Education and Human
Resources, Division of Undergraduate
Education. The NSDL Core Integration is a
collaboration between the University Center for
Atmospheric Research (Dave Fulker), Columbia
University (Kate Wittenberg) and Cornell
University (Bill Arms). The ideas discussed in
this talk do not represent the official views of
the NSF or the Core Integration team.
3

• The NSDL project
• 1996 Vision articulated by NSF's Division of
  Undergraduate Education
• 1997 National Research Council workshop
• 1998 Preliminary grants through Digital Libraries
  Initiative 2
• 1998 SMETE-Lib workshop
• 1999 NSDL Solicitation
• 2000 6 Core Integration System projects 23
  others funded
• 2001 Further collection and service projects 1
  Large Core Integration System project (total
  about 25 million/year)
• 2002 Formal release
• 2006 End of formative phase

4
Continuing questions (a) Science education How
broadly defined? (b) Funding How much with how
few dollars? (c) Education How can the NSDL
have an impact? (d) Management How can a
diverse community provide shared services?
5
Science education scope of a digital library
6
The NSF's strategy
7
NSDL collections funded by the NSF (a)
Focused collections
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
NSDL-collections funded by the NSF (b)
Aggregates and federations
12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
NSDL service projects funded by the NSF
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
Core Integration demonstrations (2000-2001)
20
(No Transcript)
21
(No Transcript)
22
The broad view of the NSDL
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
How big might the NSDL be?

• All branches of science, all levels of
  education, very broadly defined
• Five year targets
• 1,000,000 different users
• 10,000,000 digital objects
• 10,000 to 100,000 independent sites

28
The NSF cannot fund all collections
29
The Core Integration task ...
... to provide a coherent set of services across
great diversity.
30
Resources
Core Integration

Budget 4 million Staff 25 -
30 Management Diffuse

How can a small team, without direct management
control, create a very large-scale digital
library?
31
A spectrum of interoperability
32
Approaches to interoperability
The conventional approach ? Wise people develop
standards protocols, formats, etc. ? Everybody
implements the standards. ? This creates an
integrated, distributed system.
Unfortunately ... ? Standards are expensive to
adopt. ? Concepts are continually changing. ?
Systems are continually changing.

33
Interoperability is about agreements
Technical agreements cover formats, protocols,
security systems so that messages can be
exchanged, etc.   Content agreements cover the
data and metadata, and include semantic
agreements on the interpretation of the messages.
  Organizational agreements cover the ground
rules for access, for changing collections and
services, payment, authentication, etc. The
challenge is to create incentives for independent
digital libraries to adopt agreements
34
Function versus cost of acceptance
Cost of acceptance
Few adopters
Many adopters
Function
35
Example textual mark-up
Cost of acceptance
SGML
XML
HTML
Function
ASCII
36
Levels of interoperability
Level Agreements Example Federation Strict use
of standards AACR, MARC (syntax, semantic, Z
39.50 and business) Harvesting Digital
libraries expose Open Archives metadata
simple metadata harvesting protocol and
registry Gathering Digital libraries do not
Web crawlers cooperate services
must and search engines seek out information
37

• Metadata is expensive
• The NSDL cannot afford to create it manually

38
Every collection is different
39
The metadata repository
Services
The metadata repository is a resource for service
providers. It holds information about every
collection and item known to the NSDL.
Users
Metadata repository
Collections
40

• Metadata strategy Support eight standard
  formats
• Collect all existing metadata in these
  formats
• Provide crosswalks to Dublin Core
• Expose records in the metadata repository for
  others to harvest
• Concentrate on collection-level metadata
• Use automatic generation to augment
  item-level metadata

41

• NSDL metadata options Eight standard
  formats
• Dublin Core
• Dublin Core DC-Ed extensions
• LTSC (IMS)
• ADL (SCORM)
• MARC 21
• Content Standard for Digital Geospatial Metadata
  (FGDC)
• Global Information Locator Service (GILS)
• Encoded Archival Description (EAD)
• For additional information on supported formats
• http//128.253.121.110/NSDLMetaWG/IntroPage.html

42
Services strategy
43
The metadata repository as a resource

• Records will be exposed through Open Archives
  Initiative harvesting protocol.
• Core Integration team will provide some services
  based on the metadata repository.
• The architecture encourages others to build
  services.

44
Information retrieval Basic metadata
search Basic content search Combining metadata
and content James Allan, Bruce Croft
(University of Massachusetts, Amherst)
45
How search service fits into the NSDL
Provides search and discovery
functionality to portals
Metadata repository
Portal
OAI
SDLIP?
Search andDiscoveryServices
Portal
http?
Portal
Content
46

• Extending the architecture to support federations
• Extending the spectrum of search interoperability
• collections with non-DC metadata schemas
• distributed and heterogeneous collections
• richer search functionality
• geospatial search, thesaurus/concept space
  search, ...
• Supporting the creation of new and personalized
  collections
• Providing access to thesaurus and gazetteer
  services
• Terry Smith, Jim Frew (University of California,
  Santa Barbara)

47
The ADEPT approach to search interoperability
metadata repository
harvest
OAI
portal
2. harvest interpret
3. h i
metadata
ADEPT
1. map
ADEPT collection discovery
ADEPT client
ADEPT
per collection provider
48

• User profiles and authentication
• User authentication
• User registry
• Affiliations
• Privacy
• User preferences
• User Interfaces and portals
• Enable customizable user interface
• Rights management
• Kate Wittenberg, David Millman (Columbia
  University)

49
Conclusion The NSDL cannot do everything
50
Opportunities for the NSDL Categories of
material that have been given lower priority by
libraries and publishers, e.g., datasets,
software, and other dynamic content,
... Materials that are accessible for automatic
processing, e.g., scientific web sites and
databases, image collections, ... Materials
designed for education, e.g.,learning objects,
curricula, problem sets, ... Less opportunity for
the NSDL Conventional scientific literature
with restricted access