NERC Data Grid

1
NERC Data Grid

• Helen Snaithand the NDG consortium

2
What is NDG?

• NDG provides the infrastructure which allows
  users to
• Find data
• Explore what is known about datasets (including
  information about the observing or simulating
  tools)
• Access, manipulate and visualise data!

3
NDG Services

• The NDG Discovery Service
• A database of discovery information
• A web-site, to provide a portal to that database
• A set of web-services, allowing NDG consumers to
  exploit that database
• The NDG Vocabulary Service
• Databases of environmental thesauri and ontology
  tools to map between terms
• Supports machine assisted browsing eg, search
  for rainfall and find datasets with precipitation

4
What Can NERC Data Grid Do For You?

• Depends on who you are!
• Data User - eg scientist looking for data
• Data Provider - eg data centre, large research
  group
• Developer - providing access tools or using data
  sources

5
As a Data User?

• Discovery leads directly to data
• Simple search page or google-like toolbar gives
  direct access to data, not just the metadata
• Comprehensive coverage
• Seamless access
• Hides data format differences and provides
  generic visualisation tools
• Enhances interdisciplinary research

6
As a Data Provider?
NDG Security

• Provides a standards based metadata hierarchy
• Exploit the NDG metadata structures to design a
  metadata system from the ground up, or use them
  as an interoperability tool
• No need to run multiple security systems
• Increases visibility and usage of datasets
• Inclusion in NDG does not prevent inclusion in
  other data discovery or access initiatives
• Minimise user support load

7
As a Developer?

• Exploit NDG web services
• Exploit NDG modular development
• Exploit NDG metadata standards
• NDG developments are non-proprietary
• NDG standards compliance implies less future
  development

8
MetaData Framework

• Discovery
• ISO standard
• MOLES - Metadata Objects for Links in
  Environmental Science
• Detailed metadata for browsing
• CSML Climate Science Markup Language
• For accessing features within data

9
CSML in Action
10
What Cant the NDG Do?

• NDG cant generate the metadata!
• Automated manipulation and searching relies on
  comprehensive metadata, which require quality
  data management to create and maintain
• NDG does provide a clear framework for metadata
  requirements
• NDG doesnt provide information services- it
  provides data services!
• NDG does provide a framework within which it is
  possible to link data to derived information
  services- research publications, reports, policy
  etc..

11
Competitors!How is NDG different from the rest?

• e2edm, End-to-End Data Management
• Similar aims but using different technologies
• NDG is more standards compliant
• Thredds
• Provides only catalogue services
• Thredds is significantly more mature than NDG
• Suitable for data providers with hierarchical
  data / metadata
• OPeNDAP
• A protocol for accessing data
• Limited support for access control
• Requires the user to know exactly what the data
  are

12
NDG Status

• Approaching Beta
• Discovery Gateway
• Discovery Service
• CSML V2
• MOLES
• Security Infrastructure
• Vocabulary Service
• MOLES browser
• Population (MOLES CSML) by NDG partners
• Implementing testing gateways, browsers
  interfaces

ndg.nerc.ac.uk Trac wiki site