NDG Status

1
NDG Status
Bryan Lawrence (on behalf of a big team)






BADC, BODC, CCLRC, PML and SOC
2
Outline

• Motivation
• Standards
• Feature Types
• NDG Products
• Overall Architecture (Taxonomy, Deployment)
• Discovery Portal
• Data Extractor
• MOLES
• (Security, NumSim)
• CSML
• CSML
• Description
• Prototyping in MarineXML
• Round-Tripping
• NDG Timeline

3
Complexity Volume Remote Access Grid
Challenge
British Atmospheric Data Centre
http//ndg.nerc.ac.uk
British Oceanographic Data Centre
4
Integration semantics

• Want interdisciplinary semantic access to
  information, not abstract data
• getData(potential temperature from ERA-40 dataset
  in North Atlantic from 1990 to 2000)
• not getData(era40.nc, PTMP, 2050, 300340,
  190200)
• or even worse
• for j19902000
• getData(era40_j.nc, PTMP, 2050, 300340)
• Lossy is OK!
• Care less about completeness of representation
  than semantic unification

5
Integration Access Control

• Want decentralised access to resources, implies
• Data lies in
• Multiple Locations
• Protected by different user databases
  (/etc/passwd, whatever)
• Differing standards for access control (from
  none, to restricted to license holders)
• No overall agreement on access control roles!
• No central user database
• Needs to be scalable.
• When NDG started there was NO grid based paradigm
  that met these requirements.
• Now Shibboleth and/or Permis meet some of these
  requirements, but we have designed and prototyped
  a unique implementation of a new security
  infrastructure.

6
Standards

• ISO 19101 Geographic information Reference
  model

7
Standards

• Geographic features
• abstraction of real world phenomena ISO 19101
• Type or instance
• Encapsulate important semantics in universe of
  discourse
• Something you can name
• Application schema
• Defines semantic content and logical structure
• ISO standards provide toolkit
• spatial/temporal referencing
• geometry (1-, 2-, 3-D)
• topology
• dictionaries (phenomena, units, etc.)
• GML canonical encoding

from ISO 19109 Geographic information Rules
for Application Schema
8
Standards

• ISO standards
• TC211 around 40 standards for geographic
  information
• Cover activity spectrum discovery ? access ? use
• Provide a framework for data integration

9
Architecture NDG Metadata Taxonomy
not one schema, not one solution!
10
Architecture Deployment
11
Discovery Service
NDG Products Discovery Portal
http//ndg.nerc.ac.uk/discovery
NB Web Service Interface (you can do the search
from your own site and format and present the
results there!
12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
NDG Products MOLES
Ugly as sin! A hint of things to come
17
MOLES implementation

• Core linking concept is the deployment

of a Data Production Tool
at an Observation Station
on behalf of an Activity
that produces a Data Entity
Activity
DataProductionTool
ObservationStation
Links the metadata records into a structure that
can be turned into a navigable structure
Deployment
Each of the main metadata objects has security
data attached to it. This means that this can be
applied to queries on the metadata
Data Entity
18
Simulators as data production tools NumSim
NDG Products NumSim
19
(No Transcript)
20
NDG Security
21
NDG Products DataExtractor
22
(No Transcript)
23
(No Transcript)
24
NDG Products GEOSPLAT
25

• ERA40
• All driven from one CDML file, 9 TB online
  spherical harmonics, looking like 40 TB virtual
  gridded!

26
NDG-A Climate Science Modelling Language

• Aims
• provide semantic integration mechanism for NDG
  data
• explore new standards-based interoperability
  framework
• emphasise content, not container
• Design principles
• offload semantics onto parameter type
  (phenomenon, observable, measurand)
• e.g. wind-profiler, balloon temperature sounding
• offload semantics onto CRS
• e.g. scanning radar, sounding radar
• sensible plotting as discriminant
• in-principle unsupervised portrayal
• explicitly aim for small number of weakly-typed
  features (in accordance with governance principle
  and NDG remit)

27
Climate Science Modelling Language

• CSML feature types
• defined on basis of geometric and topologic
  structure

28
Climate Science Modelling Language

• CSML feature types
• examples...

29
MarineXML Testbed
For each XSD (for the source data) there is an
XSLT to translate the data to the Feature Types
(FT) defined by CSML. The FTs and XSLT are
maintained in a MarineXML registry
Phenomena in the XSD must have an associated
portrayal
Data from different parts of the marine community
conforming to a variety of schema (XSD)
The FTs can then be translated to equivalent FTs
for display in the ECDIS system
XSD
XML
Biological Species
S52 Portrayal Library
XSD
XML
Chl-a from Satellite
XML Parser
MarineGML(NDG) Feature Types
XSLT
XML
XSLT
XSLT
SeeMyDENC
SENC
XSD
MeasuredHydrodynamics
XML
XSLT
XML
XSLT
XSLT
ECDIS acts as an example client for the data.
XSD
Data Dictionary
XML
ModelledHydrodynamics
The result of the translation is an encoding
that contains the marine data in weakly typed
(i.e. generic) Features
Features in the source XSD must be present in the
data dictionary.
XSD
Feature described using S-57v3.1Application
Schema can be imported and are equivalent to the
same features in CSML
XML
S-57v3 GML
Slide adapted from Kieran Millard (AUKEGGS, 2005)
30
MarineXML Testbed
Biological sampling station with attributes for
the species sampled at each
Grid of Chl-a from the MERIS instrument on ENVISAT
Predicted and measured wave climate timeseries
(height, direction and period)
Vectors of currents from instruments
Slide adapted from Kieran Millard (AUKEGGS, 2005)
31
The Concept of re-using Features
Here structured XML is converted to plain ascii
text in the form required for a numerical model
HTML warning service pages are generated on the
fly
Here the same XML is converted to the SENC format
used in a proprietary tool for viewing electronic
navigation charts.
XML can also be converted to SVG to display data
graphically
Slide adapted from Kieran Millard (AUKEGGS, 2005)
32
CSML Round Tripping - 1
Managing semantics
33
CSML Round Tripping - 2
Managing data - 1
34
Managing Data 2
scanner
XSLT
PUBLISH
ISO19115
35
NDG Timeline

• NDG2 runs until September 2007
• NDG-Alpha (June 2006)
• Not all components in place (particularly
  delivery broker)
• Not many (maybe only DX) products will be
  deployable by non-NDG participants
• (too much hard work installing things that
  havent been optimised for installation)
• Discovery portal will be (is now) usable, linking
  to NCAR data etc, but isnt very user friendly
  (options not obvious etc).
• NDG-Beta (Feb 2007)
• Most components should work, but deployment of
  software may still be difficult by
  non-participants
• NDG-Prod (Jun 2007)
• Should be deployable and far more user friendly
  (spending from Feb-June working on deployment and
  friendliness, no new functionality)
• Last few months working on sustainability etc

http//proj.badc.rl.ac.uk/trac/roadmap