OPeNDAP Framework

1
OPeNDAP Framework

• OPeNDAP Open-source Project for a Network Data
  Access Protocol

• aka. DODS Distributed Oceanographic Data System

• DODS was originally developed to allow
  researchers transparent access to oceanographic
  data across the Internet, using their favourite
  application package

2
OPeNDAP Development

• OPeNDAP is designed to be as general as possible
  without being constrained to a particular
  discipline or world view.

• The system is a discipline neutral data access
  protocol it can be (and is being) used in
  astronomy, medicine, earth science,

3
OPeNDAP Architecture

• uses standard http protocol for transmission of
  data across the Internet

• designed to allow users to analyse data using the
  application package with which they are the most
  familiar

• supports selection of data using constraint
  expressions

• translates data from one format to another

4
OPeNDAP Servers

• Servers receive requests and provide responses
  via the Data Access Protocol (DAP).

• Servers convert the data from the form in which
  they are stored to the OPeNDAP data model.

5
OPeNDAP Clients

• Clients make requests and receive responses via
  the DAP.

• Clients convert data from the OPeNDAP data model
  to the form required in the client application.

6
OPeNDAP URL Structure

• Data are requested with a URL.
• http//dods.acecrc.org.au/cgi-bin/nph-dods/bathyme
  try/01210050.nc
• Protocol Machine name OPeNDAP server Directory
  File name

?depth01019660 Constraint

• The user can impose a constraint on the data that
  they would like to acquire from a data set by
  appending a constraint expression to the end of
  the URL.

7
OPeNDAP Data Model
For example A user might want all ocean
temperature values (with associated times and
locations) that lie between 90 and 110 m and have
uncertainties less than 1ºC.
8
Syntactic Semantic Metadata
9
Syntactic Metadata
10
Semantic Metadata
11
OPeNDAP Services

• OPeNDAP services are accessed by appending
  various extensions to the file name
• http//dods.acecrc.org.au/cgi-bin/nph-dods/bathyme
  try/01210050.nc
• Protocol Machine name OPeNDAP server Directory
  File name

.dds .das .dods .info .ascii .html .ver /
12
OPeNDAP Services - Basic

• OPeNDAP provides three basic data objects
• data descriptor structure (.dds) - syntactic
  metadata - rigid

• data attribute structure (.das) - semantic
  metadata - flexible

• data (.dods) - the actual data in a binary
  structure primarily for internal use

13
Data Descriptor Structure (DDS)

• The DDS may be requested from the server by
  appending .dds to the file name
• http//dods.acecrc.org.au/cgi-bin/nph-dods/bathyme
  try/01210050.nc
• Protocol Machine name OPeNDAP server Directory
  File name

.dds

• The DDS provides a description of the shape" of
  the data

14
DDS Example
/zobt_01.nc.dds
90-element vector called lon 90x2 element array
called lon_bnds containing 64-bit reals
60-element vector called lat 60x2 element array
called lat_bnds containing 64-bit reals
A "Grid" containing a 2-dimensional array of
32-bit reals called zobt, and two "Map" vectors
called lat, lon containing 64-bit reals
15
Grid Data Type

• The Grid is a special data type that includes a
  multidimensional array, and map vectors that
  indicate the independent variable values

The array part of the grid would contain the data
points measured at each one of the squares, the X
map vector would contain the positions of the
columns, and the Y map vector would contain the
positions of the rows.
16
Data Descriptor Structure (DDS)

• The DDS is the syntactic metadata for a data set.
  
• The DDS is rigidly constrained.
• This metadata is held in containers within the
  DDS that associate the metadata with the
  variables in the data set.
• The DDS is obtained by the server from the data
  set.

17
Data Attribute Structure (DAS)

• The DAS may be requested from the server by
  appending .das to the file name
• http//dods.acecrc.org.au/cgi-bin/nph-dods/bathyme
  try/01210050.nc
• Protocol Machine name OPeNDAP server Directory
  File name

.das

• The DAS contains information about the data, such
  as units and the names of variables

18
DAS Example
/zobt_01.nc.das
19
Data Attribute Structure (DAS)

• There is no constraint on the metadata that is
  placed in the DAS
• The DAS could be empty or could hold some
  standards compliant description of the data set.

20
Metadata Conventions
Some common metadata conventions

• COARDS - promotes the interchange and sharing of
  files created with the netCDF API

• CF - NetCDF Climate and Forecast (this
  generalises and extends COARDS)

• HDF-EOS Earth Observing System (NASA)

• FGDC Federal Geographic Data Committee

• PMEL-EPIC NetCDF

21
OPeNDAP Services - Other

• .ascii an ascii representation of the data.

• .info a more readable (or nicer) version of the
  DDS and DAS combined.

• .html form a web based form that will help to
  build an OPeNDAP URL.

• / a way of obtaining a listing of
  OPeNDAP-accessible files at a site no file name
  or constraint URL ends in a slash

• .ver returns OPeNDAP server version useful
  for seeing what features are available.

22
OPeNDAP .ascii Service
/zobt_01.nc.ascii?zobt713537144
23
OPeNDAP .info Service
/zobt_01.nc.info
24
OPeNDAP Browser Service
/zobt_01.nc.html
25
OPeNDAP Directory Service
/h88/ipcc_data/
Take note of this