Recent developments and trends in Network Access to Oceanographic In-situ Data

1
Recent developments and trends in Network
Access to Oceanographic In-situ Data

• Nancy Soreide, NOAA/PMELJohn Oz Osborne,
  NOAA/PMEL - OceanAtlas Software Joe Sirott,
  NOAA/PMEL Sirott and AssociatesTiffany Vance
  NOAA/AFSCNazila Merati NOAA/PMEL
  JISAOChristopher Moore NOAA/PMEL - JISAO

American Meteorological Society MeetingJanuary
29-Feb 3, 2006, Atlanta, GA
2
What is OPeNDAP?

• OPeNDAP (Open source Project for a Network Data
  Access Protocol) is a protocol for remotely
  accessing scientific data
• Uses HTTP as transport
• Endorsed as a standard by IOOS Data Management
  and Communications (DMAC) committee

Joe Sirott
3
But

• OPeNDAP in-situ data is poor stepchild of OPeNDAP
  gridded data
• Few desktop clients support it
• Even fewer Web apps support it
• Some recent progress (GrADS, Java OceanAtlas) but
  change is slow

Joe Sirott
4
What is Dapper?

• Web server that provides distributed access to
  in-situ data via OPeNDAP protocol
• Clients include GrADS, Matlab, LAS, NcBrowse,
  Java Ocean Atlas, DChart
• Available data
• Realtime global Argo profiles
• NDBC Time Series
• WOCE Sea Level
• PMEL in-situ data (EPIC database)
• Coming TAO NODC World Ocean Database 2001
• All components are open-source

Joe Sirott
http//www.epic.noaa.gov/epic/software/dapper/
5
What does Dapper do?

• Efficiently aggregates thousands of in-situ data
  files (gt2000000 for WODB2001)
• Handles several in-situ data formats
• Argo GDAC netCDF
• PMEL EPIC netCDF
• COARDS compliant netCDF
• Supports time series or profiles
• Supports space/time queries
• Based on HPCC funded Climate Data Portal
  (Soreide, Zhu, Denbo)

Joe Sirott
http//www.epic.noaa.gov/epic/software/dapper/
6
Dapper architectural overview

• Uses relational database to store metadata and
  coordinate boundaries for each profile/time
  series file
• Data maintained in original format
• All data delivered as OPeNDAP sequences

Joe Sirott
http//www.epic.noaa.gov/epic/software/dapper/
7
Dapper architecture diagram
OPeNDAP Protocol (HTTP)
Dapper HTTP Server
OPeNDAP clients
CDP service
netCDF service

CORBA (IIOP)

Climate Data Portal (CDP)


Java netCDF library
MySQL





netCDF profile
netCDF profile
Database loader

Joe Sirott
http//www.epic.noaa.gov/epic/software/dapper/
8
DChart

• Browser based in-situ OPeNDAP client
• Based on FY2004 HPCC funded project for shipboard
  display of cruise data
• AJAX based user interface
• AJAX is used by Google Maps

Joe Sirott
9
DChart Design Goals

• Highly responsive browser based user interface
• Low learning curve
• Rapid previewing and downloading of Dapper
  datasets
• Highly interactive map for region selection
• No Java applets or proprietary plug-ins

Joe Sirott
10
Ocean Data Navigator(aka DCHART)
New!
Joe Sirott
http//dapper.pmel.noaa.gov/dchart
11
Ocean Data Navigator(aka DChart)
New!
Joe Sirott
http//dapper.pmel.noaa.gov/dchart
12
AJAX

• The Good, the Badand the Ugly

Joe Sirott
13
Good stuff

• Much more responsive user interface
• Desktop application functionality in familiar Web
  browser interface
• Client responsible for state management
• Zero effort software updates (for users!)
• Cool factor

Joe Sirott
14
Bad stuff

• Requires modern browser
• DChart support currently limited to IE 6, Firefox
  1.0.x, Mozilla
• Covers gt 90 of users and alternative would be
  downloading desktop app
• Debugging is difficult
• A lot for developers to learn
• XML,XPath,XSLT,JavaScript, CSS, DHTML, DOM
• Libraries/frameworks immature
• Maintenance and testing difficult

Joe Sirott
15
And the ugly

• Browser incompatibilities still an issue
• GET limited to 2048 characters in IE
• IE window coordinate system different than
  Firefox
• IE memory leaks
• Legacy ActiveX reference counting
• Transparent images with alpha channel
• Disappearing DOM nodes with XSLT in IE
• Ugly kludges required
• GET/POST through Iframe to retain history

Joe Sirott
16
Lessons Learned

• Desktop applications are moving to the browser.
• Its easier to move data to servers than servers
  to data.
• The response time of the back end is critical in
  AJAX applications.
• It takes a lot more time to develop the back end
  than the user interface.

Joe Sirott
17
Java OceanAtlas

• Comprehensive Profile Data Analysis for the
  Desktop

• Data types
• Supports many profile data formats (bottle and
  CTD data)
• Argo
• Distributed data (OPeNDAP/dapper)
• Gridded netCDF data
• Time series
• Graphics
• Property-property, profile, contour, station
  value, and map plots
• Overlay section contour plots
• Contour plots on maps
• Color plots by any parameter or metadata
  (date/time, longitude, or latitude)
• Color map symbols by station calculation values
• Presentation quality graphics and choice of fonts
• Calculations
• Large selection of standard oceanographic
  calculations
• Station calculations mixed-later depth,
  interpolation to surface, neutral surface,
  extrema, statistics
• Section calculations section difference and mean
  cast

John Osborne
18
Java OceanAtlas
Surface O2
10N Residuals O2 WOCE vs. World Ocean Atlas
John Osborne
19
OceanGISIntegration of GIS and Java tools

• GIS tools
• ArcObjects, ArcGIS, ArcIMS, ArcGIS Engine
• Java tools
• MapObjects Java, Intra-Layer Calculator, VTK
  (Java wrapper), Java 3D, Oceanographic tools
  (Java), GeoTools

• Read simple shape files
• Read ocean data from dapper
• Read model output
• Rotate/pan/zoom
• Hyperlink objects and 3D picking
• Switch from 2D projections to 3D

Tiffany Vance, Christopher Moore, Nazila Merati
20
OceanGIS Dapper Data Access
Accessing ocean profile sensor data from database
(selected profiles in blue)
Tiffany Vance, Christopher Moore, Nazila Merati
21
CTD cast location (light blue) Mixed-layer
depth (dark blue)
Tiffany Vance, Christopher Moore, Nazila Merati
22
Resulting ocean profile sections analyzed with
GIS statistical analysis tools
Tiffany Vance, Christopher Moore, Nazila Merati
23
Summary

• Dapper OPeNDAP server for ocean observations
  data
• DChart AJAX Web client for dapper
• Java OceanAtlas Desktop java client for dapper
• OceanGIS uses dapper for access to ocean
  profiles
• Other dapper clients ncBrowse, GrADS, LAS

More informationNancy.N.Soreide_at_noaa.gov