A Testbed for Marine XML

1
A Testbed for Marine XML

• Gregory Johnson and Nazila Merati
• NOAA/Pacific Marine Environmental Laboratory
• Seattle, WA
• Charles Sun NOAA/NODC
• Silver Spring, MD

2
Ocean observation systems (OOS)

• PMEL and its partners carry out several large
  scale Ocean Observing Systems
• TAO, FOCI, Tsunami, Argo floats
• with hundreds of instruments
• and thousands of observations

3
Data transformations and data sharing

• Scientists spend a great majority of their time
• formatting data
• transferring data
• developing transfer protocols
• before starting data analysis and sharing

4
History of the project

• In 2003
• We heard at the ESRI User Conference that XML
  would play an important role in the new ArcGIS
  architecture.
• NOAA IT and Web developers began to discuss the
  need for standards when using XML.
• Others are looking at and defining XML standards
  for observational Marine data
• XBTs and meteorological observations
• Argo Profiling Floats as a testbed
• Identified a PMEL and NODC project that we could
  use to test XML

5
Global Argo Profiling Float Array

• A broad-scale global Ocean Observing System
• Measures ocean temperature and salinity

6
How Argo Profiling Floats work
Deployed from ship/aircraft
Drifts, transmitting data to satellite
Sinks again
Sinks
Rises, measuring pressure, temperature, salinity
(a profile)
Drifts
Drifts
7
Why use Argo Floats?

• Argo Profiling Floats
• Integral to the ocean observing system
  architecture (GEOSS and IOOS)
• Good example of a profiling data type
• Limited number of attributes associated with each
  profile
• Similar data structure to other in-situ profile
  instruments
• NODCs data manager is interested
• XML as a potential method of data sharing between
  Argo Data Centers
• Integration of Argo Profile data into GIS
• Introduce this data to a new group of GIS-savvy
  users

8
Project objectives

• Work with NODC
• to identify key data sets and define which file
  formats to work with
• Define optimal parameters for a MarineXML
  dictionary
• by using data specifications from example
  libraries /Global Master Change Directory
  glossary
• Build schema that works the best with Floats
  simple and extensible
• Assess bandwidth versus file size issues
• Test the schema
• Test feasibility of web transfers
• Load data into ArcGIS 9.x and personal
  geodatabase
• Test the metadata creator in ArcGIS
• for compatibility with MarineXML schema

9
Why use XML?

• Platform independent
• Eases data exchange and communication between
  organizations
• For operational use
• Not only for science
• Extensible
• Preserves data, metadata, quality flags and
  edited data, when they are available in the data
• Forms basis of a data management framework

10
Different types of XML explored

• Marine XML
• different flavors strong community component,
  been in use since 2001, operational at AODC and
  now large initiative for the EU
• ESML Earth Sciences Markup Language
• works with netCDF, HDF, good community backing
• Borehole XML
• good as it has a z-value, may be better as a
  descriptor of actual location rather than data
  boreholes are also more shallow that the float
  data, not quite sure what borehole data are
  stored as, mostly text files used in the
  geologic sciences
• NcML netCDF XML
• good open source standard, data already are
  stored as netCDF, its the marine standard for
  data storage, most translators are for gridded
  data
• CSml Climate Science ML
• - builds on GML specifications, newly released,
  very robust, will work with point data in netCDF
  (within reason)
• Sensor ML
• still written within the OGC specs, again more
  for descriptor, but not necessarily for data
  transfer strongly pushed as a solution for data
  dissemination for GEOSS OGC compliant

11
How does this fit in with GIS?

• Currently, PMEL has methods to put data into GIS
  
• Theyre great
• But still requires data formatting and use of
  several different packages
• The output is a shapefile to be used by ESRI
  products

12
ESRI and the geodatabase

• Changes in GIS data management
• We are moving from shapefiles and coverages to
  geodatabases for data management and data
  storage.
• Geodatabases can be useful
• In building data models for a specific industry.

13
Geodatabases and data transformations

• XML encapsulated data
• can be transferred into the geodatabase with
  ArcGIS 9
• Geodatabase XML format
• allows import and export of items and data
  to/from the geodatabase.
• transfers include domains, rules and topologies
  and adding behaviors

14
System design
15
Potential problems

• netCDF to ncML or to MarineXML
• no need to do a double jump
• Bandwidth and compression
• XML may supersize data, didnt find this
• What if the geodatabase goes away?
• Is MarineXML better than ESML/CSML?
• Can it be applied to legacy data?

16
Next steps

• Build GUI interface to create XML file, add
  metadata and attribute data to observational
  float data files to provide more comprehensive
  data to researchers
• Write documentation
• New XSLT tools from ESRI
• Test new tools
• Test transfer times

Questions?