THREDDS Data Server (TDS) Overview

1
THREDDS Data Server (TDS) Overview

• John Caron
• Unidata/UCAR
• Nov 2006

2
THREDDS Data Server (TDS)

• 100 Java
• 1.4 code base, will assume 1.5 after next release
  (3.14)
• Open source (but you have to ask)
• Runs inside of Tomcat Servlet Container
• single thredds.war

3
Data Services

• HTTP Server (file transfer)
• OPeNDAP (CDM files)
• WCS - Web Coverage Server (gridded)
• NetCDF subsetting service (gridded)
• needs some special configuration
• NcML virtual datasets (not HTTP)

4
CDM files

• General NetCDF, HDF5, OPeNDAP
• Gridded GRIB-1, GRIB-2
• Radar NEXRAD level II, NEXRAD level III, DORADE
• Point BUFR
• Satellite DMSP, GINI

5
NetCDF Markup Language (NcML)

• XML representation of netCDF metadata (like
  ncdump -h)
• Create new netCDF files (like ncgen)
• Modify existing datasets
• Add/delete/rename
• Create logical sections of existing variables.
• Create unions and aggregations of multiple
  existing datasets.

6
TDS and NcML

• Embed NcML into the TDS configuration catalog
• Server serves a virtual dataset defined by NcML
• NcML hidden from the client
• Can fix metadata problems
• Can augment metadata
• NcML aggregation replaces the old Aggregation
  Server
• Forecast Model Run Collection Aggregation

7
HTTP File Server

• Serve any file
• Bulk file transfer
• Partial file transfer (Accept-Ranges) allows
  remote netCDF subsetting access
• Efficiency depends on access pattern
• Recommend OPeNDAP for general access

8
THREDDS OPeNDAP service

• Serve any CDM file
• Uses the DODS-Java (DAP 2.0) library
• Non-streaming
• OPeNDAP latest server (Server4) now has THREDDS
  catalogs
• Catalogs instead of dods_dir
• Catalogs automatically generated
• Uses some THREDDS code (CrawlableDataset)

9
THREDDS WCS service

• Must have Grid coordinate system
• evenly spaced x,y
• WCS 1.0 specification
• Return formats
• GeoTIFF floating point
• NetCDF binary
• No reprojections, resamplings
• Still experimental, will move to WCS 1.1 when
  thats ready

10
NetCDF subsetting service

• Currently only GRIB model output files
• Allow to subset the file by
• Lat/lon bounding box
• Forecast time range
• list of Variables
• Output is a CF-1.0 netCDF file
• Variation of the WCS server
• Simplified request protocol

11
TDS Catalog Services

• THREDDS catalogs
• Client asks server what have you got?
• Catalog subsetting
• Catalog validation
• View catalog as HTML

12
Other services

• Latest Dataset
• finds most recent dataset from a collection
• CDM Coordinate System Validator
• Forecast Model Run Inventory
• Show available inventory from IDD NCEP model data
• Digital Library Record Exporting
• OAI protocol to GCMD, DLESE

13
TDS and Digital Libraries

• Translate catalog metadata records into Digital
  Library records
• Collection level records
• Inventory level records
• Send records to existing Digital Libraries
• Direct harvesting of THREDDS catalogs
• No cross-server search service in TDS

14
TDS clients

• Catalogs
• Browser
• Nj22 ToolsUI, IDV, MyWorld (Northwestern)
• OPeNDAP (lots, including IDL, MatLAB)
• WCS (still minimal, but expect to grow)
• GDAL, CADCorp, Liping Di (George Mason)
• NetCDF Subsetter
• Forms interface
• ESRI ArcGIS (manual download)
• Experimental scripts (CUAHSI)

15
Next TypedDataset Aggregation

• TypedDatasets know (some) data semantics
• Forecast Model Collection
• Combine multiple model forecasts into single
  dataset with two time dimensions
• With NOAA/IOOS (Steve Hankin)
• Point/Station/Trajectory/Profile Data
• Allow space/time queries, return nested sequences
• Start from / standardize Dapper conventions
• Collections of Radar data