GODAE Quality Control Pilot Project

1
GODAE Quality Control Pilot Project

• Status Report - June, 2008
• Final IGST Meeting
• Washington, D.C.
• James Cummings
• Naval Research Laboratory
• Outline
• Data Sources
• Output File Structure and Reference Data
• Plans and Issues

2
GODAE QC Pilot Project Data Providers

• UKMO
• January 2003 to present, real-time push to
  USGODAE server
• BMRC
• 2005 and 2006 only, one time only push to USGODAE
  server
• MEDS
• June 2004 to present, real-time push to USGODAE
  server
• Coriolis
• January 2004 to present, pull to USGODAE server
• /ifremer/coriolis/mercator at
  ftp.ifremer.fr
• CO_BA, CO_CT, CO_MO, CO_PF, CO_TE, CO_XB
• FNMOC
• March 2003 to present, real-time push to USGODAE
  server

3
GODAE QC Pilot Project Call Sign Databases

• QC results from centers separated by call sign
• profile match ups based on call sign, location
  and observation time
• database contains observed values and
  supplemental information to understand QC
  decisions
• provides history of real-time QC decisions made
  on a platform over the lifetime of the platform
  
• call sign database format is NetCDF
• based on Coriolis multiple profile NetCDF file
  structure
• easily expanded with new QC variables or new data
  providers
• sorted by observation time and then data center
  for ease of use
• starting point for follow-on analysis and
  intercomparison of QC outcomes

4
FNMOC
MEDS
Coriolis
UKMO
BMRC
GODAE QC Application Program Interface (API)

read various QC data formats, check for valid
call sign, match profiles (lat, lon, time, call
sign), populate database structure with
observation, QC codes, QC flags, and supplemental
information, resolve QC conflicts, sort by
observation time, update NetCDF call sign database
1900305.nc Argo Float
A8ES9.nc Ship XBT
52007.nc TAO Mooring
Call Sign NetCDF Databases
5
GODAE QC Pilot Project Database Variables
dac source of data (FNMOC, MEDS, Coriolis, UKMO, BMRC)
dtg observation time
dtg_qc quality observation time
lat, lon observation position
pos_qc quality observation position
n_lvl number levels
lvl observation depth levels
sal observed salinity
sal_clm salinity climatology
sal_fcst salinity forecast
sal_flg salinity QC test failures
sal_qc salinity profile quality
sal_qc_lvl salinity level quality
tmp observed temperature
tmp_clm temperature climatology
tmp_fcst temperature forecast
tmp_flg temperature QC test failures
tmp_qc temperature profile quality
tmp_qc_lvl temperature level quality
sign call sign
sign observation call sign
upd_dtg profile update time at center
qc profile QC outcomes differ
qc_lvl level QC outcomes differ
ndx index to other DAC profiles
6
GODAE QC Pilot Project Reference Data

• Analysis/Forecast backgrounds
• UKMO FOAM forecast profile
• FNMOC NCODA analysis profile (persistence
  forecast)
• Climatology
• FNMOC Navy GDEM temperature and salinity
  climatology
• BMRC WOA or CARS temperature and salinity
  climatology
• QC Conflict Variables entire profile and
  level-by-level
• simple scheme to indicate differences of QC
  outcomes among centers
• 0 profile (level) accepted (or rejected) by all
  centers
• 1 profile (level) temperature QC outcomes
  differ
• 2 profile (level) salinity QC outcomes differ
• 3 profile (level) temperature and salinity QC
  outcomes differ

7
GODAE QC Pilot Project Plans and Issues

• Plans initialize call sign databases with
  2003-2008 production run
• missing complete set of BMRC QC data files
  Gary?
• host call sign data files on USGODAE server
• install GODAE_QC_API on USGODAE server
• update call sign data base files in near
  real-time (daily, weekly?)
• Issues (1) US GODAE server hardware upgrade
• GODAE_QC_API cannot run on existing hardware
• still waiting on server hardware upgrade funded
  by ONR
• Issues (2) Data problems
• BMRC and Coriolis data are NetCDF problems with
  corrupted data (bad values) have to work through
  data problems one file at a time
• data corruption problems slowing down 2003-2008
  production run
• FNMOC, UKMO, MEDS files are Fortran readable no
  data corruption problems with these files

8

• END