Radio Occultation Data Analysis at the COSMIC Data Analysis

1
Radio Occultation Data Analysis at the COSMIC
Data Analysis Archival Center (CDAAC)
www.cosmic.ucar.edu
D. Hunt, C. Rocken, Y-H Kuo, B. Schreiner, S.
Sokolovskiy COSMIC Program Office, University
Corporation for Atmospheric Research
2
Overview

• COSMIC Description
• CDAAC data processing and data formats
• Successes
• Challenges
• CDAAC 'Live' demo

3
COSMIC at a Glance
4
COSMIC Launch
5
COSMIC Occultation Distribution
COSMIC EQUARS Radiosondes
Earth-fixed Frame
Sun-fixed Frame
6
COSMIC System
S band
S band
L band
S band
T1
T1
Payload Commands and All Real-Time Data Products
LAN
S/C Telemetry
T1
Internet
7
COSMIC Spacecraft

• Orbital Sciences ORBCOM s/c bus
• Mass 70 kg, Size 1m diameter
• Propulsion orbit insertion, maintenance,
  de-orbiting
• Attitude
• Control 2deg pitch, 5deg roll/yaw (2-sigma)
• Knowledge 1deg pitch, 2deg roll/yaw
• Nadir orientation, 3-axis stabilized
• 40 MB on-orbit storage

8
COSMIC Payloads

• Each COSMIC satellite will carry 3 complementary
  payloads
• GPS - for atmospheric/ionospheric profiling and
  scintillation
• JPL Heritage design
• Tiny Ionospheric Photometer (TIP) for F2 layer
  electron density mapping
• NRL Heritage design
• Tri-band beacon (CERTO/TBB) for high-resolution
  ionospheric imaging and scintillation studies
• NRL Heritage design

9
COSMIC GPS Receiver ARGO
JPL Design, ARGO is based on CHAMP BlackJack
Receiver Technology transfer JPL -gt Broad Reach
Engineering 4 antennas 2 occultation 2 POD
antennas Receiver Data Recorder/PC 3.5
kg Power 16W GPS 10 W Data Recorder/PC New
open loop tracking and software for rising
occultations under development at JPL
10
COSMIC Data Analysis and Archival Center (CDAAC)
11
CDAAC Responsibilities

• Process all COSMIC observations
• LEO/GPS orbit determination
• Atmospheric Ionospheric profiles
• Rapid analysis for operational demonstration
• Post-processed analysis for climate and other
  research
• Provide data to universities and research
  laboratories
• Provide data feeds (lt 3hr) to operational centers
• Archive data provide web interface

12
CDAAC Processing Flow
Atmospheric processing
Canonic Transform Abel Inversion
1-D Var Moisture Correction
LEO data
Level 0--level 1
Excess Phase
Orbits and clocks
Fiducial data
Real time Task Scheduling Software
Profiles
Ionospheric processing
Combination with other data
Excess Phase
Abel Inversion
Current processing time for 35 occultations 100
minutes of fid data 9min
13
GPS Data Processing
- GPS Orbits/ERP (Final/IGU) - Station
Coordinates - 30-sec Ground GPS Obs.
- 1-Hz Ground GPS Obs. - 50-Hz LEO Occultation
GPS Obs.
14
LEO POD Quality
CHAMP-B (EIGEN-S1) minus CHAMP-A
(EGM96) Temperature profiles
15
Calibration of excess phase delay

• Single Difference
• LEO clock errors removed
• use solved-for GPS clocks
• Main advantage Minimizes double difference errors

16
Inversion Details
17
CDAAC Hardware

• Currently consists of one Linux Beowulf style
  cluster
• Myrinet high speed interconnect
• 1 Terrabyte RAID, NFS mounted from master node
• Dual 2.0 GHz Pentium 4 Xeon cluster nodes
• One node dedicated to post-processing
• One or more nodes dedicated to real time
  processing
• Archival using NCAR mass store
• 1.0 petabyte tape farm

18
CDAAC Software

• Software written in perl, C or FORTRAN
• Currently consists of around 100 separate
  packages
• CVS modules for source developed at UCAR
• open source packages
• apache
• postgres
• perl
• perl modules
• Commercial packages
• Bernese
• F90 compiler
• Includes install system for easy compilation and
  testing on a basic Linux system

19
CDAAC post processing

• Done for climate and archival purposes
• Run on one node from one top level script
• 30-60 day latency for current CHAMP processing
• Run by hand by an operator
• Uses
• better orbits,
• more GPS ground station data
• Gridded analysis instead of prediction
• Done in day-long batches, instead of by LEO
  download

20
CDAAC real-time processing

• Run automatically as data are received on one or
  more cluster nodes
• Currently processing CHAMP data with 12-24 hour
  latency
• Results are currently generated on average 45
  minutes after receipt of CHAMP level 0 data
• We expect to tune this to lt 20 minutes for COSMIC
• Uses predicted GPS orbits and forecast weather
  grids
• Designed to allow processing of multiple missions
  (CHAMP, SAC-C, COSMIC, etc) on the same cluster
  of machines at the same time
• Processing controlled by three perl daemons
• Queue daemon (one per cluster)
• Master daemon (one per mission)
• Slave daemon (one per task)

21
Realtime processing (cont.)
Master daemon
FTP processes
Users
File Queue
Job Queue
FTP, LDM, other?
Slave daemons
Files on disk
Files on disk
CDAAC web
Processing jobs
Database
22
Question How to deliver data

• How to deliver occulation files?
• BUFR format occultation files to NESDIS
• FTP?
• HTTP?
• Unidata LDM?
• Other?
• Full resolution data from CDAAC
• FTP?
• LDM?
• Web?

23
File Formats

• Standard level 1a formats defined for all
  missions
• Standards used include
• Bernese ASCII formats
• UCAR BINEX for GPS data
• SP3 orbits
• High level (level 1b and 2) files are standard
  format NetCDF
• Uniformly accessible by libraries of database and
  statistics software
• Web tools
• Details of data formats on www.cosmic.ucar.edu
  (click on 'analysis data', then on 'file
  formats').

24
BUFR format

• Compact binary
• Multiple profile variables
• Time vs. Excess phase, satellite position
• Height vs. Bending angle, impact parameter
• NOT time vs bending angle.
• Height vs. refractivity
• Height vs. pressure/temperature/moisture
• Low resolution (200 levels for each profile)
• Is this enough for assimilation?

25
BUFR format issues

• Working with Dave Offiler on these issues
• CDAAC can only deliver bending angle, impact
  parameter and perigee lat/lon vs. height, not vs.
  time
• limitation of radio holographic methods
• Quality flags may need to be worked out.
• Occultation ID needs more flexibility than a
  single number
• Need to add a LEO ID for multiple LEO missions

26
Successes
27
Validation Statistics with NCEP
Global
CHAMP
SAC-C
(Comparisons to NCEP-NCAR Global Re-analyses (6
hr, 28 sigma levels, 1.87deg longitude, 94
gaussian latitudes, ds090.0)
28
ROSE Refractivity Comparisons
GFZ
UCAR

• Common set of over 1500 CHAMP occultations
  compared
• Three centers participated
• UCAR
• JPL
• GFZ

JPL
29
Tailored experiments with predicted occultations

• Drop sonde launched on day 2003.143 near Taiwan
• Dropped to match a predicted CHAMP occultation
• Good agreement for both moisture and temperature

30
COSMIC Challenges
31
Inversion Q/C CHAMP 2002.213-243
Reasons for Inversion Failure -- Aug 2002

• Of 8071 total occultations tracked and with
  fiducial and reference data
• 1303 Failed calibration (no excess phase file
  generated)
• Inversion Failures
• 989 Occultation starts too low
• 415 Failed difmaxref (climatology refractivity
  check)
• 159 Failed s4 (scintillation on
  occulting link)
• 52 Failed difmaxion (max L1, L2 bending angle
  check)
• 44 Occultation ends too high
• 42 Problem in Canonic Transform method
• 26 Z, N, P or T fails sanity check
• 10 Failed smean (bias WRT climatology
  bending
  angles)

32
Clock distribution problems
L1 L2
Effect of L4 (L1-L2) smoothing
33
L2 signal noise, SAC-C
34
Extrapolation of Ionospheric Correction to
process occultations with noisy L2
Dec 2001, SAC-C
Without correction
With correction
35
Super-refraction, Ducting
Near St. Helena Is.
36
Summary

• COSMIC program on track for 2005 launch
  satellites are built at Orbital, GPS at Broad
  Reach, TIB Beacon at NRL
• CHAMP and SAC-C processing up-to-date
• Near-real time processing, 1 day latency for
  CHAMP data
• 70-80 of calibrated occultations are inverted
  successfully (2002.213-243)
• Good agreement between CHAMP and SAC-C
• Good agreement with NCEP/ECMWF
• Comparisons between GFZ/JPL/UCAR successful
• Ongoing work on data delivery to weather centers
  (formats, protocols)
• Web page www.comic.ucar.edu