Title: Thomas Karl Director
1 NOAA Reanalysis User Needs AMS Town Hall Meeting
- Thomas Karl Director
- National Oceanic and Atmospheric Administration
- National Climatic Data Center
-
- AMS 88th Annual Meeting
- Town Hall Meeting
- January 21,2008
2Outline
- Purpose of Meeting
- Overview of NOAA Reanalysis Efforts
- Climate Forecast System Reanalysis and Reforecast
(CFSRR) - Data Availability Plans
- Data Access Optimization
- User Input and Open Discussion
3Open Forum and User Input
Tonights discussion is a focus on user needs
- Has the proposed file organization and groupings
reflect current user needs and expectations? - Which data need to be placed on-line given e.g.,
150TB of disk? - Are we on the right track? Do we need to re-group
data sets to create needed products such as
initialization files, analysis, or by forecast
projections? - Can some other mechanism (i.e. LDM) provide some
CFSRR output to users in near real-time, shifting
the load from the archive? - NCDC and NCEP are committed to provide the
highest resolution if possible. Tonights
discussion is a focus on user needs tempered by
available resources.
4New NOAA Reanalysis Projects
- 1) Historical SFC Reanalysis (Compo et al.,)
- 1850 to present. 60TB
- 2) Post WW-II Reanalysis (NCEP)
- 1944 to present. 235TB
- 3) Climate Forecast System Reanalysis and
Reforecast (CFSRR) Project (Saha et al.) - 1978-2008 Reanalysis and Seasonal Reforecast
915TB (Grib1) - Development of a Reanalysis Clearinghouse
- Capability for an on-going analysis of the
climate system
5CFSRR Overview (Saha, EMC/NCEP)
- Coupled Climate Forecast System (CFS) Reanalysis
and Reforecast (CFSRR) (Atm - Ocn - Land
SeaIce) has two parts - Reanalysis 31-year (1979-2009) T384L64 (32 km
and 64 hybrid layers for the atmosphere, 0.50 and
40 levels for the ocean, 4 soil levels) - Reforecast 28-year (1982-2009) T126L64 (32 km
and 64 hybrid layers for the atmosphere, 0.50 and
40 levels for the ocean, 4 soil levels) - 6 hourly Reforecast for 1 year
- NCDC and NCEP are seeking community input for
archive and access priorities to this massive
dataset - Approximately 915 Terabytes in GRIB1.
- Note this presentation assumes conversion is
needed to Grib1 as many user applications cannot
yet deal with GRIB2 directly (including NCDC).
6Proposed CFS Reanalysis Data
- CFS High Resolution Initial Conditions 72 TB
- CFS Low Resolution Initial Conditions 35 TB
- Full Data Ingest 46TB (satellite radiances,
RAOB, in-situ, etc.) - Pgbh Pressure Grib Files 0.5 x 0.5 Hourly 91TB
- Flxf T382 Gaussian Hourly 25TB
- Ocnh 0.5 x 0.5 Hourly 18TB
- Diabf 1.0 x 1.0 Hourly 27TB
- Ipvh 0.5 x 0.5 Hourly 17TB
- Monthly 1TB
- TOTAL 332 TB
7Proposed CFS Reanalysis Hi-Resolution File Level
Organization
- Siganl 3-D Hybrid Analysis
- Sfcanl Surface Analysis
- Ocnanl 3D Ocean Analysis
- Pgbh 3-D Pressure level data
- Flxf SFC fluxes, radiative fluxes, precip.
- Ocnh 3-D Ocean data
- Ipvh 3-D Isentropic level data
- Diabf Diabatic Heating, Moistening rates, etc.
- Egyh Energetics, u v, TKE, etc.
T382L64 Ocean (1/4 x ½) 6 hourly
72TB T126L64 Ocean (1/2 x 1) 6 hourly
35 TB
0.5 x 0.5 degree Hourly 91TB
T382 Gaussian Hrly 25TB
0.5 x 0.5 degree Hourly 18TB
0.5 x 0.5 degree Hourly 17TB
1.0 x 1.0 6-Hourly 27TB
0.5 x 0.5 Monthly 1 TB
8CFS 28-Year Reforecast Data1982-2009
- 6-hourly Pressure Grib 412TB
- 6-hourly Ocean 102TB
- 6-hourly Ipv 78TB
- Monthly and Time-series 3TB
- 1.0 x 1.0 degree for first 6 months of forecast
- 2.5 x 2.5 for next 6 months of forecast
-
- TOTAL 595 TB
9National Academies Recommendation
- National Research Council, Board of Atmospheric
Sciences and Climate (BASC) - Completing the Forecast Characterizing and
Communicating Uncertainty for Better Decisions
Using Weather - and Climate Forecasts
- Recommendation 3.4
- The NOAA National Operational Model Archive and
- Distribution System (NOMADS) should be maintained
and - extended to include (a) long-term archives of
global and regional - ensemble forecasting systems and their native
resolution, and - (b) re-forecast datasets to facilitate
post-processing
10Open Discussion
Proposed Data and System Access Priorities
11CFSRR System Access Prioritization 1
- Basic ftp and http scripting services to the
highest resolution data though the NCDC tape
archive services. - NOMADS on-line sub-setting services for most
requested data (tonights discussion). - Advanced GIS and other Web Based Portal Services
as resources and time permit - Issue Data access may be throttled based on
number of concurrent users, I/O restraints, and
communications bandwidth at NCDC.
12Reanalysis Data Access Prioritization 1
- Highest Resolution CFSRR data provided
- Low Res users can subset or use Post WW-II
- Access assisted by user feedback as to file level
organization (this meeting!) - Historical 20th Century SFC Reanalysis
- 60TB (Grib and HDF)
- FY10 Post WW-II U/A Reanalysis
- Low resolution 6-hourly (235TB Grib-1)
- When funded
13FEEDBACK and QUESTIONS
OPEN DISCUSSION
David.Easterling_at_noaa.gov
- Suranjana.Saha_at_noaa.gov
- CFSRR PI
- Glenn.Rutledge_at_noaa.gov
- nomads.ncdc.noaa.gov
14Backup Slides
15Overview
- To overcome a deficiency in model data access,
some of the Nations top scientists are actively
engaged in a grass-roots framework to share data
and research findings over the Internet.
- NCDC, NCEP and GFDL initiated the NOAA
Operational Model Archive and Distribution
System. - NOMADS is a distributed data services pilot for
format independent access to climate and weather
models and data.
16The NOAA Operational Model Archive and
Distribution System
NOMADS Goals
- Establish a unified climate and weather model
archive providing format independent access to
retrospective models
- promote model evaluation and community feedback
- foster research within the geo-science
communities (ocean, weather, and climate) to
study multiple earth systems using collections of
distributed data
- develop institutional partnerships and access
via distributed open standard technologies.
17Multiple paths to format independent data access
The NOMADS System Design
18NCDC HDSS Access System (HAS NCDC Archive)
Other incoming data
Data request from access serviceeg, NOMADS
Data delivery to access serviceeg, NEXRAD
NOMADS Model data
NEXRAD Radar
Major Access Services
CLASS Satellite
CDO, GIS Services In-situ
NOAA Customers
A very simplified diagram of data flow from
NCDCs HDSS tape robotics system incoming
sources