NOAA Operational Model Archive and Distribution System - PowerPoint PPT Presentation

1 / 24
About This Presentation
Title:

NOAA Operational Model Archive and Distribution System

Description:

NOAA Operational Model Archive and Distribution System ... Use benchmarks to characterize system loads in order to determine costs and give ... – PowerPoint PPT presentation

Number of Views:58
Avg rating:3.0/5.0
Slides: 25
Provided by: pc696
Category:

less

Transcript and Presenter's Notes

Title: NOAA Operational Model Archive and Distribution System


1
NOAA Operational Model Archive and Distribution
System

Distribution of Real-Time and Retrospective NCEP
Model Data Sets RT-NOMADS
Jordan C. Alpert jordan.alpert_at_noaa.gov Jun Wang
jun.wang_at_naa.gov
Alpert 6/5/03
where the nations climate and weather services
begin
2
NOMADS is an agreement between agencies who
participate to have common
  • Data and observation distribution software,
    format independent and description methodology,
  • Documentation and organization framework,
  • Forum to plan and organize
  • Funding avenue to direct .
  • for university, federal agencies and
    organizations, a mechanism to obtain support for
    the dissemination of their data sets.

3

Participants U.S.- NOAA, DOE, NASA, NCAR,
Unidata, COLA, OPeNDAP Project, Several
Universities International- CEOS NERC ESG UK
BADC BOM / Dept. of State via CAP
4
NOMADS and the NCEP Real Time Component
A solution for Recommendation 5 of Fair Weather
Effective Partnership in Weather and Climate
Services
  • Wider information dissemination through web pages
    and other avenues.
  • Framework to create specialized and value added
    products needs access to weather data.
  • Facilitate communication among commercial,
    public, research community and federal agencies.
  • NOMADS is a system for participants to interact
    with others using common protocols and formats.
    Existing differences in details are mollified by
    software.

5
RT-NOMADS at NCEP
Consists of
  • Distributed Ocean Data Server - OPEN_DAP (DODS)
    based on metadata file descriptors and XML.
  • Web based ftp, the NCEP ftp2u client and Great
    Displays (pdisp), a grid analysis and display
    client.
  • Model initialization ingredients, The minimum
    set for model rerun/retrospective analysis ready
    observations restart files.
  • The Real Time NCEP Operational model run history.

6
RT-NOMADS is a service for science, development
and public access.
  • To distribute NCEPs Operational data set to
    researchers and the public.
  • To provide server software to NOAA projects who
    need to serve their data through web services.
  • NCEP NOMADS servers are the intermediary between
    NCEP operational computers and NCDC NOMADS
    archive servers.

7
Tools for Users on the server
  • Pare down large file sizes of high resolution
    initialization and model results.
  • (re-) Group different data sets to create needed
    products such as initialization files for model
    development.
  • Subset the data sets in parameter space
  • Subset the data sets in physical space
  • (subset the grid in 4 dimensions)

8
NCEP Real-Time Server
  • Real time.
  • Operational Suite of Model products.
  • Forecast information for model comparison.
  • Operational initialization and observations.
  • Maintenance of codes, utilities for data
    manipulation and grid transformation, sub-setting
  • Documentation.
  • Serves as transfer point to NCDC for NOMADS
    archive of reruns and retrospectives.

9
The Server Client distributed relationshipA
Common Sense Approach to Data Distribution
Examples of Clients
  • ftp2u, Great Displays, and GDS allow sub-setting
    of data by parameter, by 3D region, and by time.
  • Allows for results with low band width! Eg.,
    NASA/GSFC takes 1 hour to ftp entire GFS (AVN)
    forecasts or 13 seconds to download their
    desired fields (ftp2u).
  • GDS NOMADS participants serve up their data sets
    through a client-server relationship, that is,
    the data sets are internet ready and the display
    is done by the client.

10
Extending NOMADS to Real Time
  • The A in NOMADS is for Archive.
  • NCEP historical cases for rerun/retrospective
    ocean, atmosphere and climate.
  • NCEP Real Time model output in the NOMADS
    framework so there is a seamless access to
    these important data sets at NCEP and at NCDC!
  • RT-NOMADS provides the framework for long term
    support to be available to keep up with modeling
    advances.

11
RT-NOMADS is a prototype
Goals
  • Provide proof of the concept that distributed
    systems can serve data sets across agencies.
  • Specify an Operational system for NOAA/TOC,
  • Provide current state of the art capability and
    move more complete data sets,
  • Use benchmarks to characterize system loads in
    order to determine costs and give NOAA
    administration the information it needs to triage
    data resources.

12
Data Application Protocol (DAP) using
13
GrADS client . commands
Sdfopen http//nomad29090/dods/reanalysis-2/pgb/h
gt t 1 'set gxout shaded' while (t lt 1068)
'set t 't 'q time' datesubwrd(result,3)
'draw title hgtprs 'date' 't 'd hgtprs'
'/home/wd23ja/grads/lib/cbarn.gs' 'swap' t
t 1 endwhile
14
Getting data without GrADS or a commercial client
  • http//140.90.198.1569090/dods/eta/eta20030527/et
    a_00z.ascii?tmpprs000030130 290290
  • the order of the square bracked values is
  • timelevellatlon
  • where lat is measured from the south pole (0)
  • to equator (90) to NP (180)
  • (Use a wget and the URL with the ascii response
    in cron or a cgi-bin script provides the needed
    number(s) when and where needed.)

15
NOMADS clients at NCEP
Great Displays (pdisp) and ftp2u
by Ebisuzaki and modified by
Wang
  • Real Time Operational model data sets from GFS,
    and ETA from present to one month ago.
  • Attention to accumulation variable differences
    and high and low resolution.
  • Analysis pgb, restart files.
  • Recent (real time) climate runs, ie., RSM.
  • Observations (soon).
  • Reanalysis-2 (and 1)
  • AMIP type runs (Surus finest runs eg., 28 and 64
    levels 1978-2002), sudden warming, .

16
MATLAB A Commerical Client
  • servernomad29090
  • directory/dods/gdas/rotating/ fextascii
  • modelgdas datestrYYYYMMDDHH
    varnamehgt
  • varlevs1
  • Arrc,t1,,t2,,nlev1,,nlev2,,ilat
    1,,ilat2,,ilon1,,ilon2,
  • urlstringhttp//,server,directory,model,datestr
    ,.,fext,?,varname,arrc
  • url java.net.URL(urlstring) connect to
    web page
  • Try
  • stream openStream(url)
  • ireader java.io.InputStreamReader(stream)
  • breader java.io.BufferedReader(ireader)
  • Line readLine(breader) disp(line)

17
Data Sets
Archives at NCDC
  • NOAAPORT Operations.
  • Continuation of NCEP real time model component.
  • NCEP model initialization ingredients, The
    minimum set for model rerun/retrospective
    analysis ready observations restart files.

18
Model Data Requirements
from CLASS
Current Archive NOAAPort 2TB/yr GDAS/SSI 400Gb/
yr NARR 10Tb Total 12.5Tb FY04 NCEP
Hi-Res 10TB/yr Total 22.5Tb
Legend Green- Existing Archive Blue- Expected
FY03/04
FY03
5 year retention only
Rutledge and Alpert 6/03
19
Model Data Reqs - NWP and GCMs
GCM- Global Climate Model Reanal- N. Am. Regional
Reanalysis NWP- Numerical Weather Prediction
5-year rotating NWP archive
Alpert Rutledge 6/03
20
RT NOMADS Server at NCEP
  • Two 1TB RAIDZONE servers
  • http//nomad2.ncep.noaa.gov
  • http//nomad1.ncep.noaa.gov (backup)
  • Documentation web page under construction
  • New disk space on order.

21
Non-Operational Projects can become NOMADS
participants to display their data sets
NOMADS gives guidance to
  • Install scripts for a GDS framework,
  • Setup GDS XML file see example on web page, for
    example,
  • Including GDS administration updates new
    additions transparently,
  • newxml update script to automatically add new
    directories and files.

22
(RT) NOAMDS data sets .
  • One data set (e.g, GRIB) provides the data for
    all access applications.
  • Control files (grib2ctl) and index files
    (gribmap) create unique metadata description.
  • GDS can read HDF, NETCDF as well as other formats.

23
NOMADS and the NCEP Real Time Component
A solution for Recommendation 5 of Fair Weather
Effective Partnership in Weather and Climate
Services
  • NWS should make its data and products available
    in Internet-accessible digital form.
  • Information held in digital databases should be
    based on widely recognized standards, formats,
    and metadata descriptions to ensure that data
    from different observing platforms, databases and
    models can be integrated and used by all
    interested parties in the weather and climate
    enterprise.

24
(No Transcript)
Write a Comment
User Comments (0)
About PowerShow.com