Cyberinfrastructure to Support Real-time, End-to-End Local Forecasting

1
Cyberinfrastructure to Support Real-time,
End-to-End Local Forecasting
Mohan Ramamurthy Tom Baltzer, Doug Lindholm, and
Ben Domenico Unidata/UCAR AGU Fall
Meeting December 16, 2004
2
Local NWP A Growing Activity

• Mesoscale forecast models are being run by
  universities, in real time, at dozens of sites
  around the country, often in collaboration with
  local NWS offices
• Tremendous value
• Leading to the notion of distributed NWP
• Yet only a few (OU and U of Wash) are actually
  assimilating local observations which is one
  of the fundamental reasons forsuch models!

• Applied Modeling Inc. (Vietnam) MM5
• Atmospheric and Environmental Research MM5
• Colorado State University RAMS
• Florida Division of Forestry MM5
• Geophysical Institute of Peru MM5
• Hong Kong University of Science and Technology
  MM5
• IMTA/SMN, Mexico MM5
• India's NCMRWF MM5
• Iowa State University MM5
• Jackson State University MM5
• Korea Meteorological Administration MM5
• Maui High Performance Computing Center MM5
• MESO, Inc. MM5
• Mexico / CCA-UNAM MM5
• NASA/MSFC Global Hydrology and Climate Center,
  Huntsville, AL MM5
• National Observatory of AthensMM5
• Naval Postgraduate School MM5
• Naval Research Laboratory COAMPS

3
Science Drivers for Local Modeling

• Many weather phenomena that affect society and
  commerce occur on the mesoscale. E.g.,
  squall-lines, snowbands, hurricanes downslope
  windstorms, lake-effect snowfall, etc.
• Need high-resolution local modelling to
  accurately resolve and predict these phenomena
• Utilize dense local observations (e.g.,
  Mesonets)
• Resolve local topography
• Collaboration with local NWS forecast offices

Show examples
4
Technology Trends Enabling A New Generation of
Local NWP Activities

• Commodity microprocessors inexpensive but
  powerful workstations/clusters
• High-bandwidth networks (e.g., Internet 2)
• Transparent data access and delivery
• Community Models (MM5, WRF)
• Local observatories (e.g., mesonets)
• Community codes for data assimilation (e.g.,
  3DVAR, ADAS)

5
Numerical Weather Prediction Key Steps

• Analysis/Assimilation
• Quality Control
• Retrieval of Unobserved
• Quantities
• Creation of Gridded Fields

• Observations Previous Model Forecast
• Mobile Mesonets
• Surface Observations
• Upper-Air Balloons
• Commercial Aircraft
• Geostationary and Polar Orbiting Satellite
• Radar Data
• Wind Profilers
• GPS/Met instruments

Prediction PCs to Teraflop Systems

• Product Generation,
• Visualization,
• Dissemination

• End Users
• NWS
• Private Companies
• Students

6
Unidata Technologies That can be Used in Local
Modeling

• Local Data Manager data transport
• Data streams IDD and CONDUIT Relaying and
  accessing data
• Decoders Data conversion
• NetCDF libraries and tools Data infrastructure
• OPeNDAP Remote data access (Collaborator)
• THREDDS Cataloging data
• GEMPAK and IDV - Visualization
• GIS Integration tools (in future)

7
Real-time Data Distribution
Model
Satellite
Radar
There are over 150 university sites in North and
South America, Europe, and Asia that receive
real-time data using the Unidata Local Data
Manager Plus there are over 300 LDM sites in
NWS, NOAA, NASA, KMA, Taiwan, and Spain that are
not part of the open IDD.
8
LDM in Action
SuomiNet
WSR 88-D Data
LDM is providing a variety of real-time
meteorological observations and model output from
operational prediction systems for local NWP
initialization
9
Todays Local NWP Process at Many Universities
User running local analysis and display tools
Meteorological Assimilation System
Decoders
Decoders
Decoders
Regional Model Hosted on local hardware
Assimilated Data For Initial Conditions
10
Todays Local NWP Process at Many Universities
There is no Data Sharing (other than with local
NWS offices)
User running local analysis and display tools
Meteorological Assimilation System
Decoders
Decoders
Decoders
Regional Model Hosted on local hardware
Assimilated Data For Initial Conditions
11
OPeNDAP Servers
Unidata Motherlode Server
Unidata LEAD Testbed
There are many OPeNDAP servers for operational
and historical data, but none outside of Unidata
LEAD for real-time local NWP output
12
Remote Data Access and Catalogs
Developed for real-time WRF predictions from
University of Illinois. Courtesy Brian Jewett
13
Integrated Data Viewer

• Unidatas newest scientific analysis and
  visualization tool
• Provides 2, 3 and 4-D displays of geoscientific
  data
• Stand-alone or networked application, providing
  client-server data access via multiple protocols
• Java-based tools Runs on Windows, Macs and Unix
  machines

14
Remote Visualization of Local NWP Output
Developed for real-time WRF predictions from
University of Illinois. Courtesy Brian Jewett
15
GEMPAK Example

• Some sites convert their forecast output into a
  format compatible with GEMPAK analysis and
  visualization tool
• Enables integration of local model output with
  other operational data sets

16
Thematic Real-time Environmental Distributed Data
Services (THREDDS)
To make it possible to publish, locate, analyze,
visualize, and integrate a variety of
environmental data
Combines push with several forms of pull and
digital library discovery
Connecting People with Documents and Data
17
LEAD A Large-ITR Effort

• Linked Environments for Atmospheric Discovery
• Identify, Access, Assimilate, Predict, Manage,
  Mine, and Visualize a broad array of
  meteorological data and model output, independent
  of format and physical location
• A range of Grid and Web Services will be
  developed for dynamic, on-demand, end-to-end
  weather prediction
• Institutions U. Oklahoma, Unidata, U. Alabama,
  U. Illinois, U. Indiana, Millersville U., Howard
  U. and Colorado State U.

18
Web Services

• They are self-contained, self-describing, modular
  applications that can be published, located, and
  invoked across the Web.
• Web Services are emerging as tools for creating
  next generation distributed systems that are
  expected to facilitate program-to-program
  interaction without the user-to-program
  interaction.
• Besides recognizing the heterogeneity as a
  fundamental ingredient, these web services,
  independent of platform and environment, can be
  packaged and published and they can communicate
  with other systems using the common protocols.

19
LEAD Vision
User Orchestrates Web Services to Create
Regional Forecast
User running local analysis and display tools