Retrieval, Analysis and Visualization of Multiple Heterogeneous Radar Data Using Intelligent Agents

1
Retrieval, Analysis and Visualization of
Multiple Heterogeneous Radar Data Using
Intelligent Agents

• Kurt D. Hondl
• DOC/NOAA/OAR
• National Severe Storms Laboratory
• HPCC 2005 - COL/CE/06

2
Objective

• Develop data ingest, algorithm processing and
  visualization strategies to handle disparate
  radar data
• Radars include
• Weather Surveillance Radar 88 Doppler (WSR-88D)
• FAA Terminal Doppler Weather Radar (TDWR)
• Other FAA radars may also provide data
• Dual Polarized WSR-88D research radar (KOUN)
• National Weather Radar Testbed - Phased Array
  Radar (PAR)
• Collaborative Adaptive Sensing of the Atmosphere
  (CASA)
• NetRad
• Shared Mobile Atmospheric Research Teaching
  Radars (SMART-R)
• Canadian Radars

3
Issues with Heterogeneous Radar Networks

• Differences include
• Wavelength
• Differences in attenuation
• Scanning Strategy
• Update rates for various radar systems VCPs
• Beamwidth
• Data Quality
• Calibration and sensitivity
• Storm Motion and Evolution
• Different volume update periods for different
  systems

4
Intelligent Agents

• What are Intelligent Agents
• Intelligent Agents are software elements that
  work without the assistance of users by making
  choices. Choices are based on rules that software
  developers have identified and built into the
  software.
• In our context, we treat each range gate
  observation from each radar as a separate object
  or agent. Each agent moves with the storm motion
  and collaborates with other agents when it is
  time to merge into the 3D grid. The agents
  then expire when their information has been
  superseded by new data.

5
Status

• Demonstrated use of Intelligent Agents
• Combining of WSR-88D network data in 3D
• Handles time-to-space correction
• Handles beam blockage
• Handles missing (thresholded) data vs unobserved
  data
• Multiple methods of combining WSR-88D data
• Testing on a national scale
• Ingesting the various radar data formats
• Developing ingest methods to handle
  non-sequential data
• NWRT Phased Array Radar
• Reading data radial-by-radial

6
Remaining Work

• Develop intelligent agents for non- WSR-88D
  radars
• Perform quality control
• Handle differences and discrepancies
• Implement agents into WDSS-II 3D reflectivity
  merger
• Test and evaluate heterogeneous network compared
  to WSR-88D network

7
(No Transcript)
8
Easy-to-Use Interface for Radar Data Quality
Control and Error Estimation

• Qin Xu
• DOC/NOAA/OAR
• National Severe Storms Laboratory
• HPCC 2005 - COL/CE/07

9
Easy-to-Use Interface for Radar DataQuality
Control and Error Estimation

• Objective Develop an easy-to-use interface
  created by using GTK (GIMP Toolkit) and OpenGL
  (Open Graphics Language). Unique features of
  this interface Build upon the infrastructures
  of CRAFT and ESDIM (Environmental Services Data
  and Information Management Program) and the
  Management System for Realtime Radar Wind
  Retrievals (product of FY03 HPCC). Leverage
  real-time radar data processing capabilities at
  NSSL and NCEP to track data quality problems and
  to estimate wind observation errors compared with
  model predictions.

10
Status

• Received award notification on March 21,
  2005.
• Started to work on the overall design in
  April, and so far, completed the design and
  implementation of real-time data link to
  operational RUC model outputs via NSSL WDSS-II.
  Some new data thinning strategies have been
  developed and tested to meet the required
  real-time computational efficiency. Background
  data are sampled from operational RUC model
  outputs and will used for the covariance
  estimation.

11
(No Transcript)
12
Real-time Distribution of WDSS-II Algorithm
Information
COL/CE/08

• Dr. Russell Schneider
• DOC/NWS/Storm Prediction Center
• HPCC 2005

13
Objectives

• Increase the size of the WDSS-II domain made
  possible using HPCC-2004 project
• Create a custom collection algorithm to combine
  information from multiple algorithms
• Create database of algorithm detections to aid
  real-time forecast verification based on
  algorithm reports.

14
(1) Increasing size of domain

• SPC Received Pioneer grant from Dept. of Commerce
  to purchase 7 additional servers.
• National Severe Storms Laboratory (NSSL) donated
  a few more machines and storage space.
• Benchmarked requirements for required domain size
  and created requisition order.
• Working on procurement of servers.

15
(2) Custom collection algorithm

• Created custom collection algorithm.
• User creates a XML file that specifies
• Which algorithms to combine
• How to combine them (max? min? fuzzy average of
  several fields?)
• Threshold value
• Optionally, polygons on the earths surface
• Could correspond to threat areas to aid
  algorithm report verification
• Collection algorithm can be fed new polygons at
  any time.

16
(3) Database of algorithm detections

• Once procured machines arrive
• Run algorithms on larger domain.
• Experiment with various collection algorithm
  settings.
• When primary SPC contributor returns from sick
  leave
• Ingest threat area forecasts and convert them
  into polygons.
• Run collection algorithm in real-time with
  appropriate settings.
• Develop statistics for comparisons.
• Create database of forecasts and algorithm
  detections.

17
JADE A Web-based Development and Testing
Environment for Remotely Developed QPE
Applications

• PI Steve Vasiloff
• DOC/NOAA/OAR
• National Severe Storms Laboratory
• HPCC 2005 - DIS/CE/09

18
Status

• A new server has been purchased to facilitate
  JADE on the NSSL high-performance computing (HPC)
  cluster
• Staff have been identified to develop the JADE
  design and infrastructure to accommodate the
  linkages and tool kits specified within JADE
• Initial JADE design is underway
• Collaboration and leveraging has started with
  UCAR/Unidata and the Unidata/Center for Analysis
  and Prediction of Storms Linked Environments for
  Atmospheric Discovery project

19
(No Transcript)
20
Urban Flash Flood Monitoring System -- Prototype
Development

• PI Ken Howard
• DOC/NOAA/OAR
• National Severe Storms Laboratory
• HPCC 2005 - DIS/CE/10

21
Status

• Hardware required for ingesting WSR-88D and TDWR
  and the processing of the high-resolution
  quantitative precipitation estimates is currently
  under procurement
• Software applications for ingesting and quality
  control of the radar data streams have been
  completed and tested
• Software applications for QPE generation are
  under initial development
• Background GIS data sets are being compiled for
  prototype site locations
• Due to various delays, milestone dates have been
  revised forward by 6 months

22
(No Transcript)
23
Accessing Real Time Canadian Radar Data

• PI Kevin Kelleher
• DOC/NOAA/OAR
• National Severe Storms Laboratory
• HPCC 2005 - TTR/CE/12

24
Status

• Completed a read program for a test data set for
  the Canadian high resolution data
• Eight radars are sending realtime data to NCDC,
  who then send it to NSSL via FTP
• Have recently contacted two technical radar
  experts within the Canadian Met office to obtain
  various radar characteristics (VCPs, gate
  spacing, elevation angles, rpm, etc)
• Initial deployment of the real time ingest for
  the Canadian radar data is expected to take place
  in July 2005