WDSS-II Training Module IV

1
WDSS-II Training Module IV

• Algorithms and Tools

2
General Notes

• Output from WDSS-II applications may be shared
  across multiple machines
• Any application can use the output of another
  application as input
• The wg display is an example of this
• It provides input/launch to the Filter
  algorithms
• It uses products from other algorithms
• Real-time and data playback modes are
  essentially the same modes of operation

3
WDSS-II application types

• Data ingest applications (ingestors)
• Single-source algorithms
• Usually single-radar applications
• Multi-source algorithms
• Combine input data from multiple sources of one
  or more instrument types
• General use tools
• Data filters, objective analysis tools, data
  remapping, data converters, verification tools,
  etc.

4
WDSS-II primary data types

• LatLonGrid geographic projection
• Equal spacing in degrees latitude and longitude
• RadialSet cylindrical projection
• Accommodates any number of radials with variable
  radial widths
• PolarGrid an indexed RadialSet
• DataTable for point data
• Trends
• tracks
• CartesianGrid equidistant projection
• equal spacing in N/S/E/W directions
• Other types to be described in a later
  presentation

5
Data ingest

• Data-ingesting programs read raw data files and
  convert them to one of the internal WDSS-II
  formats
• New input types are easy to add
• Maintains a consistent internal structure for
  data sharing among applications

6
WDSS-II Real-time data flow
Single-radar products
Legend
WSR-88D data (level 2)
RUC analysis data (grib)
satellite data
Data sources are in ovals
Satellite data are required to be in netcdf
format.
Applications are in boxes
gribToNetcdf
nse1
w2cloudcover
swatScit2D
w2circ
Scit2D (table)
AzShear Divergence AzShear layers
1If nse is not used as an input, then
PolarHail.xml and ssaparm.dat should be updated
twice daily. It is highly recommended to use nse
data if accurate hail guidance are desired.
7
The most-used single-source algorithms

• w2qcnn quality control neural network
• May use radar-only data, or radar plus cloud
  cover information
• Output ReflectivityQC ReflectivityQComposite
• http//cimms.ou.edu/lakshman/Papers/qcnnjam.pdf
• w2circ radial velocity derivatives
• Produces rotational (AzShear) and divergent
  (Divergence) shear fields for every tilt
• Also produces layer maxima (e.g. 0-3 km MSL)

8
The most-used single-source algorithms

• nse near-storm environment
• Parameters are derived from the RUC model
  analysis
• Provides input to other algorithms
• Output similar to SPC mesoanalysis web page

9
Other single-source algorithms

• w2hail hail grids and echo tops
• w2vil VIL and composite reflectivity
• netssap the original SSAP
• MDA, TDA, SCIT, HDA, DDPDA
• Requires copy of .dat configuration files in
  working directory
• dealias independent executable of WSR-88D build
  10 dealiasing
• Note that dealiasing is usually done
  automatically in data ingest process for WSR-88D
  data (ldm2netcdf)

10
WDSS-II Real-time data flow
Multi-radar products
Legend
nse
Data sources are in ovals
Scit2D (x N radars or from w2merger)
AzShearlayer
ReflectivityQC (x N radars)
If nse is not used as an input, then
MRScitHail.xml should be updated twice daily. It
is highly recommended to use nse data if accurate
hail guidance are desired.
qcinfo
Applications are in boxes
w2merger
w2merger
QCTimeInfo
MergedAzShearlayer
scit3D
MergedReflectivityQC MergedReflectivityQCCompo
site VIL products Reflectivity_X1C EchoTop_Y2 HY2_
Above_HX1 (Height Above Isosurface) MESH /POSH
/ SHI (Hail) Scit2D (from 3D grids) 1isosurface(C)
2reflectivity value (dBZ)
w2segmotion
MR_Celltable
ClusterTable MergedReflectivityQCCompositeForeca
st (15,30,45,60 min) Windfield
w2accumulator
(RotationTracks)
MESH Tracks (2 hr, 6 hr, etc)
11
w2merger

• Multi-radar data merging
• 2D or 3D
• Continuously updating
• The grid is updated each time data are received
  from any source
• Writes output at user-specified time intervals
• Any resolution (Vertical/horizontal)
• Also runs algorithms on the 3D data field
• http//cimms.ou.edu/lakshman/Papers/w2merger.pdf

12
w2merger preparations cache

• Pre-compute the radars that will sample the grid
  point (the cache)
• Makes all computations faster
• Beam blockage is considered
• Use program createCache (once for each radar)
• w2merger will create a cache on-the-fly if one is
  not available, but
• It will not include terrain data
• Data will not be processed until the cache
  creation is complete (which might take a while)

13
w2merger preparations cache

• By default, the cache is stored in
  /.w2mergercache
• It might be big! If you are finished processing
  a domain, you should delete it
• A cache may be extracted from a cache with larger
  spatial extents (createCache e)
• Within NSSL extract from /mnt/radararchive
• Another option createSubdomains create caches
  for all radars in the domain

14
w2merger preparations cache

• You may reduce the number of radars that affect a
  point by running postprocessCache
• e.g. if you only want the 3 best radars to
  impact the calculation at a point

15
Merging strategies

• Different products may require different ways of
  combination
• Set through the -C option
• Some examples
• Reflectivity ExponentialTimeAndDistance or
  Distance
• AzShear MagnitudeMaximum
• Velocity InverseVAD or MultiDoppler
• Choose the most appropriate method for the
  product you are merging.
• There are others see w2merger usage for list
• If you need a different merging option, add it!

16
Running merging and algorithms separately

• Algorithms may be run each time w2merger writes
  out 3D grids of reflectivity data
• If the merger is CPU-intensive or I/O-intensive,
  then run the algorithms separately, perhaps on
  another machine
• w2merger option -C 10

17
w2merger algorithms(-a option)

• Composite or VerticalMaximum
• vertical maximum at each lat/lon
• VerticalMinimum
• vertical minimum product at each lat/lon
• AbsMax or AbsoluteMaximum
• abs-max product at each lat/lon. The result
  retains the sign of the maximum.
• VIL
• vertical integrated liquid product at each
  lat/lon (assumes that the 3D grid is a grid of
  Reflectivity)
• Includes different integration strategies (e.g.
  along storm tilt, VIL Density, etc)

18
w2merger algorithms(-a option)

• HDA
• produces SHI, POSH, and MESH at each lat/lon
  (assumes that the 3D grid is a grid of
  Reflectivity).
• SCIT
• creates 2D storm cell features from the
  multi-radar grid (assumes a grid of
  Reflectivity).
• LayerAverage or Isotherms
• produces Reflectivity at various isotherms (0,-10
  and -20C), ReflectivityBelowZero,
  LowestReflectivity, etc.

19
w2merger supplemental output

• MergerInputRadarsTable
• Provides information about the current data
  streams
• Age
• Tile
• VCP
• Useful for determining which radars went into the
  output

20
w2segmotion storm segmentation and motion
estimation

• Multiple scales
• Can generate statistics based on storm areas
• Motion estimates feed back into w2merger for
  time/space correction
• http//cimms.ou.edu/lakshman/Papers/kmeans_motion
  .pdf

21
Mr. SCIT (Multi-radar storm cell identification
and tracking

• scit3D executable
• Use -g option for Scit2D features generated by
  w2merger
• Use -t option to ingest grid fields of various
  parameters that should be added to the output
  table
• Environmental data from RUC analysis
• Precipitation rate field
• Etc.
• Produces MR_CellTable output

22
w2accumulator

• Take the
• Maximum
• Minimum, or
• Sum
• of all tables or grids produced over a
  specified time interval. E.g.
• 2-hour max MESH a hail swath
• 6-hour precipitation rate integration
• 4-hour max of 0-3 km Azimuthal Shear (Rotation
  Tracks)
• DataTable, RadialSet, or LatLonGrid

23
Other useful algorithms

• w2cloudcover estimate cloud cover over a region
  using IR satellite and surface temperature
• w2vortdiv compute vorticity and divergence from
  a 2D wind field
• w2alarm collect statistics within an
  earth-relative polygon for any grid

24
Data Converters

• w2awipsnc convert WDSSII netcdf grid files to
  AWIPS format
• w2cropconv convert and remap any WDSSII
  RadialSet or LatLonGrid to a LatLonGrid
• w2csv2table convert a CSV file (spreadsheet) to
  a WDSS-II DataTable
• w2table2csv vice versa

25
Data Converters

• w2geotiff convert a WDSSII netcdf file to a
  geoTIFF file
• A TIFF image file with geographic information
  tags (for GIS)
• w2grib2conv convert a WDSS-II file to GRIB2
• netcdf2ldm convert a set of WDSSII netcdf files
  to WSR-88D level II format
• Can replace AliasedVelocity with Velocity,
  Reflectivity with ReflectivityQC for example

26
Objective analysis / filters

• w2smooth smooth the data using one of many
  strategies
• Gauss
• Cressman
• Percent (e.g. median)
• Oriented
• Ellipse
• Various wavelets

27
Objective analysis / filters

• w2threshold Thresholds one field based on
  another
• Example, remove VIL in areas where the IR
  temperature is gt 250K
• Various options to smooth (using w2smooth
  internally) and/or segment field

28
Objective analysis / filters

• w2oban convert point data to a LatLonGrid
• w2morph morphological filters
• Dilate
• Erode
• w2contour create contours of a data field

29
File manipulation

• w2get copy a file via rssd
• w2mirror mirror all the files listed in an lb to
  a different machine
• Limits the number of users hitting a real-time
  machine
• w2simulator simulate real-time data playback
• w2stitcher stitch together two different domains
  into one larger one

30
Suggested exercise on archive data

• Download KTLX and KINX data from May 20, 2001
  from 2100 to 2200 UTC from NCDC
• Convert it into WDSS-II netcdf format
• Run w2vil to produce VIL estimates in
  rapid-update mode
• Merge the VIL estimates using w2merger
• What is a valid combination strategy here?
• createCache before merging!
• Compute VIL from merging reflectivity data
• Compare the two VIL estimates
• Find their difference field using w2scoregrid

31
Suggested exercise on real-time data

• Connect to two adjacent radars that are currently
  experiencing weather
• Look at the 2DConUS index
• Overlay the radarsites shapefile
• Find LB names from the tensor list
• Create cache for domain using createSubdomains.
• Extract from /mnt/radararchive
• Run w2vil, w2merger and w2scoregrid as described
  before.
• Set up a w2alg.conf to do this.

32
End of WDSS-II Training Module IV

• What to do next
• Practice running some algorithms and tools.
• You will not be able to follow module 6 (writing
  a WDSS-II algorithm) unless you are familiar with
  how WDSS-II algorithms in general work.
• Run both a single-radar algorithm and a
  multi-sensor algorithm.
• Run both on archived cases and real-time cases.
• Next module Configuring WDSS-II