IPWG Validation current status and future directions

1
IPWG Validationcurrent status and future
directions
Chris Kidd Beth Ebert John Janowiak
The University of Birmingham, Birmingham,
UK Bureau of Meteorology, Melbourne,
Australia CPC/NCEP/NWS/NOAA, USA
4th International Precipitation Working Group
(IPWG) Workshop 13-17 October 2008 Beijing, China
2
IPWG validation sites

• Addresses IPWG objectives
• 2) Establish standards for validation and
  independent verification of precipitation
  measurements
• 3) Foster the exchange of data on
  intercomparisons of operational precipitation
  measurements from satellites
• Main regions (initiated 2002) are
• Australia, United States, Europe and South
  America,
• with Japan, South Africa and others for selected
  periods
• Common graphical, descriptive and statistical
  analysis of daily, 0.25 resolution products.

4th International Precipitation Working Group
(IPWG) Workshop 13-17 October 2008 Beijing, China
3
GV-IPWG synergies
After Turk Arkin, BAMS 2008
GVGround Validation
Both approaches are complementary
4th International Precipitation Working Group
(IPWG) Workshop 13-17 October 2008 Beijing, China
4
Global precipitation data sets
4th International Precipitation Working Group
(IPWG) Workshop 13-17 October 2008 Beijing, China
5
IPWG Intercomparison regions
Near real-time intercomparison of model
satellite estimates vs radar/gauge
4th International Precipitation Working Group
(IPWG) Workshop 13-17 October 2008 Beijing, China
6
4th International Precipitation Working Group
(IPWG) Workshop 13-17 October 2008 Beijing, China
7
Processing system
Initial setup Setting of dates Cleaning out
old/decayed data
Remapping of data to regional grid or 5 km PSG
projection
Results generation Statistical
analysis Graphical output
Acquiring data Searching existing data Listing
missing data Creation of .netrc file ftp data
sources
Web pages Generate HTML files Copying to server
8
Processing checks
9
Processing checks
Set up list of past dates/days Usually okay
sometimes needs tweaking
Prepares products into common format Usually okay
Checks for a products results Okay if no
results, but not if bad data
Generates outputs Okay if there is rain
Generates raw HTML Occasional issues with server
FTP runs several times 4K buffer limit on macros
Automated systems they are NOT!
10
Validation products

• Both gauge and radar are used depending upon
  the real-time accessibility to these data sets
• Australia gauge analysis
• United States gauge and radar analysis
• Europe radar analysis
• South America Gauge analysis

European radar data European composite with
different radar and calibration and QC
strategies nominal 5km 15-minute product
available within 1 hour errors include range
effects and anaprop (shipping, etc) Gauge data
09-09Z data (dated at end of period) 6000 daily
gauges with mean 10km spacing available 1 month
in arrears
4th International Precipitation Working Group
(IPWG) Workshop 13-17 October 2008 Beijing, China
11
Radar vs gauge data
Radar (daily integrated)
Gauge data
4th International Precipitation Working Group
(IPWG) Workshop 13-17 October 2008 Beijing, China
12
Overall results

• There is a clear seasonal dependence of satellite
  product performance (poorer in winter)
• Models overestimate rain area/extent
• Satellite products underestimate rain area/extent
• Significant day-to-day variations
• Surface contamination of satellite products
  noticeable in winter

Performance dependent upon precipitation extent
and intensity, spatial and temporal resolution of
product implications for high-resolution
precipitation products
4th International Precipitation Working Group
(IPWG) Workshop 13-17 October 2008 Beijing, China
13
The future of IPWG validation regions
What are the key requirements in the future to
make best of our own limited resources? What are
the requirements of the user community? What are
the requirements of the algorithm/product
developers? What new sources of data are
available? Should we go beyond daily regional
comparisons (local-global, instantaneous-seasonal)
?
4th International Precipitation Working Group
(IPWG) Workshop 13-17 October 2008 Beijing, China
14
The future of IPWG validation regions

• New sites
• BALTEX region limited number of algorithms (and
  need to read HDF5!) also Canada (limited
  coverage)
• High-latitude oceanic location (Iceland
  Shetlands?)
• Broaden existing regions (e.g. Europe)
• New comparison techniques
• high-resolution products will need to rely upon
  descriptive statistics
• implementation of fuzzy-logic statistical
  techniques
• New performance results
• Seasonal and global comparisons

4th International Precipitation Working Group
(IPWG) Workshop 13-17 October 2008 Beijing, China
15
The future of IPWG validation regions
International framework
Current European radar network used in IPWG
EUMETNET radar coverage
4th International Precipitation Working Group
(IPWG) Workshop 13-17 October 2008 Beijing, China
16
Latitude profiles rainfall occurrence
17
Monthly and seasonal validation

• Monthly and seasonal diagnostic validation
  summaries

4th International Precipitation Working Group
(IPWG) Workshop 13-17 October 2008 Beijing, China
18
The future of IPWG validation regions

• Suggested analysis levels
• real time analysis 'correctness' of near
  real-time products should these be easily
  available (cics?)
• 'historical' analysis 'authorised' release of
  quality-controlled products
• high-resolution analysis similar set-up to
  daily/0.25 degree products (including
  instantaneous)
• 'training' data sets example data sets from a
  comprehensive range of precipitation products
  common naming convention and format

4th International Precipitation Working Group
(IPWG) Workshop 13-17 October 2008 Beijing, China
19
Personal recommendations

• Evaluation of combined techniques
• individual component products for combined
  algorithm
• actual combination techniques
• Standardisation of precipitation product format
• simple 2D arrays with accompanying text files
• common format variable size
• quantitative resolution 0.1 mm/day (WMO
  standard)
• error fields can be accompanying files
• Naming conventions standardised

The IPWG gauntlet . to generate global 1km, 1
minute rainfall estimates.
4th International Precipitation Working Group
(IPWG) Workshop 13-17 October 2008 Beijing, China
20
Anagnostou Hossain Satellite error propagation
in flood prediction
Northern Italy Basin scales from 116 to 1200
km2, storm induced flash flood event (Nov
1996) Satellite products Kidd, 1h/4km, and 3B42,
3h/25km reference rainfall radar at
1h/1km Ensembles generated based on SREM2D
Hydrological model simulations based on tRIBS
distributed rainfall-runoff model
High resolution products do work!
21
Recommendations
To establish guidelines for satellite
precipitation product data establish common
criteria that can and will be used to promote
long-term data sets
To widen our validation regions to provide
globally representative (inter)comparisons at
daily scales, but also monthly and seasonal
scales.
Investigate statistical tests that are more
relevant to the data sets we are dealing with,
and that be applied over a range of spatial and
temporal scales
To promote near real-time high resolution
(sub-daily, lt0.25 degree) precipitation
inter-comparisons
4th International Precipitation Working Group
(IPWG) Workshop 13-17 October 2008 Beijing, China