Ensemble Probabilistic Forecast and Uncertainty

1
Ensemble Probabilistic Forecast and Uncertainty

• Yuejian Zhu
• Environmental Modeling Center
• NCEP/NWS/NOAA
• June 2004
• Acknowledgements
• Zoltan Toth, Tim Marchok, Bill Bua

2
References

• 1. Toth, Talagrand, Candille and Zhu, 2003
• "Probability and ensemble forecasts" book
  chapter.
• 2. Zhu, 2004
• Prob. forecasts and evaluations based on a
  global ensemble prediction system
• In book of Observation, theory and modeling
  of atmospheric variability
• 3. Zhu, Iyenger, Toth, Tracton and Marchok, 1996
• "Objective evaluation of the NCEP global
  ensemble forecasting system"
• AMS conference proceeding.
• 4. Zhu, 2004
• Ensemble prediction to evaluate flow
  dependent variations in predictability
• COAA 2004 conference preprint
• 5. Toth, Zhu and Marchok, 2001
• "The use of ensembles to identify forecasts
  with small and large uncertainty".
• Weather and Forecasting
• 6. Zhu, Toth, Wobus, Rechardson and Mylne, 2002
• "The economic value of ensemble-based weather
  forecasts BAMS
• 7. Buizza, Houtekamer, Toth, Pellerin, Wei and
  Zhu, 2004
• Assessment of the status of global ensemble
  prediction, accepted by MWR

3
Clear sky, no precipitation
4
20mm/24hrs (0)
Precipitation
2mm/24hrs (30)
5
(No Transcript)
6
Ensemble Forecasts

• 1. Why do we need ensemble forecast?
• Look at following schematic diagrams

7
Ensemble Forecasts (continue)
Deterministic forecast
Initial uncertainty
Forecast probability

Verified analysis
8
Simple Measurement
9
Probabilistic Forecast (1)

• Introduce two characteristics
• reliability and resolution
• reliability --- forecast lt-gt
  observation
• the property of statistical consistency
  between predicted
• probabilities and observed frequencies of
  occurrence of
• the event under consideration.
• resolution --- forecast/observation lt-gt
  climatology
• the ability of a forecast system to discern
  sub-sample
• forecast periods with different relative
  frequencies of event
• (the difference between sub-sample and
  overall sample
• climatology).
• reliability and resolution together determine the
  usefulness
• of a probabilistic forecast system.

10
Probabilistic Forecast (2)

• 1. Talagrand Distribution (histogram
  distribution)
• Sorting forecast in order, to check where
  the analysis is falling
• Reliability measurement, system bias
  detected.
• If analysis and model are bias-free
• U sharp --- spread is too small
• flat line --- perfect spread to cover
  uncertainty

avg distribution
11
Probabilistic Forecast (3)

• 1. Talagrand distribution (continue).
• outlier evolution by different leading
  time
• adding up two outliers subtract the
  average.
• ideal forecasts will have zero outliers.

Due to inability of ensemble to capture model
related errors?
12
Probabilistic Forecast (4)

• 2. Outlier maps (2-dimension).
• flow dependent systematic model errors
• . 40 members at 4 different lead time.
• . measured by normalized distance.
• d (AVGe-ANL)/SQRT(1/(n-1) SPRDe)
• The model errors evaluation ( outliers ) is a
  part of model bias estimation ( see map ). The
  "Normalized Distance" is defined as the
  difference of AVE and ANL over the square root of
  normalized ensemble spread. Therefore, the
  positive distance means there are positive biases
  for ensemble forecasts, the negative means there
  are negative biases for ensemble forecasts.
• ---gt show example maps.
• . area where consecutive ensembles fail ( missed
  )
• -- initial errors ? / errors in boundary
  forcing ?
• model development - identify problem
  area.
• . weather system where consecutive ensembles
  fail ( missed )
• -- inability to capture model related
  errors?

13
Probabilistic Forecast (Useful tools)

• 1. Small and large uncertainty
• Productions spaghetti diagram,
• spread plots, norm. spread and RMOP
• for example of evaluation

14
Probabilistic Forecast (useful tools)

• ... Small and large uncertainty.
• 1 day (large uncertainty) 4 days (control)
  10-13 days (small uncertainty)

15
Ensemble Forecast for Uncertainty (1)
By Bill Bua
16
Ensemble Forecast for Uncertainty (2)
By Bill Bua
17
Ensemble Forecast for Uncertainty (3)
By Bill Bua
18
Ensemble Forecast for Uncertainty (4)
19
1. By using equal climatological bins
(e.g. 10 bins, each grid points)2. Counts of
ensemble members agree with ensemble mean, (same
bin)3. Construct n1 probabilities for n
ensemble members from (2).3. Regional (NH,
weighted) Normalized Accumulated Probabilities
(n1)4. Calculate RMOP based on (3), but 30-d
decaying average.5. Verification information
(blue numbers) historical average (reliability)
20
Ensemble mean
10 Climatological equally likely bins
Example of 1 grid point
10 ensemble forecasts
The value of ensemble members agree to ensemble
mean is 4/10 or 40 (probability) There are 10512
points ( values ) for global at 2.5 2.5 degree
resolution
10 ensemble members could construct 11
probabilities categories, such as 0/10 (0),
1/10(10), 2/10(20), 3/10(30), 4/10(40),
5/10(50), 6/10(60), 7/10(70), 8/10(80),
9/10(90), 10/10(100) Sum of each grid point for
above 11 probabilistic categories by area
weighted and normalized for global or specified
region Get 0/10 1/10 2/10 3/10
4/10 5/10 6/10 7/10 8/10 9/10
10/10 .029 .047 .077
.085 .100 .135 .116 .089 .081
.070 .177 sum of these 1.0
(1.007 here) 2.9 7.6 15.3 23.8
33.8 47.3 58.9 67.8 75.9 82.9 100
accumulated values There is 30-day decaying
average of above values ( last line ) in the
data-base and updated everyday. Assume these are
30-day decaying average values In this case,
point value is 4/10, RMOP value of this point is
33.8
21
China
22
General public for past 7 years
23
Specific request
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
By Tim Marchok
28
Application and discussion

• Spaghetti diagram, ensemble mean and spread/norm
  spread, PQPF, calibrated PQPF, precipitation
  type, and RMOP all on out web-page
• HPC has used some of them on daily discussion (
  such as RMOP)
• NRL will use our RMOP, or adopt the method
• How about NMC/CMA?

• Is there any other measurement for probabilistic
  forecast and uncertainty?
• Is there any other variables for RMOP?
• How to apply these measurement to regional
  ensemble forecast?