Economic Value Decision Make

1
Economic Value Decision Make

• Yuejian Zhu
• Environmental Modeling Center
• NCEP/NWS/NOAA
• Shanghai, China
• October 6th 2006
• http//wwwt.emc.ncep.noaa.gov/gmb/ens/
• http//wwwt.emc.ncep.noaa.gov/gmb/yzhu/
• Acknowledgements
• Z. Toth and S. Lord (NCEP)
• R. Buizza (ECMWF) and P. L. Houtekamer(MSC)

2
Contents

• Introduction and useful references
• Why do we need ensemble forecast
• Methodologies of ensemble model forecast
• Review of statistical ensemble forecast
• The skill of ensemble model forecast
• Examples of high predictable system
• The application of ensemble forecast
• International research projects
• Discussion and conclusions

3
References

• Zhu, 2005 Ensemble forecast A new approach to
  uncertainty and predictability AAS
• Toth, Talagrand, and Zhu, 2006 The attributes
  of forecast system book chapter. Cambridge
  University Press
• Toth, Talagrand, Candille and Zhu, 2003
  "Probability and ensemble forecasts" book
  chapter.
• Zhu, 2004Probabilistic forecasts and
  evaluations based on a global ensemble prediction
  system In book of Observation, theory and
  modeling of atmospheric variability,
• Zhu, Iyenger, Toth, Tracton and Marchok,
  1996"Objective evaluation of the NCEP global
  ensemble forecasting system" AMS conference
  proceeding.
• Toth, Zhu and Marchok, 2001"The use of ensembles
  to identify forecasts with small and large
  uncertainty". Weather and Forecasting
• Zhu, Toth, Wobus, Rechardson and Mylne, 2002The
  economic value of ensemble-based weather
  forecasts BAMS
• Buizza, Houtekamer, Toth, Pellerin, Wei and Zhu,
  2005 Assessment of the status of global
  ensemble prediction MWR
• more related articles

4
GEFS configurations
5
One day advantage
Due to model imperfection
6
Prob. Evaluation (cost-loss analysis)

• Based on hit rate (HR) and false alarm (FA)
  analysis
• .. Economic Value (EV) of forecasts

Ensemble forecast
Average 2-day advantage
Deterministic forecast
7
NCEP ensemble mean performance for past 5-year
8
Ranked probabilistic skill scores
NCEP ensemble probabilistic performance for past
5-year
Economic values for 110 cost/loss ratio
9
Prob. Evaluation (useful tools)

• ... Small and large uncertainty.
• 1 day (large uncertainty) 4 days (control)
  10-13 days (small uncertainty)

10
Relative Operating Characteristics area (ROC area)
f(noise)
f(signal)
1
 
           
 

 
 
 
Hit rate
 
0
1

False alarm rate
Decision threshold
11
Prob. Evaluation (cost-loss analysis)

• Based on hit rate (HR) and false alarm (FA) rate.
• 1. Relative Operating Characteristics (ROC) area
  - Appl. of signal detection theory for measuring
  discrimination between two alternative outcome.
• ROCarea Intergrated area 2 ( 0-1
  normality )

h/(hm)
Relative Operating Characteristics
-------------------------- o\f
y(f) n(f) --------------------------
y(o) h m -------------------------
- n(o) f c -------------------
-------
f/(hf)
12
(No Transcript)
13
Example of cost-loss analysis (economic values)

• Wind sheer damages the airplane
• Un-protect airplane (loss)
• 2-million dollars for each airplane
• Protect airplane (cost)
• 20,000 dollars for each airplane
• For decision makers !!!
• 1100 cost-loss ratio for this case
• Probabilistic forecast and forecast reliability
• Typhoon Mai-sha affected Beijing City
• Un-protect (may loss)
• Flooding, traffic and others.
• Protect (definitely cost)
• Activities will be cancelled
• Labors cost
• Others
• Scientific decision ???
• Anyone counts this ratio?

14
Decision Theory Example
Forecast?
YES NO
Critical Event sfc winds gt 50kt Cost (of
protecting) 150K Loss (if damage ) 1M
Hit False Alarm
Miss Correct Rejection
YES NO
150K
1000K
Observed?
150K
0K
15
High predictable heavy precipitation event
GFS ENS
February 12-13 1997 (Southern Louisiana flooding)
Location and intensity
GFS made a very good forecast, But Ensemble made
a excellent forecast.
16
HIGHLY PREDICTABLE HEAVY PRECIPITATION EVENT
(20010313)
17
High Predictable Heavy Precipitation Events
(20010113)
Ensemble-based precipitation forecasts gave
relatively high probability values for the half
and one inch thresholds for the 24-hr period
ending 031312 for the Gulf states with 1 through
8 days lead time. The corresponding observed
precipitation amounts indicate that the forecasts
were rather successful. The high predictability
in precipitation was associated with high
confidence (and well verifying) forecasts for 500
hPa height. The cut-off low over the SW US that
allowed Pacific air to reach the Gulf of Maxico
at low latitudes (over and south of Baja CA) was
well predicted, with high confidence, at various
lead times (see, for example, at 4, 7, and even
at 10 days). Red colors in these charts over the
cut-off low correspond to an area associated with
high predictability.
18
RMOP
19
DETERMINSTIC/PROBABILISTIC FORECASTQPF .vs. PQPF

• Northern California State Christmas-New Year
  flooding.
• Winter storm last more than 10 days.
• Total precipitation amount exceeding 660mm over
  the huge area.
• The homes of 100,000 residents who has been
  evacuated.
• Some stranded residents has to be rescued by
  helicopter.
• Caused a lot of damages include road, bridge and
  resident houses.

Photo from Washington Post
20
24 hours observation
GFS ENS
21
(No Transcript)
22
Ensemble Based Hurricane Track Plots
Karl (09/18)
Frances (08/28)
23
Example of probabilistic forecast in terms of
climatology
24
ENSEMBLE 10-, 50- (MEDIAN) 90-PERCENTILE
FORECAST VALUES (BLACK CONTOURS) AND
CORRESPONDING CLIMATE PERCENTILES (SHADES OF
COLOR)
25
The pre-NWP forecast accuracy

• A schematic illustration of the increase of
  RMSE with forecast time. The pre-NWP forecaster
  started from a persistence forecast which he
  skillfully extrapolated into the future,
  converging towards climate for longer ranges

A?2
persistence
A
meteorologist

• The time unit can be anything from hours to
  days depending on the parameter (hours for
  clouds, days for temperature)

26
NWP more accurate - but also less
persistence
A?2

• A good NWP model is able to simulate all
  atmospheric scales throughout the forecast. It
  has the same variance as the observations and the
  persistence forecasts, which yields an error
  saturation level 41 above the climate

worlds best NWP
A
meteorologist
27
The art of good forecasting

• The way out of the dilemma
• Combine the high accuracy of NWP in the
  short range with a filtering of the
  non-predictable scales for longer ranges
• This can be done both with and without the EPS

A?2
persistence
worlds best NWP
A
meteorologist
modified NWP forecast
28
Thank You!!!