Applications of Ensemble Prediction a Historical Perspective

1
Applications of Ensemble Prediction - a
Historical Perspective
Steve Tracton Office of Naval
Research Arlington, VA
(Formally of NWS/NCEP)
2
KEY POINTS

• THERE ARE INEVITABLE UNCERTAINTIES IN NWP DUE TO
  UNCERTAINTIES IN INITIAL CONDITIONS AND MODEL
  FORMULATION

SCREAMING MESSAGE
THERE WILL ALWAYS BE VARYING DEGREES OF
UNCERTAINTIES IN FORECASTS (Chaos Theory)
3
KEY POINTS

• THERE ARE INEVITABLE UNCERTAINTIES IN NWP DUE TO
  UNCERTAINTIES IN INITIAL CONDITIONS AND MODEL
  FORMULATION
• ENSEMBLE PREDICTION FROM EARLY 90S ON,
  REVOLUTIONARY CHANGE IN THE THRUST OF OPERATIONAL
  NWP (WAVE OF THE FUTURE),

First formal attention to the real possibility of
OPERATIONAL ensemble prediction at ECMWF Workshop
on Predictability In the Medium Range and
Extended Range, 1986
Ultimately led to operational GLOBAL EPS at
ECMWF and NMC in Dec 1992
Followed from Sufficient CPU resources
becoming available Scientific basis for
generating dynamically constrained
initial state perturbations (SVs, Breeding)
Development of output products
4
Give Me Odds
ENSEMBLE FORECASTING
5
KEY POINTS

• THERE ARE INEVITABLE UNCERTAINTIES IN NWP DUE TO
  UNCERTAINTIES IN INITIAL CONDITIONS AND MODEL
  FORMULATION
• ENSEMBLE PREDICTION FROM EARLY 90S ON,
  REVOLUTIONARY CHANGE IN THE THRUST OF OPERATIONAL
  NWP (WAVE OF THE FUTURE), THE OBJECTIVES BEING
  TO

PROVIDE RELIABLE INFORMATION ON FORECAST
UNCERTAINTIES FROM THE SPREAD (DIVERSITY) AMONGST
ENSEMBLE MEMBERS

• USE TO
• Ascertain most likely deterministic prediction
• Confidence in deterministic forecast
• Same, plus identifying relative likelihood of
  alternative scenarios
• Full probability distribution maximum
  information

NET RESULT - ENHANCE UTILITY/VALUE OF NWP FOR
VIRTUALLY ALL APPLICATIONS
NOT NECESSARILY SKILL
6
Providing EPS Acceptance and Use
/
ECMWF
MRF
NOGAPS
UKMET
7
RUN TO RUN MODEL CONTINUITY
WHICH SOLUTION IS PREFERRED? ? DETERMINISTIC
THINKING
TODAYS DAY 3
YESTERDAYS DAY 4
GOOD
NEW 84 HR MRF
OLD 108 HR MRF
BAD
NEW 96 HR ECMWF
OLD 120 HR ECMWF
8
HPCs Extended Forecast Discussion(released Mar
2, 2001, at 338 p.m.)
EPS NOW
CONSIDERED INDISPENSIBLE TO HPC MEDIUM RANGE
FORECASTS (Jim Hoke)
STEEP LEARNING CURVE FOR NEW PARADIGMNO A-
PRIORI BEST SINGLE OUTCOME
DETERMINISM IS DEAD ?? NOT YET BUT

9

American Meteorological Society (AMS)
Statement Enhancing Weather Information with
Probability Forecasts (3/02 BAMS!) The AMS
endorses probability forecasts and
recommends their use be substantially
increased.
Would allow user (not forecaster!) to make
decisions based on quantified uncertainties with
resulting economic and social benefits ( e.g.,
from taking umbrella along, through canceling a
trip ,to evacuation from an impending threat)
10
Schematic of how a probabilistic model forecast
can be used for risk-based decision-making.
P1 P2 P3 P4 . . PN
F(P)
Model
Pcr
Compare T with Tcr Take
appropriate action
T
P
P1, P2 PN predicted variables of interest,
for example, precipitation amount Pcr
user-specified critical value of P which, if
exceeded, requires an action or decision T
chance of critical value being exceeded Tcr
user-specified tolerance level (depending on
societal, monetary, and/or environmental
considerations).
11

American Meteorological Society (AMS)
Statement Enhancing Weather Information with
Probability Forecasts (3/02 BAMS!) The AMS
endorses probability forecasts and
recommends their use be substantially
increased.
Would allow user (not forecaster!) to make
decisions based on quantified uncertainties with
resulting economic and social benefits ( e.g.,
from taking umbrella along, through canceling a
trip, to evacuation from an impending threat)
Probability Forecasts are particularly useful,
even necessary, to reliably provide early
warnings of extreme weather events
TO AVOID, E.G.
12
MAJOR SNOWSTORM AMBUSHES WASHINGTON
Not Good- especially when effecting DC (just
after announce-ment of new Super Computer by
NWSHQ
13
FOR FORECASTERS, ANOTHER
14

American Meteorological Society (AMS)
Statement Enhancing Weather Information with
Probability Forecasts (3/02 BAMS!) The AMS
endorses probability forecasts and
recommends their use be substantially
increased.
Would allow user (not forecaster!) to make
decisions based on quantified uncertainties with
resulting economic and social benefits ( e.g.,
from taking umbrella along through canceling a
trip to evacuation from an impending threat)
Requires Conveying rationale and nature of
uncertainties Products/graphics/presentations
that are readily comprehensible and relevant
15
KEY POINTS

• THERE ARE INEVITABLE UNCERTAINTIES IN NWP DUE TO
  UNCERTAINTIES IN INITIAL CONDITIONS AND MODEL
  FORMULATION
• ENSEMBLE PREDICTION - REVOLUTIONARY CHANGE IN THE
  THRUST OF OPERATIONAL NWP (WAVE OF THE FUTURE)
  - CONSISTS OF MULTIPLE PREDICTIONS FROM SLIGHTLY
  DIFFERENT INITIAL CONDITIONS AND/OR WITH VARIOUS
  VERSIONS OF MODELS, THE OBJECTIVES BEING TO
• REALIZING THE PRACTICAL UTILITY/VALUE OF
  ENSEMBLES ACCOMPLISHED VIA A VARIETY OF PRODUCTS
  DESIGNED TO CONDENSE AND MAXIMIZE INFORMATION
  CONTENT FOR USERS
• Products/graphics/presentations must be
  readily comprehensible
• and relevant
• (MUST BE USER SPECIFIC AND USER
  FRIENDLY)
• ?

USER FEEDBACK ESSENTIAL
16
Products Stamp maps
17

• TYPES OF PRODUCTS
• Roots of EPS products largely provided by Ed
  Epstein with graphical depictions to illustrate
  how uncertainty information could enhance
  forecast value (1971)

But, no further formal consideration of post
processing and presenting EPS output until 1992
ECMWF Workshop on New Developments in
Predictability (List of recommended generic
products largely reflected concept
demonstration mode of EPS experiments at
NMC/CAC, Tracton and Kalnay, 1993)
? SPAGHETTI CHARTS now one of the most
recognizable and symbolic products of EPS
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21

• TYPES OF PRODUCTS
• SPAGHETTI CHARTS
• MEAN/SPREAD
• PROBABILITIES
• STORM TRACKS
• CLUSTERS
• VERTICAL PROFILES
• METEOGRAMS
• ENSEMBLE DERIVED MOS

22
010519/0000V63 SREFX-CMB LIFTED INDEX PROB 0F lt
-4
23
(No Transcript)
24
5-Day ECMWF Ensemble Prediction of Typhoon Rusa
25

• TYPES OF PRODUCTS
• SPAGHETTI CHARTS
• MEAN/SPREAD
• PROBABILITIES
• STORM TRACKS
• CLUSTERS
• VERTICAL PROFILES
• METEOGRAMS
• ENSEMBLE DERIVED MOS

CAN BE APPLIED TO VIRTUALLY ALL MODEL AND MODEL
DERIVED PARAMETERS AND MODEL OUTPUT DRIVERS OF
SECONDARY SYSTEMS, E.G., WAVE, HYDROLOGICAL,POLUTI
ON DISPERSION MODELS
ULTIMATELY CAN (MUST) BE PROPOGATED TO USER
SPECIFCIC QUANTITIES, E.G., UMBRELLA SALES,
ENERGY USEAGE, TYPE OF SMART MUNITIONS, ETC.,
ETC., ETC., ETC., ETC., ETC .
26
MANY NEW AND VARIED PRODUCTS POSSIBLE BUT,
MOST SIGNIFICANT ADVANCE OVER PAST YEARS HAS BEEN
EXPONENTIAL GROWTH IN GRAPHICS CAPABILITIES -
ANIMATION, ZOOMING, 3-D, ETC. - AND PROLIFERATION
OF PCs AND HIGH SPEED INTERNET
27
MANY NEW AND VARIED PRODUCTS POSSIBLE BUT,
MOST SIGNIFICANT ADVANCE OVER PAST YEARS HAS BEEN
EXPONENTIAL GROWTH IN GRAPHICS CAPABILITIES -
ANIMATION, ZOOMING, 3-D, ETC. - AND PROLIFERATION
OF PCs AND HIGH SPEED INTERNET
SO, REAL CHALLENGE IS MORE FULLY EXPLOITING
POTENTIAL VALUE OF EPS IN BOTH PUBLIC AND PRIVATE
SECTORS
28
CONVEYING UNCERTAINTY TO PUBLIC VIA MEDIA - A
CRITICAL LINK (see IABM.org)
Bob Ryan
29
MON TUES WED
THURS FRI
40
36 28 23 21
31
40
30
20
THIS
10
30
INCONSISTENT WITH PREDICTABILITY LIMITS
NOT THIS
31
Visualizing UncertaintyinMesoscale Meteorology
APL Verification Methodology 21 May 2002 Scott
Sandgathe
32
(No Transcript)
33
Questions ??
34
SPAGHETTI CHART
35
What a mess!!
36
END
37
BACKUP SLIDES
38
EVEN EXPERTS CAN HAVE A BAD DAY
39
(No Transcript)
40

• High Resolution Mesoscale models
• allow us to see features not in coarser models
• But even small timing and placement errors can
  be significant in attempt to accurately forecast
  details (see Mass, et al., 3/02 BAMS!!!).
• But But Forcaster judgement could mitigate

• One model (even with forecaster input)
• is an all or nothing proposition gt

One detailed mesoscale model based
forecast could allow the user to make
highly specific and detailed inaccurate
forecasts. (after Grumm)
41

• Why we need ensembles
• Deal with uncertainties in analyses and model
  formulation
• But Requires tradeoffs when computer resources
  limited (e.g., model resolution)
• But But Mesoscale predictability often
  substantially controlled by synoptic
  predictability (and uncertainties therein)
• gt Subjective or statistically based
  downscaling possible to get
  uncertainties in mesoscale weather
• Ideal Ensembles with highest resolution
  justifiable
• Compromise Combination of single (or few) high
  resolution and coarser resolved ensemble

42
ADDITIONAL WEB SITES WITH ENSEMBLE PRODUCTS
AND INFORMATION Univ of Utah http//www.met.utah
.edu/jhorel/html/models/ model_ens.html
FNMOC http//152.80.49.204/PUBLIC/ Canada htt
p//www.cmc.ec.gc.ca/rpn/ensemble_products/ index.
html CDC http//www.cdc.noaa.gov/map/maproom/EN
S/ ens.html State College http//bookend.met.psu
.edu/ensembles/Ensemble.html
43
Give Me Odds
ENSEMBLE FORECASTING
44
(No Transcript)
45
(No Transcript)
46
EVEN EXPERTS CAN HAVE A BAD DAY
47
YOUR FORECAST HAS A 30 CHANCE OF BEING
70 CORRECT
48
3-day forecast from 00 UTC 11/2/01, spaghetti
diagram for ensemble
global
Uncertain location of incoming western trough
Uncertain amplitude of eastern trough
From CDC web site http//www.cdc.noaa.gov/map/i
mages/ens/ens.html
49
SPAGHETTI DIAGRAM
564 DM CONTOUR NCEP ENSEMBLE
SOME NCEP ENSEMBLE MEMBERS OFFER A MORE DIGGY
CENTRAL U.S. TROF THAN THE OPERATIONAL MRF..BUT
NOT AS STRONG AS THE ECMWF
DAY3