Probability Forecasts from Ensembles and their Application at the SPC

1
Probability Forecasts from Ensembles and their
Application at the SPC

• David Bright
• NOAA/NWS/Storm Prediction Center
• Norman, OK
• AMS Short Course on
• Probabilistic Forecasting
• January 9, 2005
• San Diego, CA

Where Americas Climate and Weather Services Begin
2
(No Transcript)
3

• Outline
• Motivation for ensemble forecasting
• Ensemble products and applications
• Emphasis on probabilistic products
• Ensemble calibration (verification)
• Decision making using ensembles

4

• Outline
• Motivation for ensemble forecasting
• Ensemble products and applications
• Emphasis on probabilistic products
• Ensemble calibration (verification)
• Decision making using ensembles

5

• Daily weather forecasts begin as an initial-value
  problem on large supercomputers
• To produce a skillful weather forecast requires
• An accurate initial state of the atmosphere to
  begin the model forecast
• Computer models that realistically represent the
  evolution of the atmosphere (in a timely manner)
• With a reasonably accurate initial analysis of
  the atmosphere, the state of the atmosphere at
  any subsequent time can be determined by a
  super-mathematician." (Bjerknes 1919)

6
Example Determinism
60h Eta Forecast valid 00 UTC 27 Dec 2004 PMSL
(solid) 10m Wind 1000-500 mb thickness (dashed)
7
Example Determinism
60h Eta Forecast valid 00 UTC 27 Dec 2004 PMSL
(solid) 10m Wind 1000-500 mb thickness
(dashed) Precip amount (in) and type (bluesnow
greenrain)
8
Example Determinism
60h Eta Forecast valid 00 UTC 27 Dec 2004
Truth 00 UTC 27 Dec 2004
9
Example Determinism
?
60h Eta Forecast valid 00 UTC 27 Dec 2004
Truth 00 UTC 27 Dec 2004

• Ignores forecast uncertainty
• Potentially misleading
• Oversells forecast capability

10
The Butterfly Effect

• Ensemble forecasting can be traced back to the
  discovery of the "Butterfly Effect" (Lorenz 1963,
  1965)
• Atmo a non-linear, non-periodic, dynamical system
  causes even tiny errors to grow upscale ...
  resulting in forecast uncertainty and eventually
  chaos

11
The Butterfly Effect

• Discovery of the butterfly effect (Lorenz
  1963)
• Simplified climate model When the integration
  was restarted with 3 (vs 6) digit accuracy,
  everything was going fine until

Time
12
The Butterfly Effect

• Solutions began to diverge

Solutions diverge
Time
13
The Butterfly Effect

• Soon, two similar but clearly unique solutions

Solutions diverge
Time
14
The Butterfly Effect
Chaos

• Eventually, results revealed two uncorrelated
  and completely different solutions (i.e., chaos)

Solutions diverge
Time
15
The Butterfly Effect
Chaos

• Ensembles can be used to provide information on
  forecast uncertainty
• Information from the ensemble typically consists
  of
• Mean
• (2) Spread
• (3) Probability

Solutions diverge
Time
Ensembles useful in this range!
16
The Butterfly Effect
Chaos
Ensembles extend predictability

• Ensembles extend predictability
• A deterministic solution is no longer skillful
  when its error variance exceeds climatic variance
  
• An ensemble remains skillful until error
  saturation (i.e., until chaos occurs)

Solutions diverge
Time
Ensembles especially useful in this range!
17

• NWP models...
• Doubling time of small initial errors 1 to 2
  days
• Maximum large-scale (synoptic to planetary)
  predictability 10 to 14 days

Its hard to get it right the first time!
18
Example Synoptic Scale Variability7 day
forecast NCEP MREF 500 MB Height
GFS -12h Control
GFS Control Forecast
GFS Pert
European Model and Start
19
Example Mesoscale Variability1.5 day forecast
NCEP SREF Precipitation

• Reveals forecast uncertainty, e.g., se U.S.
  precip
• Sensible weather often mesoscale dominated

20
Sources of Uncertainty in NWP

• Observations
• Density
• Error
• Representative
• QC
• Analysis
• Models
• LBCs, etc.

Satellite
Land
RMSD ECMWF-NCEP 500 mb Hght (5 winters)
21
Schematic Illustration Ensemble
ConceptsAnalysis, Model, and Subgrid-scale
Errors...
All plausible atmospheric states
All equally-likely solutions
All equally-likely ICs
22
Error Growth with Time Idealized
Limit of ensemble skill
No correlation to initial conditionschaos!
Forecast Error
Limit of single model skill
Expected climate variability
We can use ensembles (e.g., probabilities, etc.)
to extend predictability ( 3 to 4 days for
synoptic scale pattern) until the forecast
becomes chaotic.
23
500 mb Hght (Dec. 2004 Greater U.S. Area)
Error Growth with Time GFS
120 m
1.41 x Climate SD
GFS
100 m
Ens Means
Climate SD
80 m
RMSE
Limit of ensemble skill 10.5 days
40 m
Limit of deterministic skill 7.5 days
20 m
1
2
3
4
5
7
8
9
10
11
12
Days
24
Ensembles vs. Determinism
Determinism
Ensemble
Evaluating Weather Forecasts
25

• Outline
• Motivation for ensemble forecasting
• Ensemble products and applications
• Emphasis on probabilistic products
• Ensemble calibration (verification)
• Decision making using ensembles

26
Definitions

• SREF NCEP Short Range Ensemble Forecast (5
  Eta-BMJ 5 EtaKF 5 RSM)
• MREF NCEP Medium-Range Ensemble Forecast (GFS)
• Mean Arithmetic average of members
• Spread Variance or Standard Deviation
• Probability of members meeting some condition
• Calibrated Probability As above, but adjusted
  to reflect expected frequency of occurrence

27
SPC Approach to Ensembles

• Develop customized products based on a particular
  application (severe, fire wx, etc.)
• Design operational guidance products that
• Help blend deterministic and ensemble approaches
• Facilitate transition toward probabilistic
  thinking
• Aid in critical decision making
• Increase confidence
• Alert for rare but significant events

28
Ensemble Means
F15 SREF MEAN 500 MB HGHT,TEMP,WIND
29
Synoptic-Statistical RelationshipsMean Spread

• Examples of simple relationships between
  dispersion patterns and synoptic interpretation
  can be defined.
• Obtain a quick overview of range of weather
  situations from ensemble statistics.

30
Ensemble Mean Spread
F15 SREF MEAN/SD 500 MB HGHT
31
Ensemble Mean Spread
Increased spread
Less predictability
Less forecast confidence
500 mb Mean Height (solid) and Standard Deviation
(dashed/filled)
F048
F000
F096
F144
32
Ensemble Mean Normalized Spread
500 mb Mean Height and Normalized
Variance Normalize the ensemble variance by
climatic variance Values approaching 2 (dark
color fill) gt Ensemble variance saturated based
on climo
F048
F000
F096
F144
2
33
Another way to view uncertaintySpaghetti
500 mb Member Height Spaghetti - 5640 meter
contour
F048
F000
F096
F144
34
F63 SREF POSTAGE STAMP VIEW PMSL, HURRICANE
FRANCES
Red EtaBMJ Yellow EtaKF Blue
RSM White OpEta
SREF Member
35
Spatial Variability Median Range
All 16 members have gt500 J/kg CAPE
At least 1 member has gt 500 J/kg
Median
F15 SREF MEDIAN/RANGE CAPE
36

• Creation of Severe Wx Diagnositics
• - Calculated Craven-Brooks
• Significant Severe
• parameter for each member

All 16 members have gt 10,000 m3/s3
At least 1 member has gt 10,000 m3/s3
Median
F15 SREF MEDIAN/RANGE MLCAPE X 0-6 KM SHEAR
37
Ways to view central value Mean

• Arithmetic mean
• Easy to compute and understand
• Tends to increase coverage of light pcpn and
  decrease max values.

3-hr Total Pcpn NCEP SREF F63 Valid 09 Oct 2003
00 UTC
38
Ways to view central value Median

• Median
• If the majority of members dont precip, will
  show large areas of no precip. Thus, often
  limited in areal extent.

3-hr Total Pcpn NCEP SREF
39
Ways to view central value Probability Matching

• The blending of two PDFs, when one provides
  better spatial representation e.g., ensemble
  mean QPF and the other greater accuracy e.g.,
  QPF from all members. See Ebert (MWR 2001)
  for more info.
• Rank Ens Mean Rank Member QPF
• 1 ?
  1
• 2 ?
  16
• 3 ?
  32

40
Ways to view central value Probability Matching

• Probability matching
• Ebert (2001) Found to be the best ensemble
  averaged QPF
• Max values restored pattern from ens mean

3-hr Total Pcpn NCEP SREF
41
Uncalibrated probabilities Fraction of members
meeting some condition
42
Probabilistic Output of Basic Products 2 m
Dewpoint
Probability 144h 2 meter Td lt 25 degF
43
Probabilistic Output of Derived Products Haines
Index
Probability 144h Haines Index gt 5
44
Probability Convective Pcpn gt .01
Prob Conv Pcpn gt .01 Valid 00 UTC 20 Sept 2003
45
Probability Convective Pcpn gt .01
Prob Conv Pcpn gt .01 Valid 00 UTC 20 Sept 2003
46
Pcpn probs due to physics - No EtaBMJ members?!
Spaghetti Different physics
Note clustering by model
Red EtaBMJ Yellow EtaKF Blue RSM
47
Spaghetti Outliers
Red EtaBMJ Yellow EtaKF Blue RSM White
solid 12 KM OpEta (12 UTC)
12 UTC operational Eta clearly an outlier from 09
UTC SREF - Is this the result of ICs or
resolution? - Is this a better fcst
(updated info) or an outlier
F39 SREF SPAGHETTI (1000 J/KG)
48
Extreme Values Lowest RH
144h Minimum RH from any ensemble member Worst
case scenario
49
Extreme Values

• Any member can
• contribute to the
• max or min value
• at a grid point

F15 SREF MAXIMUM FOSBERG FIREWX INDEX
F15 SREF MINIMUM 2 METER RH
50
Combined Probability Charts
CAPE (J/kg) Green solid Percent Members gt 1000
J/kg Shading gt 50 Gold dashed Ensemble mean
(1000 J/kg) F036 Valid 21 UTC 28 May 2003

• Probability surface CAPE gt 1000 J/kg
• Generally low in this case
• Ensemble mean lt 1000 J/kg (no gold dashed line)

51
Combined Probability Charts
10 m 6 km Shear (kts) Green solid Percent
Members gt 30 kts Shading gt 50 Gold dashed
Ensemble mean (30 kts) F036 Valid 21 UTC 28 May
2003

• Probability deep layer shear gt 30 kts
• Strong mid level jet through Iowa

52
Combined Probability Charts
3 Hour Convective Precipitation gt 0.01
(in) Green solid Percent Members gt 0.01 in
Shading gt 50 Gold dashed Ensemble mean (0.01
in) F036 Valid 21 UTC 28 May 2003

• Convection likely WI/IL/IN
• Will the convection become severe?

53
Combined Probability Charts
Prob Cape gt 1000 X Prob Shear gt 30 kts
X Prob Conv Pcpn gt .01

F036 Valid 21 UTC 28 May 2003

• Combined probabilities very useful
• Quick way to determine juxtaposition of key
  parameters
• Not a true probability
• Not independent
• Different members contribute

54
Combined Probability Charts
Prob Cape gt 1000 X Prob Shear gt 30 kts
X Prob Conv Pcpn gt .01

F036 Valid 21 UTC 28 May 2003

• Combined probabilities a quick way to determine
  juxtaposition of key parameters
• Not a true probability
• Not independent
• Different members contribute
• Fosters an ingredients-based approach on-the-fly

Severe Reports RedTor BlueWind GreenHail
55
Combined Probability Charts
Combined or Joint Probabilities - Not a true
probability - An ingredients-based,
probabilistic approach - Useful for
identifying key areas
Ingredients for extreme fire weather conditions
over the Great Basin
F15 SREF PROBABILITY P12I x RH x WIND x TMPF (lt
.01 x lt 10 x gt 30 mph x gt 60 F)
56
Combined Probability Charts
Ingredients for extreme fire weather conditions
over the Great Basin
F15 SREF PROBABILITY TPCP x RH x WIND x TMPF (lt
.01 x lt 10 x gt 30 mph x gt 60 F)
57
Elevated Instability General ThunderNCEP SREF
30 Sept 2003 09 UTC F12
Mean MUCAPE/CIN (Sfc to 500 mb AGL)
Mean LPL (Sfc to 500 mb AGL)
58
Parcel Equilibrium Level NCEP SREF 30 Sept 2003
09 UTC F12
Mean Temp (degC) MUEquilLvl (Sfc to 500 mb AGL)
Prob Temp MUEquilLvl lt -20 degC (Sfc to 500 mb
AGL)
59
Lightning Verification
Gridded Lightning Strikes 18-21 UTC 30 Sept
2003 (40 km grid boxes)
60
Dendritic Growth Potential
NCEP SREF 7 Oct 2003 21 UTC F15 Probability
dendritic conditions (solid/shaded) Mean PMSL
(white solid), Mean 1000-500 mb dZ (dashed), Mean
10m Wind

• Find SREF members with
• Saturated layers gt 50 mb deep
• Temp range 11 to 17 degC
• Omega lt
• -3 microbar/s

61
Microphysical Example

• Probability cloud top temps gt -8 degC

Probability cloud top temps lt -12 degC
Ice Crystals Unlikely
Ice Crystals Likely
NCEP SREF 7 Oct 2003 21 UTC F15
62
Banded PrecipitationCombined Probabilities
NCEP SREF 7 Oct 2003 21 UTC F15

• Probability MPV lt .05 PVU (saturated 900 to 650
  mb layer) x Probability Deep Layer FG gt 1

63
Banded Precipitation
GOES 10 IR - 8 Oct 2003 1215 UTC
64
Identifying Rare Events (Low end example
Wind/Small Craft Advisory)

• 9 Oct 2003 09 UTC F63 fcst
• Prob sfc winds gt 30 mph
• (mean 10m wind vector shown)
• Difficult to forecast for every grid point

Saturday afternoon forecast (11 Oct)
65
Identifying Rare Events (Low end example
Wind/Small Craft Advisory)

• 9 Oct 2003 09 UTC F63 fcst
• Now consider an area /- 90 mi of a point (see
  Legg and Mylne 2004)

30 chance small craft advy winds over Monterey
Bay and offshore waters Saturday afternoon
66
Mode
Most Common Precip Type (Snow Blue) Mean
Precip (in) Mean 32o F Isotherm
F015 SREF Valid 00 UTC 21 December 2004
67
Probability Dendritic Layer gt 50 mb
F015 SREF Valid 00 UTC 21 December 2004
68
Probability of Banded PrecipitationPotential
Probability MPV lt .05 PVU (saturated 900 to 650
mb layer) x Probability Deep Layer FG gt 1
F015 SREF Valid 00 UTC 21 December 2004
69
Probability Omega lt -3 microbar/sMedian Depth
of Dendritic Layer
F015 SREF Valid 00 UTC 21 December 2004
70
Probability Omega lt -3 microbar/s
F015 SREF Valid 00 UTC 21 December 2004
71
Probability 6h Precip gt .25
F015 SREF Valid 00 UTC 21 December 2004
72
(No Transcript)
73

• Outline
• Motivation for ensemble forecasting
• Ensemble products and applications
• Emphasis on probabilistic products
• Ensemble calibration (verification)
• Decision making using ensembles

74
Combine Thunderstorm Ingredients into Single
Parameter

• Three first-order ingredients (readily available
  from NWP models)
• Lifting condensation level gt -10o C
• Sufficient CAPE in the 0o to -20o C layer
• Equilibrium level temperature lt -20o C
• Cloud Physics Thunder Parameter (CPTP)
• CPTP (-19oC Tel)(CAPE-20 K)
• K
• where K 100 Jkg-1 and CAPE-20 is MUCAPE in the
  
• 0o C to -20o C layer

75
Example CPTP One Member
18h Eta Forecast Valid 03 UTC 4 June 2003
Plan view chart showing where grid point
soundings support lightning (given a convective
updraft)
76
SREF Probability CPTP gt 1
3 hr valid period 21 UTC 31 Aug to 00 UTC 01
Sept 2004
15h Forecast Ending 00 UTC 01 Sept
2004 Uncalibrated probability Solid/Filled Mean
CPTP 1 (Thick dashed)
77
SREF Probability Precip gt .01
3 hr valid period 21 UTC 31 Aug to 00 UTC 01
Sept 2004
15h Forecast Ending 00 UTC 01 Sept
2004 Uncalibrated probability Solid/Filled Mean
precip 0.01 (Thick dashed)
78
Joint Probability (Assumed Independence)
P(CPTP gt 1) x P(Precip gt .01) 3 hr valid period
21 UTC 31 Aug to 00 UTC 01 Sept 2004
15h Forecast Ending 00 UTC 01 Sept
2004 Uncalibrated probability Solid/Filled
79
Uncalibrated Reliability (5 Aug to 5 Nov 2004)
Frequency 0, 5, , 100
Perfect Forecast
No Skill
Climatology
P(CPTP gt 1) x P(P03I gt .01)
80
Adjusting Probabilities

• Calibrate based on the observed frequency of
  occurrence
• Very useful, but may not provide information
  concerning rare or extreme (i.e., low
  probability) events
• Use statistical techniques to estimate
  probabilities in the tails of the distribution
  (e.g., Hamill and Colucci 1998 Stensrud and
  Yussouf 2003)

81
Ensemble Calibration

• Bin separately P(CPTP gt 1) and P(P03M gt 0.01)
  into 11 bins (0-5 5-15 85-95 95-100)
• Combine the two binned probabilistic forecasts
  into one of 121 possible combinations (0,0)
  (0,10) (100,100)
• Use NLDN CG data over the previous 60 days to
  calculate the frequency of occurrence of CG
  strikes for each of the 121 binned combinations
• Bin ensemble forecasts as described in steps 1
  and 2 and assign the observed CG frequency (step
  3) as the calibrated probability of a CG strike
• Calibration is performed for each forecast cycle
  (09 and 21 UTC) and each forecast hour domain is
  entire U.S. on 40 km grid

82
Before Calibration
83
Joint Probability (Assumed Independence)
P(CPTP gt 1) x P(Precip gt .01) 3 hr valid period
21 UTC 31 Aug to 00 UTC 01 Sept 2004
15h Forecast Ending 00 UTC 01 Sept
2004 Uncorrected probability Solid/Filled
84
After Calibration
85
Calibrated Ensemble Thunder Probability
3 hr valid period 21 UTC 31 Aug to 00 UTC 01
Sept 2004
15h Forecast Ending 00 UTC 01 Sept
2004 Calibrated probability Solid/Filled
86
Calibrated Ensemble Thunder Probability
3 hr valid period 21 UTC 31 Aug to 00 UTC 01
Sept 2004
15h Forecast Ending 00 UTC 01 Sept
2004 Calibrated probability Solid/Filled NLDN
CG Strikes (Yellow )
87
Calibrated Reliability (5 Aug to 5 Nov 2004)
Frequency 0, 5, , 100
Perfect Forecast
Perfect Forecast
No Skill
Climatology
No Skill
Calibrated Thunder Probability
88
Adjusting Probabilities

• Calibrate based on the observed frequency of
  occurrence
• Very useful, but may not provide information
  concerning extreme (i.e., low probability) events
• Use statistical techniques to estimate
  probabilities in the tails of the distribution
  (e.g., Hamill and Colucci 1998 Stensrud and
  Yussouf 2003)

89
Adjusting Probabilities

• Consider 2 meter temperature prediction from NCEP
  SREF
• Construct a rank histogram of the ensemble
  members (also called Talagrand diagram)
• Rank individual members from lowest to highest
• Find the verifying rank position of truth (RUC
  2 meter analysis temperature)
• Record the frequency with which truth falls in
  that position (for a 15 member ensemble there are
  16 rankings)

90
Adjusting ProbabilitiesUncorrected Talagrand
Diagram
2m temperature ending 27 December 2004
Under-dispersive
Truth is colder than all SREF members a
disproportionate amount of time
Clustering by model
Uniform Distribution
Warm bias in 15h fcst of 12 UTC NCEP SREF
91
Use 14-day bias to account for bias in forecast
Members 1 through 15 of NCEP SREF
92
Adjusting ProbabilitiesBias Adjusted Talagrand
Diagram
2m temperature ending 27 December 2004
Large bias eliminated but remains under-dispersive
Uniform Distribution
Near neutral bias in 15h fcst of 12 UTC NCEP SREF
93
Adjusting Probabilities

• Build the pdf by using observed data to fit a
  statistical distribution (Gamma, Gumbel, or
  Gaussian) to the tails
• This produces a calibrated pdf based on past
  performance
• Past performance does not guarantee future
  results.

94
Adjusting ProbabilitiesCorrected Talagrand
Diagram
2m temperature ending 27 December 2004
SREF probabilities now reflect expected occurrence
of event even in the tails
Uniform Distribution
Uniform distribution in 15h fcst of 12 Z SREF
95
Adjusted Temperature Fcst
Max temp (50) valid 12 UTC 5 Jan to 00 UTC 6 Jan
2004
96
Probabilistic Temperature ForecastNorman, OK
(95 Confidence)
Norman, OK Temp Forecast from SREF
Actual mins maxes indicated by red dots
2.5
50.0
Temp (degF)
2.5
Dec 27
Dec 28
Dec 29
Local Time ?
4 AM
6 PM
Mid
Mid
97
Probabilistic Meteogram
Probability of severe thunderstorm ingredients
OUN Runtime 09 UTC 21 April

• Information on how ingredients are evolving
• Viewing ingredients via probabilistic thinking

98
Probabilistic Meteogram
Probability of severe thunderstorm ingredients
OUN Runtime 09 UTC 21 April

• Information on how ingredients are evolving
• Viewing ingredients via probabilistic thinking

99

• Outline
• Motivation for ensemble forecasting
• Ensemble products and applications
• Emphasis on probabilistic products
• Ensemble calibration (verification)
• Decision making using ensembles

100
Decision Making

• Probabilities from an uncalibrated,
  under-dispersive ensemble system are still useful
  in quantifying uncertainty
• Trends in probabilities (dprog/dt) may indicate
  less spread among members as t ? 0

101
12h Prob Thunder
12h Prob Severe
Trend over 5 days from NCEP MREF (Valid 22 Dec
2004)
Day 6

• Increased probabilistic resolution as event
  approaches
• Run-to-run consistency
• Time-lagged members (weighted) add continuity to
  forecast

Day 5
Day 4
Day 3
Day 2
Prob Thunder
Prob Severe
102
Results
103
Decision Making

• Probabilities from an un-calibrated,
  under-dispersive ensemble system are still
  useful to quantify uncertainty
• Trends in probabilities (dprog/dt) may indicate
  less spread among members as t ? 0
• Decision theory can be used with or without
  separate calibration

104
Decision Theory Example

• Consider the calibrated thunderstorm forecasts
  presented earlier see Mylne (2002) for C/L
  model

Observed
User Electronics store Critical Event Lightning
strike/surge Cost to protect 300 Expense of a
Loss 10,000
a F.A. C.R. Miss F.A.
Hit C.R. C/L a 0.03
Forecast
If no forecast information is available, user
will always protect if a lt o, and never protect
if a gt o, where o is climatological frequency of
the event
105
Decision Theory Example

• If the calibration were perfect, then user would
  seek protective action whenever forecasted
  probability is gt a.

But, forecast is not perfectly reliable
106
Decision Theory Example

• Apply a cost-loss model to assist in the decision
  (prior calibration is unnecessary)
• Define a cost-loss model as in Murphy (1977)
  Legg and Mylne (2004) Mylne (2002)
• This can be done without probabilistic
  calibration as the technique implicitly
  calibrates based on past performance
• V Eclimate - Eforecast
• Eclimate Eperfect

107
Decision Theory Example
V Eclimate - Eforecast Eclimate -
Eperfect
V a general assessment of forecast
value relative to the perfect forecast (i.e.,
basically a skill score). V 1 indicates a
perfect forecast system (i.e., action is taken
only when necessary) V lt 0 indicates a system
of equal or lesser value than climatology
108
Decision Theory Example
Costs
Observed

• V Eclimate - Eforecast
• Eclimate - Eperfect
• Eclimate min (1-o)F.A. oHit
• (1-o)C.R. oMiss
• Eperfect oHit
• Eforecast hHit mMiss fF.A. rC.R.
• o climatological freq h m

Forecast
Performance
Observed
Forecast
109
Decision Theory Example
Maximum Potential Value of the Forecast and its
Associated Probability
1.0
Action probability for a .03 is 7 with V .64
10
Potential Value
0.5
Never Protect
Always Protect
0.0
0.01
0.10
.14
.008
0.001
1.00
a Cost/Loss Ratio (log scale)
110
Summary

• Ensembles provide information on mean, spread,
  and forecast uncertainty (probabilities)
• Derived products viewed in probability space have
  proven useful at the SPC
• Combined or joint probabilities very useful
• When necessary, ensembles can be calibrated to
  provide reliable estimates of probability and/or
  aid in decision making

111
SPC SREF Products on WEB

• http//www.spc.noaa.gov/exper/sref/

112
References

• Bright, D.R., M. Wandishin, R. Jewell, and S.
  Weiss, 2005 A physically based parameter for
  lightning prediction and its calibration in
  ensemble forecasts. Preprints, Conference on
  Meteor. Appl. of Lightning Data, AMS, San Diego,
  CA (CD-ROM 4.3)
• Cheung, K.K.W., 2001 A review of ensemble
  forecasting techniques with a focus on tropical
  cyclone forecasting. Meteor. Appl., 8, 315-332.
• Ebert, E.E., 2001 Ability of a poor man's
  ensemble to predict the probability and
  distribution of precipitation. Mon. Wea. Rev.,
  129, 2461-2480.
• Hamill, T.M. and S.J. Colucci, 1998 Evaluation
  of Eta-RSM ensemble probabilistic precipitation
  forecasts. Mon. Wea. Rev., 126, 711-724.
• Legg, T.P. and K.R. Mylne, 2004 Early warnings
  of severe weather from ensemble forecast
  information. Wea. Forecasting, 19, 891-906.
• Mylne, K.R. 2002 Decision-making from
  probability forecasts based on forecast value.
  Meteor. Appl., 9, 307-315.
• Sivillo, J.K. and J.E. Ahlquist, 1997 An
  ensemble forecasting primer. Wea. Forecasting,
  12, 809-818.
• Stensrud, D.J. and N. Yussouf, 2003 Short-range
  ensemble predictions of 2-m temperature and
  dewpoint temperature over New England. Mon. Wea.
  Rev., 131, 2510-2524.
• Wilks, D.S., 1995 Statistical Methods in the
  Atmospheric Sciences. International Geophysics
  Series, Vol. 59, Academic Press, 467 pp.