WDTB Winter Weather Workshop July 2003 Ensemble Forecasting

1
WDTB Winter Weather Workshop July 2003Ensemble
Forecasting

• The quick-reference/main-points version
• Stephen Jascourt Stephen.Jascourt_at_noaa.gov
• COMET NWP resource
• Some slides prepared by Jun Du (NCEP) and Bill
  Bua (COMET), and their material included
  contributions from Steve Tracton (formerly NCEP)
  and Zoltan Toth (NCEP)

2
What kinds of information can an ensemble add to
the forecast?
INFO on DISTRIBUTION of SCENARIOS - how many
scenarios - how likely is each - how
sharply defined is each
Increased accuracy!
Forecast skill 1humans, 2ensemble means,
3individual model runs
Ensembles identify high vs. low predictability

• Ensembles vs. one higher-resolution run
• even small timing and placement errors can be
  significant in attempt to accurately forecast
  details orographic local winds and precip highly
  sensitive to synoptic wind direction (see Mass,
  et al., 3/02 BAMS!!!).
• One detailed mesoscale model based forecast
  could allow the user to make highly specific
  and detailed inaccurate forecasts. (after Grumm)

3
Note ensembles run at lower resolution and
resolution decreases during the forecast period
GFS and GLOBAL ENSEMBLES
2002 until Oct 29 After Oct 29, 2002
GFS 00, 06, 12, 18 UTC MRF with late data
cutoff is gone, fields labeled MRF are same as
fields labeled AVN
T254 L64
T170 L42
T170 L42
T126 L28
T62 L28
2002 added hours
84h 180h 384h
3 ½ d 7 ½ d 16 d
84h 180h 384h
3 ½ d 7 ½ d 16 d
until mid-2003 Starting
sometime 2003
T126 L28
T126 L28
Global Ensembles
T62 L28
T62 L28
84h 180h 384h
3 ½ d 7 ½ d 16 d
84h 180h 384h
3 ½ d 7 ½ d 16 d
11 members (1 control, 10 perturbations) 11
members (1 cont., 10 pert) 00 UTC, 12
UTC 00 UTC, 06 UTC,
12 UTC, 18 UTC
4
Short-Range Ensemble Forecasts (SREF)

• NOW
• 5 Eta 48 km (control 2 perturbation pairs)
• 5 Regional Spectral Model 48 km (control 2
  perturbation pairs)
• RSM has very old AVN/MRF physics, not
  upgraded version
• 5 Eta members using Kain-Fritsch convective
  parameterization
• 21, 09 UTC to 63 hours, in time for your use
  with 00, 12 UTC Eta
• CHANGES PLANNED FOR FALL/WINTER 2003
• Resolution change to 32 km grid-spacing
• RSM physics upgraded to current GFS physics
• BUFR sounding and surface data available for all
  members
• 2 configurations being tested, best will be used
• Same as current
• Many different convective parameterizations
• RUC and ARPS might maybe perhaps get added to
  SREF

5
Types of Products, How to use
6
http//wwwt.emc.ncep.noaa.gov/mmb/SREF/SREF.html N
CEP Short range ensembles on the web. Note wwwt
address may change to www
7
Mean and Spread Interpretation
spread standard deviation. Highest lowest
is bigger than spread
8
3 day forecast from 00 UTC 11/2/01
Global
Depth uncertainty - how strong will trough be?
Phase uncertainty - where will the trough axis be?
SD - meters
9
Mean and Spread Advantages

• Compact communication
• Can see field over entire domain
• Ensemble mean on average has greater skill than
  any individual member
• Spread (sample standard deviation) quantifies the
  degree of uncertainty

10
Mean and Spread Limitations

• Assumes normal distribution of forecasts (bell
  curve with maximum likelihood at mean)
• Mean may hide important details
• Bi- or multi-modal solutions
• Timing problems in prediction of features
• COMMON PROBLEM cyclone/shortwave timing
  mismatches all members have wave/cyclone but
  shows up much weaker in ensemble mean (phase
  cancellation)
• Precipitation forecasts (particularly where
  convective precipitation is expected to be
  important see next slide)
• Can use spread as guide to where mean may not be
  communicating the correct information, and use
  additional tools to make further assessments

11
Example where ensemble mean is misleading
4 ensemble members each one has one precip bulls
eye of 1.0 but each predicts the bulls eye in a
different location
Ensemble mean has 4 precip bulls eyes, each of
0.25
12
Spaghetti Diagram Interpretation
Phase uncertainty
Amplitude uncertainty
13
Spaghetti Diagram Interpretation Clustering
Clustering
Ensemble mean
14
3-day forecast from 00 UTC 11/2/01, spaghetti
diagram for ensemble
global
Uncertain location of incoming western trough
Uncertain amplitude of eastern trough
From CDC web site http//www.cdc.noaa.gov/map/i
mages/ens/ens.html
15
Spaghetti Diagrams Advantages

• Avoids the assumption of normally distributed
  data and pitfalls thereof
• Can tell if ensemble mean, if present, is
  representative of the ensemble as a whole
• Shows spread among ensemble members and whether
  there is clustering of members around two or more
  forecasts
• Shows mode (I.e. most frequently occurring
  solution)
• Indicates outliers which may overly influence the
  ensemble mean and spread

16
Spaghetti Diagrams Limitations

• Limited to one or only a few contours
• Cannot see full field of interest over the full
  domain
• May not choose the right contour (use ensemble
  mean/spread to make the best choice)

17
PROBABILITY CHARTS percentage of
ensemble members with value exceeding threshold
SREF
Percentage of members with QPF gt .25/24h
010519/0000V63 SREFX-CMB 24HR PQPF OF .25
18
Probability charts Advantages

• Depicts probabilities for exceeding critical
  value in a compact manner
• Variable of interest is seen over the full domain
• Uses actual distribution of data from ensemble
  members to determine probabilities

19
Probability charts Limitations

• Do not get information on full PDF
• Only know percentage of ensemble members that
  exceed the value (sampling problem of limited
  ensemble size)
• Need to use several threshold values for complete
  picture
• Does not depict maximum value

20
Individual Station Plots MeteogramsEnsemble
mean and all members. Experimental, available by
fall 2003linked on http//www.emc.ncep.noaa.gov/m
mb/research/meso.products.html
21
Precip-type algorithmsensemble of ptype from
operational Eta. Available during fall/winter
linked on http//www.emc.ncep.noaa.gov/mmb/researc
h/meso.products.html
22
SREF
Storm tracks
Sequence provides info on envelope of storm tracks
010519/0000V63 SREFX-CMB SFC LOWS
23
Spread appears small.
Spread appears high.
But what happens when we scale it by recent
spread for same location and forecast length (132
hours in this example)?
24
.but is actually larger than usual for this
location/season
.but actually relatively low for this location /
season!
25
Calibrated output

• Bias-corrected probabilities
• match forecast probabilities to verification
  stats
• example if 0.5 precip verifies 30 of
  occasions when 50 of ensemble members QPF gt .5,
  then PQPF50 gives CPQPF30

Probability (PQPF) gt 0.5 in 24h period
Calibrated probability (CPQPF)
wet bias
bias removed lower probabilities
26
Relative Measure of Predictability (RMOP)measure
of how likely the ensemble mean is,distinguish
predictable vs. unpredictable situationshttp//ww
wt.emc.ncep.noaa.gov/gmb/ens/index.html (note
wwwt may become www)

• Based on last 30 days of ensemble performance to
  take into account regime predictability and
  general model performance
• Ensemble mean and each ensemble member placed in
  equally likely climatological bins (bins vary
  seasonally and geographically to account for
  typical variability)
• RMOP colors with percentage below color bar show
  the percentile rank of todays forecast compared
  to the last 30 days for number of ensemble
  members agreeing with their ensemble means
  (agreeing with in the same bin)
• For example, red (90) means the ensemble
  distribution has more members in the same bin as
  the mean than 90 of the cases in the past 30
  days, suggesting this is among the most
  predictable forecasts in the last month
• RMOP probability numbers (above the color bar)
• Calibrated probability that ensemble mean will
  verify based on how often the ensemble mean
  verified when the same number of ensemble members
  were in the bin containing the ensemble mean
  during the past 30 days

27
Block Relatively predictable
90 of the time in the last 30 days, fewer
ensemble members fell into the same bin as the
ensemble mean (I.e. relatively high
predictability as weve defined it here), BUT
only verified 43 of the time!
28
Unpredictable heights in strong gradient
Ridge/Trough Highly predictable
29
SREF Behavior Current Configuration

• RSM and Eta tend to group into separate clusters
  (especially for QPF), mostly because
• RSM based on GFS analysis, Eta based on Eta
• SREF Eta tends to have smaller spread than the
  RSM
• Eta vs. Eta with Kain-Fritsch convection
• Increased ensemble spread
• Could be result of using KF/mass-flux scheme or
  use of 4th order (less damping) diffusion scheme
• Sharper gradients, including in stability
  parameters
• Tendency for larger instability
• Consistent with later triggering of convective
  scheme than for BMJ
• But BMJ often has very high surface CAPE over
  warm water, apparently due to lack of vertical
  mixing

30

• 2 main forecast tips
• Ensembles cannot compensate for very large
  analysis errors when analysis errors are big,
  the verification may not lie within the envelope
  of ensemble solutions. 1 rule when using NWP is
  always check for problems in the analysis
• Ensemble mean and overall pattern of distribution
  of ensemble members is far more consistent
  run-to-run than any one individual model

31
Additional Ensemble Links
Note wwwt.emc.ncep.noaa.gov may change to
www.emc.ncep.noaa.gov http//eyewall.met.psu.edu/
mos/index.html - global ensemble
MOS http//wwwt.emc.ncep.noaa.gov/gmb/ens/trainin
g/.html CASES!!! http//wwwt.emc.ncep.noaa.gov/
gmb/ens/index.html - global ensemble
home http//wwwt.emc.ncep.noaa.gov/mmb/SREF/SREF.
html - SREF home http//eyewall.met.psu.edu/ensem
bles2/index.html - global anomaly http//eyewall.
met.psu.edu/SREF/index.html - SREF
anomaly http//www.meteo.psu.edu/gadomski/ewall.
html - maps http//www.hpc.ncep.noaa.gov/ensemble
training - Pete Manousos web page http//www.ecm
wf.int/newsevents/training/rcourse_notes/GENERAL_C
IRCULATION/CHAOS/Chaos.html
32
COMET NEW NWP PCU1 ensembles module by Bill Bua,
including cases, ready by fall! Ask about NCEP
model concerns as they arise! COMET
newsgroups http//www.meted.ucar.edu/nwp/newsgroup
s/index.htm Winter Weather Refresher
powerpoint http//www.meted.ucar.edu/comm_highere
d/winterpp.ppt (2002 version)
33
PRECIP TYPE follow-up to Dan Baumgardts
talk Info on the SNRA variable from the Eta
model, Comparison with the Baldwin algorithm and
example skew-T plots from the model with output
provided from both methods is shown in the middle
of the What can you expect from the Eta-12?
Visitview teletraining available online
at http//www.cira.colostate.edu/ramm/visit/eta12.
html There are 4 pages of interest beginning with
the one titled Precip type in model and output.
It can be viewed directly on the web or you can
download the visitview file and view it in
visitlocal mode