Ensemble Predictions: Understanding Uncertainties

1
Ensemble Predictions Understanding
Uncertainties
Lars LandbergWind Energy Department Risø
National Laboratory DTUDenmark
Gregor Giebel, Jake Badger, Risø National
Laboratory DTUHenrik Aalborg Nielsen, Torben
Skov Nielsen, Henrik Madsen, Pierre Pinson, IMM
DTUKai Sattler, Henrik Feddersen, Henrik Vedel,
Danish Meteorological Institute
2
Contents

• Where do we come from?
• Two new things
• Ensemble forecasts
• More than one NWP
• Conclusions

3
About forecasts

• Good to know the expected production
• Even better to know the uncertainty

NWP
Output Production Uncertainty
Model
Obs
4
In the old days (late 90ties!)
5
The idea of ensembles / Spaghetti plot

• An ensemble of multiple forecasts, done from
  different initial conditions, or different
  numerical models / model runs, should give a
  measure of forecast uncertainty
• Assumption There is a connection between spread
  and skill

6
Ensemble predictions

• Ensembles try to catch more of the variety of the
  weather
• Proper ensembles deliver plume of possible
  futures
• Multi-model ensembles can increase accuracy
• Connection between spread skill?
• Might be better use of computer resources

Source NCEP/NCAR
7
2 proper ensembles

• ECMWF51 members, up to 10 days ahead, global
  domain, made by adding singular vectors
• NCEP/NCAR12 members, up to 84 hours ahead,
  global domain, made through bred modes
• resolution (both) 80 km
• Variables wind (speed and dir) _at_10m

8
PSO-Ensemble project

• 3-year Danish national project (now finished)
• Uses ECMWF and NCEP ensembles
• Risø, IMM, DMI Danish utilities
• Ensembles same meteo model, many slightly
  different runs -gt finds many probable futures -gt
  uncertainty bands
• Important result The quantiles as coming from
  the ensembles are not directly applicable as
  power quantiles!
• Demo ran 1 year (using ECMWF) and counting, used
  for
• trading over weekend,
• weekly fuel demand forecasts and for
• maintenance / power plant repair scheduling

9
Visualisation All members

• Shows all the 51 ECMWF members without
  transformation
• The black line is the control run (the best guess
  of ECMWF)

10
Visualisation most quantiles

• Showing most derived and transformed quantiles
  between 5 and 95
• Too much spread to be useful at a glance (outer
  quantiles are doubtful)

11
Visualisation only 25 and 75

• Only central quantiles

12
PSO-Ensemble, unadjusted quantiles!!
13
PSO-Ensemble, adjusted quantiles
14
How to use a quantile forecast

• Informal
• Is the forecast uncertain or not?
• Can we be sure to have more than 50 of
  installed capacity?
• If we need to take out a conventional plant for
  revision within the next week when should we do
  that?
• Formal
• Given up- and down-regulation costs the quantile
  cup/(cup cdown) should be used as the bid on
  the spot market.
• If we have many quantiles (the full p.d.f.) the
  optimal bid can be derived from any cost
  function.

15
Are the quantiles reliable?

• E.g. is the actual production below the 25
  quantile in 25 of the cases?
• Can be checked by grouping the data (here by
  horizon).

16
Accuracy of quantiles
Deviations up to 5, but often less. Some
curvature can presumably be removed by tuning of
the model.
17

• More than one NWP forecast

18
Doubling the number of NWP

• Used DMI and DWD for six test cases in Denmark
• Result the combination of inputs is better

G. Giebel, A. Boone A Comparison of DMI-Hirlam
and DWD-Lokalmodell for Short-Term Forecasting.
Poster on the EWEC, London, Nov 2004
19
Conclusions

• Ensemble predictions can not be used directly as
  a measure of the error
• Errors are modelled much better if ensemble
  predictions are used
• Two NWP forecasts improve the predictions by more
  than 1 m/s!
• In general we are getting much better af
  predicting the power output