Verification of Dichotomous Lightning Forecasts at the European Storm Forecast Experiment (ESTOFEX)

1
Verification of Dichotomous Lightning Forecasts
at the European Storm Forecast Experiment
(ESTOFEX)

• Pieter Groenemeijer
• Oscar van der Velde
• Helge Tuschy
• Christoph Gatzen
• Johannes Dahl
• Nicholas Verge

Institute for Meteorology and Climate Research,
Research Centre / University Karlsruhe European
Severe Storms Laboratory Laboratoire d'Aérologie,
Université Paul Sabatier, Toulouse Institute for
Atmospheric Physics, German Aerospace Center (DLR)
2
Overview

• About ESTOFEX
• ESTOFEX forecasts
• Verification of lightning forecasts
• Verification of threat level forecasts

3
about ESTOFEX

• the name of a web-based platform for the exchange
  of knowledge about convective storm forecasting
• the name of a the forecast experiment that is
  carried out by some of its active members
• goal of the experiment is...
• to find out how convective storms can best be
  forecast in Europe

4
about ESTOFEX

• ESTOFEX issues convective forecasts for the
  next day
• These forecasts consist of two parts
• forecast discussion
• forecast map

Storm ForecastValid Sat 02 Jun 2007 0600 to
Sun 03 Jun 2007 0600 UTCIssued Sat 02 Jun 2007
0922Forecaster VAN DER VELDE SYNOPSISAn
mid/upper cut-off low dwells slowly southward
with its center just west of Italy. The
associated surface pressure field weakens during
the period. High pressure...
5
ESTOFEX is not...

• a new weather service in Europe
• (no such centre without an initiative from
  NMSs)
• funded
• the start of a commercial company

6
ESTOFEX forecasts

• The forecast maps show lines that represent

thunder area level 1 level 2 level 3
level 1, level 2, level 3 are threat levels for
convective wind gusts, tornadoes and large hail
7
ESTOFEX forecasts

• After the forecast has
• been issued, verification
• data appears on the
• map.

Lightning strikes source UK. Met
Office arrival-time-difference system
Hail reports Tornado reports Wind gust
reports Heavy rain reports (source ESSL)
8
Verification of lightning forecasts

• Predictand
• the occurrence of lightning within 40 km of a
  point
• Dichotomous forecast

one or more strike detected
event occurred? event occurred?
yes no
event forecast? yes hit false alarm
event forecast? no miss correct negative
9
Forecast lightning (1 Sep. 2006 31 Aug. 2007)

• number of forecast periods
• (of 346 forecasts) in which
• lightning was forecast to occur
• within 40 km of each point

10
Detected lightning (1 Sep. 2006 31 Aug. 2007)

• number of forecast periods
• (of 346 forecasts) in which
• lightning occurred within
• 40 km of a point
• More lightning is forecast than
• occurs in reality.
• The forecasts have a positive bias.

11
Forecast skill indicators

• False Alarm Rate
• false alarms / (hits false alarms)
• Probability Of Detection
• hits / (hits misses)
• Heidke Skill Score

12
Probability Of Detection(1 Sep. 2006 31 Aug.
2007)
POD hits / (hits misses)

• High POD over...
• the western and
• central European continent
• the central and eastern
• Mediterranean Sea
• Low POD over...
• the Atlantic Ocean
• Scandinavia
• Eastern Europe
• Northern Africa

13
False Alarm Rate(1 Sep. 2006 31 Aug. 2007)
FAR false / (hits false)

• Low FAR over...
• central and eastern parts of
• the European continent
• High FAR over...
• western and northern Europe
• parts of the Mediterranean
• the Black Sea

14
Heidke Skill Score(1 Sep. 2006 31 Aug. 2007)
HSS 0 means that the forecast is no better than
chance HSS 1 means that the forecast is
perfect HSS assumes that a miss is just as
bad as a false alarm. HSS indicates the
forecasts are best over the central and eastern
European continent and the southern
Mediterranean Forecasts are bad over western and
northern Europe
15
Temporal evolution of forecast skill
16
Temporal changes in performance
17
Individual forecaster performance2006 and 2007
summer periods compared
Probability of Detection
False Alarm Ratio
18
Individual forecaster performance2006 and 2007
summer periods compared
Probability of Detection
False Alarm Ratio
19
Future plans

• Polychotomous lightning forecasts

event occurred ? event occurred ?
Yes No
event forecast? gt 60
event forecast? 30-60
event forecast? 15-30
event forecast? 5 -15
event forecast? lt 5
20
Future plans

• switch to probabilistic forecasting of lightning
• start rigorous verification of severe weather
• (hail, tornadoes, severe winds)
• switch to probabilistic forecasting of severe
  weather
• test the skill of model-derived parameters for
  both lightning and hail, tornadoes winds and
  compare with human forecasters (interesting!)

21
Final remarks

• a discussion of individual storm events is
  presented on the poster lead-authored by Helge
  Tuschy and in Christoph Gatzens talk
• all verification data is available through the
  forecast archive at www.estofex.org
• I am happy to answer your questions on ESTOFEX
  and the verification of its forecasts