Addressing Model Errors in the UK Met Office

1
Addressing Model Errors in the UK Met Office

• Tim Hewson
• Chief Forecaster / Forecasting Research
• Met Office
• Exeter, England
• (at SUNY, Albany, NY until Feb 2005)

2
UK Met Office Forecasting Structure
3
Structure of Talk
How is the forecast constructed, and what
systematic errors need to be overcome?

• 1. Deputy Chief Medium Range
• Guidance components
• Rationale from raw to modified field
  modification
• Verification
• 2. Chief Short Range
• Guidance components
• Examples upgraded field modification tools
• Verification
• Discussion of Systematic Model Errors included..

4
Compiling Guidance
? ?
??
Recent observational data
?
Mesoscale model run(s)
? ?
Known model weaknesses
? ?
?
Short Range
Medium Range
( ? ? )
( ? )
Short Range Ensembles
?
Global model runs
? ?
? ?
Medium Range Ensembles
??
5
1. Medium Range Guidance

• Cover lead time of about 30 hrs to 10 days
• Rationale
• combine charts, text, tabular and graphical
  output to represent
• a most-probable consensus forecast
• error bounds on various aspects of that forecast

6
Principles of the Consensus Forecast

• This is a weighted, multi-model, mid-range
  forecast, in which gradients have been preserved
  (unlike an ensemble mean, in which gradients
  weaken as lead time increases)
• So how do we weight the different models?
• Can we account for known strengths and weaknesses
  of the different model formulations?

7
Deriving the Consensus forecast

• Two stages
• 1. Deciding what the forecast should look like
  (and if a particular run can be used unmodified)
• 2. Applying the relevant changes to a selected
  model run

• 1 - subjectively allow for
• Relative model accuracy
• Seasonal differences
• Regional differences
• Regime dependancy of errors (information
  lacking!)
• Forecast data times
• Ensembles (clustering etc)
• Systematic errors

8
Verification statistics suggest how to weight
different centres forecasts
NH rms MSLP error vs Lead Time
1 5
days 10
9
Seasonal differences (NH mslp, RMS at T72)

• EC Best throughout then UKMET, but NCEP
  consistently better in summer

10
Regional Performance Europe, vs Lead Time

• Europe-based models perform better in forecasting
  for Europe

11
Regional Performance N America, vs Lead Time

• Relative to performance over Europe UKMET does
  worse over US/Canada, GFS better

12

• Note also that plots can vary greatly for other
  parameters, and for other levels
• Need to know which parameters are most important
  for the task in hand?
• Much use is also made of ECMWF ensemble data

13
Examples

• Underlying Strategy
• First ascertain the key meteorological
  feature(s),
• Then incorporate the different model and ensemble
  solutions accordingly, weighting as appropriate
  (on screen or on paper) to arrive at a consensus
  solution

14
EG1 Frontal wave is main Feature for
UK Consensus would move GM wave (red) towards
NW, perhaps weakening it a little
VT 00z 24/7/02, mostly T48 GM, EC, FR, NCEP, DWD
15
EG2 Cold front is the main feature Consensus
would move GM front south, possibly hinting
at wave development, as GM underdoes waves, and
as NCEP shows one.
16
Cyclone Database - Snapshot
17
GM cyclone spectra for year 2000, categorised by
max wind speed within 300km radius of centre
North Atlantic Domain
18
Deriving the Consensus forecast

• Two stages
• 1. Deciding what the forecast should look like
  (and if a particular run can be used unmodified)
• 2. Modifying a selected model run

19
Modifying a Selected Model Run

• Use Field Modification tool (devised by Eddy
  Carroll) AFTER a model run has finished
• Allows quick, interactive, dynamically consistent
  changes to be made to a set of 3-d fields from
  one model run, eg
• Move low or front
• Deepen/fill low
• Introduce low or wave
• Relies on modification vectors applied to PV
  distribution, followed by PV inversion with
  changed boundary conditions
• Equivalent translation vectors applied to ppn and
  RH simple boundary layer model used for surface
  winds
• Temporal consistency achieved via time-linking
  (with decay parameter)
• Precipitation rate type, winds etc can also be
  adjusted directly and time-linked

See Carroll, Meteorological Applications, 1997
for field modification description
20
Field Modification Example moving a low with
slight deepening
21
Resulting fields (takes 2 seconds)
22
Initial Fields
23
500mb ht and 100-500mb Thickness - before
24
500mb ht and 100-500mb Thickness - after
25
Raw Model Forecast
26
Modified Forecast
27

• Time-linking of changes enables a
  meteorologically and dynamically consistent chart
  sequence representing the most probable
  (consensus) forecast evolution to be produced

28
Objective Verification
29

• Subjective verification shows similar results to
  mslp verification (though needs an overhaul!)
• Objective verification of other parameters higher
  up in atmosphere (eg 250 and 500mb ht) shows
  minimal difference between mod and unmod just a
  slight improvement at longer leads
• PV inversion within field modification is thus
  not adversely influencing levels remote in height
  from the lower levels that are generally targetted

30
2. Short Range Guidance

• Cover lead time up to about 30 hrs
• Rationale
• combine charts, text, tabular and graphical
  output to represent
• a most-probable consensus forecast
• Output is mainly graphical 3 hourly frames
  depicting mslp, rainfall rate, ppn type
  (convective/dynamic rain/snow), cloud cover
• Special emphasis on hazardous / severe weather
• Warning issue as required
• Methodology
• Blend output from limited number of models (eg 3)
  with inferences from observations imagery, and
  also adjust according to known model weaknesses
  (highest weight usually for UK mesoscale (12km)
  model).

31
Short Range Forecast Example
32
(No Transcript)
33
B
B
A
A
OBSERVED
FORECAST

• Northerly Winter Marine Convection example
• A precipitation too extensive, dynamic rather
  than convective
• B precipitation rate variation too small, more
  inland penetration
• 12km model, 38 levels, parametrised convection
  (Gregory-Rowntree)
• convective life-cycle is zero

A
B
34
Impact of higher resolution
4km run
Radar
12km run
35
Impact of higher resolution

• 4km resolution runs have
• Convection scheme used for low CAPE, switched off
  for high CAPE (assumed resolved explicitly)
• This can give
• more inland penetration of showers ?
• greater rate variations ?
• local rates that are too extreme ?
• 4km resolution represents a transition
• 1km should prove better. Introduce operationally
  in 5-10 years?
• much testing / development still required -
  ongoing

36
Boscastle Floods August 16th 2004
12km RUN
37
Snow Example - 30/1/2003 12 hour forecasts
38
Harpenden, N of London 31 January 2003 (c/o John
Davies)

• Persistence of heavier convective component
  (especially inland) can lead to extra cooling,
  and missed snow

39
Severe Weather Verification

• From the perspective of hazardous weather, were
  there serious errors in either the modified or
  raw model forecasts?

• Fewer errors in Modified (blue) than in raw
  (red)
• Warm air (summer) convection is the most
  difficult to deal with
• Forecasters biggest contribution is in cold air
  convection

40
Verificationresults approx 1 year

• As regards giving an appropriate graphical
  impression of the weather experienced, which
  forecast was better modified, raw model or
  neither?

Modified was
41
Snow Forecasts Objective Verification
Hit Rate (POD)
False Alarm Ratio
Modified
Raw
Uses SYNOP observations
42
Lead Time Gain
Can be used to put together composite
modification indices, as a measure of
forecaster contribution, which in turn can be
compared with modification time
½ (ab) (at time T1)
43
(No Transcript)
44
Orographic precipitation

• Smoothed orography (in new model New
  Dynamics) reduces upslope rainfall, and
  similarly reduces the rain shadow
• Older model better (even if for the wrong
  reason!)
• Magnitude of impact is proportional to flow
  strength
• Important for QPF

45
Severe windstorms
38 Levels (operational)

• High resolution required (90 levels?) to model
  sting jet
• Mslp may be OK but winds not

90 Levels
Greater strength along downward trajectory
c/o Pete Clark JCMM, Reading
46
Summary (1)

• Met Office guidance centre forecasters have the
  facility to modify numerical model output fields
  at all lead times
• In the short range, small changes to model fields
  are fairly common, and are usually based on
  current trends and knowledge of model systematic
  errors
• Changes usually relate to precipitation
  distribution, intensity and type, and cloud cover
• The forecaster is 4 times more likely to improve
  the forecast than make it worse
• Cold air convection, including snow forecasts,
  shows biggest forecaster contribution
• Summer (warm air) convection is the most
  difficult to improve upon
• In the medium range issuing unmodified output is
  more common, especially at T36 and 48, though
  any changes that are made to mslp and other
  fields can be substantial. The basis for changes
  is usually model (including ensemble)
  discrepancies.
• According to standard rms error statistics, there
  are improvements in both 700mb RH and mslp
  fields, though RH seems to show more added
  value

47
Summary (2)

• Compositing together various verification
  measures into a single modification index,
  utilising the concept of lead time gain,
  indicates
• A reduction in forecaster contribution in the
  short range, between T6 and T30, as current
  trends become less relevant
• In the medium range an increase in forecaster
  contribution with lead time. This is partly
  logistical because longer lead-time forecasts
  are generally issued last, implying that the
  number of available model runs increases for
  longer leads
• DESPITE high quality models, high quality
  forecasters (!) and high quality observational
  data, some severe weather events are still poorly
  forecast, even at very short range
• Regular dialogue between forecasters and model
  developers, backed up by verification results
  (particularly those relating to systematic
  errors) enables model development to focus on
  practical forecasting problems

48
Summary (3)

• Model Systematic Errors include
• 20 shortfall in number of modest frontal waves
  forecast by T48
• Orographic precipitation and rain shadow
  underdone with smoothed orography
• Insufficient inland penetration of showers in
  marine convection
• Insufficient rate variation in marine convection
• Insufficient cooling of lower troposphere by
  (unrepresented) heavier convection, implying
  severe errors in snow forecasts
• Inadequate representation of sting jet
  phenomenon, which leads to underprediction of
  severe windstorms
• Answers include
• Increased horizontal resolution
• Increased vertical resolution
• BUT increased resolution brings also its own
  problems CARE REQUIRED!

49
(No Transcript)
50
Types of Error corrected / introduced

• What was it, specifically, about the good
  forecast that made it better?"

51
(No Transcript)
52
(No Transcript)