Observation Targeting

1
Observation Targeting

• Andy Lawrence
• Predictability and Diagnostics Section, ECMWF
• Acknowledgements
• Martin Leutbecher, Carla Cardinali,
• Alexis Doerenbecher, Roberto Buizza

2
Contents

• Introduction
• What is Observation Targeting?
• Targeting Methodology
• Example using a simple model
• Kalman Filter techniques
• Singular Vector techniques
• Summary of research issues
• Operational targeting principles
• Previous targeting campaigns
• Operational structure and schedules
• Results from ATReC 2003
• Verification of forecast impacts
• Future of observation targeting

3
What is observation targeting?

• Techniques that optimize a flexible component of
  the observing network on a day-to day basis with
  the aim to achieve specific forecast
  improvements.

4
The concept of observation targeting

• Forecast an event.

Use adjoint model transform algorithm to derive
data-sensitive areas.
Add extra observations
OBS
OBS
OBS
Improve forecast?
Verify with corresponding analysis.
5
The concept of observation targeting

• Q If we have the capability to add observations
  in data-sensitive areas to improve the forecast
  of a specific event, can these locations be
  determined using objective (model-based) methods?

OBS
OBS
OBS

• A This is an optimization problem with two
  constraints
• Probability of making an analysis error at a
  particular location
• The intrinsic ability of the flow at that
  location (i.e. sensitivity)

6
Where are observations needed to improve a
18-hour forecast?

• Sensitive areas Verification region

7
Where are observations needed to improve a
42-hour forecast?
8
Where are observations needed to improve a
66-hour forecast?
9
Methodology

• The Observation Targeting Question
• How do identify optimal sites for additional
  observations?

OR How do we predict changes of forecast
uncertainty due to assimilation of additional
observations?
10
Methodology

• Information needed to answer this question
• Knowledge of the statistics of initial condition
  errorsand how they change due to an assimilation
  of additional observations.
• Gaussian error statistics
• Kalman filter techniques

• Knowledge of the perturbation dynamics from the
  observation time to the forecast verification
  time
• NWP studies suggest that perturbation dynamics
  are approximated by a linear propagator defined
  by ensemble-based techniques or tangent-linear/
  adjoint techniques.

For linear perturbation dynamics and Gaussian
error statistics, optimal state estimation can be
approximated by the Extended Kalman Filter which
can also be used to select optimal sites for
additional observations.
11
Extended Kalman Filter

• Targeted observations in the framework of an
  Extended Kalman Filter
• (Illustration using the Lorenz-95 system -
  Leutbecher, 2003)

• A chaotic system, where a time unit of 1
  represents 5 days
• dxi
• dt -xi-2 xi-1 xi-1 xi1 xi F
• with i 1,2,40
• x0 x40 , x1 x39 , x41 x1 and F 8

12
Planet L95 routine observing network

• For routine observations
• Observations are constructed by adding noise
  (representing unbiased and uncorrelated normally
  distributed errors) to values taken from a
  truth run..become available every 6 hours.
• Over land (positions 21-40) we have observations
  at all locations ?0 0.05?clim
• Over ocean (positions 1-20) we have observations
  at cloud-free locations ?0 0.15?clim

13
Planet L95 forecast errors

• 2-day forecast errors for Europe

14
Planet L95 targeted observation

• A single observation over the ocean, with the
  error characteristics of a land observations is
  considered (i.e. at i1,20).
• The aim of this is to provide a better forecast
  over Europe on Planet L95.

15
Covariance prediction with the Kalman filter
(routine additional observations)

• Covariance evolution within the Kalman filter
• For Routine observations
• Analysis Step at time tj (Pra)-1 (Prf)-1 HrT
  Rr-1 Hr
• Forecast Step tj ? tj ? Prf M Pra MT

For Routine additional observations at position
i (where i 1,20) Analysis Step at time
tj (Pia)-1 (Prf )-1 HrT Rr-1 Hr HiT Ri-1 Hi
Forecast Step tj ? tj ? Pif M Pia MT
Optimal position i (where i gives the maximum
reduction of forecast error variance)
maxi1..20 trace ( LEu (Prf - Pif ) LTEu
)
16
Optimal position for an additional observation

• Distribution of optimal position for an
  additional observation in L95 Atlantic
• (to improve the 2-day forecast over Europe)

17
Planet L95 Forecast impacts

• How to measure the improvements in forecast
  skill
• Compare with randomly placed observations
• Compare with impact obtained with observations
  that actually reduce the forecast error the most
  ( although never achievable as it requires
  information at verification time position
  depends on realization of actual errors!)
• Compare with the impact of an observation added
  at a fixed site optimized for the forecast goal
  (a hard test).
• No such test with NWP system and real
  observations yet.
• How does last test look with L95?

18
Fixed location v. adaptive day-to-day location
19
Approximations of the Kalman Filter

• Method is good for a simple model, BUT
• an extended Kalman filter is too expensive for a
  full NWP model
• Targeting would require to run the Kalman filter
  several times
• (i.e. each configuration of the additional
  observations considered in the
• planning process is one among several feasible
  ones).

• Solution
• Calculate forecast error variance in small
  relevant subspace!
• Perturbations of ensemble members about the mean
  a proxy for data assimilation scheme (ETKF).
• Based on singular vector schemes computed with an
  estimate of the inverse of the routine analysis
  error covariance metric as the initial time
  metric.

20
Ensemble Transform Kalman Filter (ETKF)

• Used in targeting for Winter Storms
  Reconnaissance Program (WSRP) (Bishop,
  Etherton and Majumdar, 2001)
• Assumes linearity.
• linear combination of ensemble perturbations
  ensemble mean model trajectory.
• Assumes optimal data assimilation.
• No covariance localisation.

• Reduction of forecast error variance using the
  ETKF (Majumdar, 2001)
• Signal forecast error (routine additional)
  forecast error (routine)

21
Singular Vector Schemes
Singular vectors identify the directions (in
phase space) that provide maximum growth over a
finite period of time.

• Dependent on model characteristics and
  optimization time
• Growth is measured by the inner product (or
  metric or norm)
• If the correct norm is used, the resulting
  ensemble captures the largest amount of forecast
  error variance at optimization time (assuming
  that the forecast error evolves linearly).
• For targeting Forecast error variance prediction
  from t0 to tv replaced by variance
  predictions in a singular vector subspace.
• Data assimilation can either use a full Kalman
  filter or Optimal Interpolation.

22
Singular vector targeting method 1

• Singular Vector-based reduced-rank estimate
• Initial time metric is the inverse of the routine
  analysis error covariance matrix (Pra)-1
• SVs computed with this metric evolve into the
  leading eigenvectors of the routine error
  covariance matrix
• trace (Leu Pf LTEu ) .

• Compute variance of forecast errors only in a
  subspace of leading singular vectors
• trace (Ân LEu Pf LTEu ÂnT) instead of trace (LEu
  Pf LTEu )
• Here, Ân denotes the projection on the subspace
  of the leading n (left) singular vectors of LEuM.

• Data assimilation uses full Kalman filter

23
Reduced-Rank approximation of covariance forecast
step

• Analysis error covariances in the subspace
  spanned by the leading n SVs are represented by
• Routine network Vn VnT where VnT ( Pra ) -1
  Vn I
• Modified network Vn ?i ?iT VnT where (Vn ?i )T
  ( Pra ) -1 (Vn ?i) I

The transformation matrix ?i is the inverse
square root of the n x n matrix,Ci, that
expresses the modified analysis error covariance
metric in the basis of the singular vectors Ci
VnT ( Pra ) -1 Vn In VnT HiT R 1 Hi Vn
.
Using these representations (of the aecm), the
forecast error variance in the verification
region becomes trace (Ân LEu ( Pfj? ) LTEu ÂnT
) ?nj1?2j routine network trace
(?T diag (?21 ?2n ) ? ) modified network where
?j denotes the singular value of the j-th SV vj
24
Singular vector targeting method 2

• Approximate full Kalman filter by replacing the
  routine forecast error covariance matrix Pfr by a
  static background error covariance matrix B
  (Optimal Interpolation scheme)
• In the variance prediction (for
  targeting) and
• In the assimilation algorithm
• Analysis error covariance matrix given by
• A-1 B-1 HTR-1H

In L95 system, static background error covariance
matrix B ? (xf - xt ) (xf - xt)T ? is a
sample covariance matrix computed from (forecast
truth) differences from a 1000 day sample ?
combine with reduced-rank technique
25
L95 comparisons SV reduced-rank schemes

• Method 1 Method 2
• Full KF- SV subspace Optimal Interpolation
  SV subspace

26
L95 comparisons SV full rank v reduced-rank
schemes

• Distribution of 2-day forecast errors over Europe
• Full KF v. Method 1 ( Reduced-rank Kalman
  filter)
• Full KF v. Method 2 ( Reduced-rank OI)

27
SV dependency on initial time metric

• Targeted observations should be directed to
  sensitive regions of the atmosphere.

• Correct metric is dependent on the purpose for
  making the targeted observations (study precursor
  developments or to improve forecast initial
  conditions).

28
Hessian Singular Vectors

• The Hessian of the cost-function provides the
  estimate of the inverse of the analysis error
  covariance matrix J(x) Jb (x) Jo
  (x)
• ??J B-1 HTR-1H A-1

• Initial error estimates are consistent with the
  covariance estimates of the variational data
  assimilation scheme (incorporates error
  statistics).

29
Total energy SV v Hessian SV
TESV Full Hessian SV
Partial Hessian SV
??J Jb Jo ??J Jb
30
Hessian reduced-rank estimate

• Similar to Kalman Filter/ OI- reduced rank
  estimate but is based on a subspace of Hessian
  Singular vectors vi computed with the metric
  ??Jroutine (using only observations from the
  routine network in Jo) (Leutbecher, 2003)

• Efficient computation of the Hessian metric for
  modified observation network (routine
  additional) in the subspace
• Cij vTi ??Jmod vj vTi (??Jroutine HTa
  R-1a Ha) vj
• ?ij (Ha vi)T R-1a Ha vj

• Estimate of forecast error variance reduction due
  to additional observations
• trace (I C-1 diag (?21. ?2n))
• where ?j denotes the singular value of the
  routine Hessian SV vj

31
Comparison of flight track ranking

• Winter Storms Reconnaissance
• Program 2003
• Additional observations on 4th Feb 00UT
• for forecast verification time 6thFeb 00UT
• Hessian Reduced-rank estimate ETKF

32
Summary Research Issues
Aim of observation targeting Prediction of
forecast error variance due to modifications of
observing network.

• Factors that affect skill of forecast error
  variance predictions in operational NWP
• Covariance estimates
• Skill of forecast error variance predictions
  depends on quality of background error covariance
  estimateincorporate flow-dependant wavelet
  approach.
• Account for correlations between observation
  error in current schemes (Important for satellite
  data with observation error correlations in space
  and between channels ? optimal thinning - Liu
  Rabier, 2003)
• Predict spatial resolution of routine
  observations. Harder to do with day-to-day
  variability of targeted observations,
  particularly for satellite data affected by cloud.

33
Summary Research Issues

• Error dynamics
• TL/AD model simplified due to resolution and
  physical process parameterisation. Advances in
  formulation (moist processes, sensitivity of
  observation targeting guidance to spatial
  resolution) should improve variance predictions.
• Validity of tangent linear assumption
• Gilmour et al. 2001 probably not useful beyond
  24h but measure of nonlinearity dominated by
  small scales
• Reynolds Rosmond 2003 SVs usefully up to 72h
  (diagnostic in SV-space and scale dependant
  diagnostic)

• Reduced Rank SVs (Subspace) How many SVs are
  needed to reliably predict forecast error
  variance reductions?
• L95 1 SV is sufficient the leading SV explains
  a large fraction of the total forecast error
  variance in the verification region (Rank 1KF ?
  full KF)
• NWP For rank (LEuM) ? number of grid-points in
  verification region times number of variables.

34
Summary Research Issues

• Contribution of model error?
• To initial condition error at t0
• To growth of error from t0 to tv

• Targeting methodology
• Perhaps combination of SV and ensemble-based
  approach? (expensive, as requires a dedicated
  ensemble).
• Does validity of linear transformation in ETKF
  technique extend further than the validity of the
  TL-approximation?

• Observation types
• Forecast error variance reductions can be
  determined for different observation types in
  sensitive areas.
• Will the abundance of satellite data eliminate
  the need for in-situ measurements?
• Satellite sampling is limited through cloud
  layers ? in-situ measurements useful if
  dynamically-sensitive areas are beneath clouds.

35
Previous targeting campaigns

• Targeted observation techniques and methods were
  tested during numerous
• operational campaigns
• FASTEX (Fronts and Atlantic Storm-Track
  Experiment 1997)
• Improving forecasting of atmospheric cyclone
  depressions forming in the North-Atlantic and
  reaching the west-coast of Europe.
• NORPEX (North Pacific Experiment 1998)
• North Pacific winter-season storms that affect
  the United States, Canada, and Mexico.
• WSRP (Winter Storms Reconnaissance Program
  1999-2006)
• North-Eastern Pacific storms affecting the
  west-coast of the United States.
• Use of targeted observations has a positive
  effect on forecast skill (Majumdar et al. 2001)
• ATReC (Atlantic THORPEX Regional Campaign 2003)
• North-Atlantic storms affecting east-coast
  United States and Europe.

36
Operational Structure for Observation Targeting
1 2 3 4
5 tc td t0 tv
time
37
Operational Structure for Observation Targeting
1 2 3 4
5 tc td t0 tv
time
2. Sensitive area prediction Compute which
configuration for adaptive observations (to be
taken at t0) is likely to best constrain error
of forecast for (tv,R).
Operations Centre
38
Operational Structure for Observation Targeting
1 2 3 4
5 tc td t0 tv
time
3. Select and request additional observations at
td.
39
Operational Structure for Observation Targeting
1 2 3 4
5 tc td t0 tv
time
4. Observing platforms deployed at t0 and
observations taken.
Operations Centre
Data Monitoring
Observation control centre
AMDAR
Radiosonde
Res. Aircraft
ASAP
40
Operational Structure for Observation Targeting
1 2 3 4
5 tc td t0 tv
time
5. Forecast verification time at tv
Operations Centre
Operations Centre
Data Monitoring
Observation control centre
AMDAR
Radiosonde
Res. Aircraft
ASAP
41
Atlantic THORPEX Regional Campaign 2003

• First field campaign in which multiple observing
  systems were used.
• Dropsondes from research aircraft, ASAP ships,
  AMDAR, land radiosonde sites.
• Observation targeting guidance to predict the
  sensitive areas based on
• UKMO ETKF based on ECWMF ensemble
• Meteo-France total energy SVs run on a
  (possibly perturbed) trajectory
• NRL SVs and sensitivity to observations
• NCEP ETKF based on combined ECWMF and NCEP
  ensembles.

• ECWMF 2 flavours of total energy SVs and
  Hessian SVs
• Config. initial norm TLM res. TLM physics
• TE-d42 Total Energy T42 dry
• TE-m95 Total Energy TL95 moist
• H-d42 Hessian T42 dry

42
ATReC_029_1 Obs. time 20031208, 18UT Ver.
time 20031211, 00 (54h opt)

• Dry TESV (ECMWF) Moist TESV (ECMWF)
• Hessian SV (ECMWF) ETKF (Met Office using
  ECMWF ensemble)

43
Sensitivity area prediction
How do we determine where to send the
observations? In ATReC 2003, level of agreement
between the sensitive area predictions can be
quantified in terms of geographical
overlap. Sensitive area 1 Sj of area
a Sensitive area 2 Sk of area a Geographical
overlap Ojk area ( Sj ? Sk ) /a

• ECMWF calculated overlap ratios for a 4 x 106
  km2 (Leutbecher et al. 2004)
• SAP1 SAP2 Number of cases with overlap gt0.50
• SV TE-d42 SV TE-m95 42/43 98
• SV TE-d42 SV H-d42 35/42 83
• SV TE-d42 ETKF 31/67 46
• SV H-d42 ETKF 32/67 48

44
TESV vs Hessian SV vs ETKF
ETKF sensitivity across Atlantic, main area
40-55N,20-30W secondary max 45N60W (this is also
secondary area for Hessian SVs). Main HSV and dry
TESV centre is South tip of Greenland (and down
to 50N), and back into Canada at 60N. Moist TESVs
have significant area 35N65W (Odette)
45
Case 43 Obs. time 8th Dec 18UT, Verif. time 11th
Dec 00UT

Radiosonde, Satellite rapid-scan winds, AMDAR
flights
46
Case 47 Obs time 11th Dec 18UT, Verif. Time 13th
Dec 12UT
Radiosondes, ASAP, Satellite, AMDAR
47
Case 36 Obs time 4th Dec 18UT, Verif. Time 6th
Dec 12UT

• Radiosonde, ASAP, AMDAR

48
Case 37 Obs. Time 5th Dec 18UT, Verif. Time 7th
Dec 12UT

• Radiosonde, ASAP, Satellite, AMDAR and Dropsondes

49
East Coast USA storm 5-7th December 2003
A major winter storm impacted parts of the
Mid-Atlantic and Northeast United States during
the 5th-7th. Snowfall accumulations of one to two
feet were common across areas of Pennsylvania
northward into New England. Boston, MA received
16.2 inches while Providence RI had the greatest
single snowstorm on record with 17 inches,
beating the previous record of 12 inches set
December 5-6, 1981. (from http//www.met.rdg.ac.u
k/brugge/world2003.html)
NASA-GSFC, data from NOAA GOES
50
Summary of ATReC forecast Impacts

• Control (routine observations)
• ATReC (routine additional observations)

51
Case 37 Obs. Time 5th Dec 18UT, Verif. Time 7th
Dec 12UT
Control ATReC (routine additional
observations) ATReC (routine additional
observations in target area)
52
Some ATReC 2003 conclusions

• Statistical verification of forecast impacts
• Large sample needed to estimate forecast skill
  differences in verification regions.
• Data denial experiments may provide larger sample
  size.
• Sensitive area prediction method
• Sample similar observational coverage for both SV
  and ETKF sensitive regions not possible during
  ATReC.
• ATReC can be used as to plan future adaptive
  observation campaigns
• Use observation targeting for medium range.
• Improve efficiency so as to shorten warning time
  for observation providers.
• Identify cost-effective observation platforms
  (i.e. dropsondes).
• Cancel cases that show sharp reduction in
  uncertainty as observation time approaches.

53
Forecast sensitivity
From Cardinali Buizza, 2003
54
Observation Contribution to Forecast
Total Contribution
Mean Contribution
55
Forecast and Analysis Sensitivity
56
Forecast impact of observations

• Forecast sensitivity to observations has been
  computed for the campaigns showing a forecast
  impact (ATreC-Control)/Control 10
• In general, results show that in 13 cases out of
  38 9 positive and 4 negative
• Targeted observations decrease the forecast error
  in verification area for 60 of cases (ALTHOUGH
  high skill in control forecasts).
• Differences in forecast impact come also from the
  continuous assimilation cycling that provides
  different model trajectories
• From the campaigns of 5th Dec at 18 UTC -
  Targeted observations improved the forecast of a
  cyclone moving along the east coast of North
  America for which severe weather impact was
  forecast.

57
Future of Observation Targeting

• Studies of the value of targeted observations
• Buizza et al. (2005) performed experiments on
  100 cases to
• Assess impact of observations taken in Pacific
  and verified over North America
• Assess impact of observations taken in Atlantic
  and verified over Europe
• Compare impact of observations taken in SV-target
  areas to observations taken in random areas.
• Estimate the average impact of targeted
  observations.

• Adding targeted observations in the Pacific is
  expected to have a rather small impact (5 on
  z500/z1000 forecast errors over North America)
• Adding targeted observations in the Atlantic is
  expected to have an even smaller impact (3 on
  z500/z1000 forecast errors over Europe)
• Value of observations taking in oceans is
  regionally dependant and depends on the
  underlying observation system.
• Up to the optimization time, the value of
  observations taken in SV-target areas is higher
  than the value of observations taken in similar
  size random areas.

58
Future of Observation Targeting

• Data denial studies (Kelly, Buizza, Thepaut,
  Cardinali)
• Relatively inexpensive and incorporate a large
  number of cases ( 6 months).
• Assess value and impacts of specific types of
  observation platforms.
• Hurricane targeting (Majumdar et al. 2005, 2006)
• Improve short-range forecasts of cyclone tracks
• Provides useful comparison on ETKF and SV
  targeting techniques.
• Satellite Targeting
• Targeting provides utilisation of dynamical
  thinning techniques.
• Improved channel selection can reduce problems in
  cloudy conditions

59
Future of Observation Targeting

• Targeted observation campaigns
• Winter Storms Reconnaissance Program (ongoing)
  (NCEP/ NRL)
• European THORPEX Regional Campaign 2007 a
  virtual targeting experiment.
• African Monsoon Multidisciplinary-Analyses (AMMA)
  Observing System test 2005-2010
• Pacific-Asian Regional Campaign (PARC) 2008.
• Improve adaptive parts of the observing network
• New platforms (e.g driftsondes) are being
  developed
• Rocketsondes
• Aerosondes