Observing System Design and Targeted Observing

1
Observing System Design and Targeted
Observing Carolyn Reynolds, Naval Research
Laboratory, Monterey, CA

• II. Targeted Observing
• Current techniques and programs
• Predicting data impact
• Issues
• Validation and sampling
• Nonlinearities
• Model error

• I. Observing system design
• Simple model results
• Effective observing networks
• Impact of time-dependent basic state
• Considerations for Operational Systems

Thanks to Craig Bishop, Rolf Langland, Nancy Baker
2
For time-independent dynamics and observation
operators M and H, Kalman filter covariances
become time-invariant
Observing System Design Idealized System

• For Q0, Pf takes form of outer product of
  transformed eigenvectors of M.
• Eigenvalues of M projection of eigenvectors
  onto R-1 determine transform
• Only growing normal modes required to precisely
  represent error covariances. Rank of error
  covariance much smaller than d.f. of M.
• Does not hold with Q?0 nor with time evolving M.

Bishop, Reynolds, Tippett JAS 2003
3
Effective Observing Network Simple Global Model
(1449 degrees of freedom)

• Given current observing network, find location
  of additional column observation that minimizes
  trace (Pf)
• Repeat

4
Effective Observing Networks Simple Global Model
Effective observing network produces forecast
error variances several times smaller than other
types of networks
5
Effective Observing Networks Simple Global Model
Relative difference between observing networks
decreases as networks become more dense.
6
Effective Observing Networks Cost Function
Minimizing trace(Pf) will give difference
locations for different forecast lengths. For
12-hour forecasts 23 obs out of 150 at
500 mb For 72-hour forecasts 33 obs
out of 150 at 500 mb
7
Effective Observing Networks Cost Function
Place observations at one level only. Which level
to pick? 200-mb obs (top) give lower trace(Pf)
than 500-mb obs (middle). However, if interest
is in jet regions, 500-mb obs are better. How do
you pick forecast error component to minimize?
8
Effective Observing Networks Function of M
Effective observing networks will change with the
basic state. Real atmospheric state changes
quickly Adaptive observing use (time-dependent)
dynamics to inform configuration of adaptive
component of observing network What does real
observing network look like?
Zonal
Blocked
9
FNMOC NOGAPS DATA ASSIMILATION RADIOSONDES
Radiosondes provide atmospheric profiles with
high vertical resolution. Very few radiosondes
over ocean basins. (Arlene dropsondes in pink).
10
Dynamic Amplification of Perturbations
Forecasting is a Global Problem
11
FNMOC NOGAPS DATA ASSIMILATION Buoy Coverage
Surface observations over the ocean Buoys.
12
FNMOC NOGAPS DATA ASSIMILATION Ship/Coastal
Coverage
Surface observations over the ocean Ships.
13
FNMOC NOGAPS DATA ASSIMILATION Meteorological
Data Collection and Reporting System
14
FNMOC NOGAPS DATA ASSIMILATION International
Aircraft Meteorological Data Report
15
FNMOC NOGAPS DATA ASSIMILATION DMSP Special
Sensor Microwave/Imager Ocean Surface Winds
Surface observations over the ocean Surface wind
speeds from satellite.
16
FNMOC NOGAPS DATA ASSIMILATION CIMSS/Univ. of
Wis., Feature Tracked Winds Coverage
Feature-track winds provide some upper-level data
over oceans.
17
FNMOC NOGAPS DATA ASSIMILATION Advanced
Microwave Sounding Unit All Data
Steve Swadley, NRL
Assimilation of satellite radiances can provide
temp and humidity profiles. 67,000 observations
from one satellite for 1 channel in 1 3-hr
interval.
18
FNMOC NOGAPS DATA ASSIMILATION Advanced
Microwave Sounding Unit Data after thinning and
QC
Steve Swadley, NRL
After data thinning and QC, reduced from 67,000
to 2,000. Is there a better way to perform
spatial thinning?
19
FUTURE DATA ASSIMILATION Amount of satellite
data is increasing dramatically
New Satellite Sounder (hyperspectral)
Current Satellite Sounder AMSU-A
Select 10-100 channels out of 8000 to be
assimilated
8 channels currently assimilated
20
Selective Thinning of Data OSEs, OSSEs,
Observation Sensitivity

• Observing System Experiments (OSEs) Data denial
  experiments. Can be very useful, but expensive.
• Observing System Simulation Experiments Run
  analysis-forecast cycle with and without
  simulated observations. Tests hypothetical data,
  but need accurate error statistics.
• Observation Sensitivity Use the adjoints of the
  forecast model and data assimilation system to
  find sensitivity of forecast errors to
  observations. Efficient but assumes perfect
  model and linear error growth.

21
Observation Sensitivity using the Adjoint of the
DA System
ef lt(xf xt), C(xf xt)gt Jf ef /
2 ?Jf/ ?xf C(xf xt) ?Jf/ ?xa LT ?Jf/ ?xf
?Jf/ ?y HPbHT R-1HPb ?Jf/ ?xa xa xb
PbHTHPbHT R-1 (y Hxb)
Observation sensitivity (?Jf/ ?y)is used to
estimate how forecast error is changed by adding
small perturbations to actual or hypothetical
observations. 2 gradient calculations can be used
to estimate the impact of observations on the
reduction in forecast errors between 48-h and
42-h forecasts.
Baker and Daley 2000, Langland and Baker, 2004
22
Data Selection (Intelligent Thinning of Satellite
Data)
December 2003
CHANNEL
Global Forecast Error Reduction (J kg-1) 42-h
Forecast Error minus 48-h Forecast Error
8
8
5
5
7
6
6
NOAA-16
NOAA-15
Efficient ways to estimate data impact critical
for intelligent selection of satellite data
Langland and Baker
23
Observing System Design

• Effective observing network design will be
• A function of metric
• A function of the dynamics
• Daily
• Regime (blocked vs zonal)
• Seasonal or interannual
• Need efficient ways to select satellite data

24
Observing System Design and Targeted Observing

• II. Targeted Observing
• Key components of analysis error
• Current techniques and programs
• Predicting data impact
• Issues
• Validation and sampling
• Nonlinearities
• Model error

25
Forecast Errors and Key Initial Perturbations
Singular Vectors and Pseudo-inverse Corrections

• Forecast error corrections using SV-based
  pseudo-inverse
• Meo ef M UDVT eo M-1ef V
  D-1UTef
• Compose Pseudo-inverse of 3 leading SVs to find
  fast-growing component of initial perturbation
• eo3 V3 D3-1U3Tef ? vk dk-1 lt ukef gt
• Subtract eo3 from analysis then run nonlinear
  corrected forecast
• Compare the nonlinear corrected forecast with
  linear corrected forecast eflin ef - ?
  uklt ukef gt

k1,3
k1,3
Do SVs look like forecast errors? Can
pseudo-inverse initial perturbations improve
forecasts?
26
Forecast Errors and Key Initial Perturbations
Singular Vectors and Pseudo-inverse Corrections
Small changes to initial state can result in
significant error reduction. What is the impact
of additional observations in these key regions?
27
Adaptive Observing Simple-model Experiments
Lorenz and Emanuel (1998) better to place
supplemental observations where analysis errors
are greatest, rather than where forecast is most
sensitive. Hansen and Smith (2000) Dynamical
guidance useful for adaptive observations as long
as linear assumption valid. Morss et al (2001)
Bigger impact from adaptive observations in
sparse networks than in dense networks. Combine
information about dynamics and initial errors,
analysis error covariance singular vectors
Ehrendorfer and Tribbia (1997).
28
Optimal Perturbation Growth Day-to-Day Changes
ltLptBLptgt ltp0Ap0gt
10 Jan
13 Jan
11 Jan
14 Jan
12 Jan
15 Jan
29
Atmospheric Adaptive Observing Techniques

• Ensemble Transform Kalman Filter (Bishop et al.
  2001)
• Use ensembles to construct approximate error
  covariances
• Assess various adaptive observing configurations
  for hypothetical reduction in forecast error
  variance
• Singular Vectors
• Add additional observations to sensitive
  regions
• Sensitive to metric Approximations to Pa-1
• Hessian SVs (ECMWF, Barkmeijer et al. 1998)
• Variance SVs (NRL, Gelaro et al. 2002)
• Analysis Error Covariance SVs (Hamill et al.
  2003)
• Observation Sensitivity (Baker and Daley 2000)
• Use adjoint of forecast model and data
  assimilation system to find sensitivity of
  forecasts to changes in the observations.

30
Ensemble Transform Kalman Filter

• Use Ensemble Transform Kalman Filter (ETKF) for
  quantitative estimate of forecast error variance
  reduction
• Signal of obs associated with obs operator H at
  forecast time t is
• xa(t)-xf(t)L(t,ta)PfHT(HPfHTR)-1y-Hxf(ta)

Signal Covariance S(tH)L(t,ta)PfHT(HPfHTR)-1HP
fL(t,ta)T Prediction Error Covariance
P(tH)L(t,ta)PfL(t,ta)TQ-S(tH) where Q is
model error. Thus SIGNAL VARIANCE REDUCTION
IN ERROR VARIANCE
Bishop et al. (2001)
31
Adaptive Observing ETKF for Winter Storm
Reconnaissance
Signal variance for flight track 23 is largest.
Signal variance for flight track 46 is largest
assuming track 23 observations already
assimilated.
Majumdar et al. 2002
32
Observation Sensitivity using the Adjoint of the
DA System

• Previous targets based on forecast sensitivity to
  changes in analysis (xa). Now also use analysis
  sensitivity to changes in observations (y).
• Quantify expected impact of additional
  observations on forecast error variance
• Examine impact of hypothetical data distributions
  and observing platforms

1Navy Operational Global Atmospheric Prediction
System 2NRL Atmospheric Variational Data
Assimilation System
Baker and Daley, 2000
33
Adaptive Observation Products for THORPEX
Variance singular vector target regions (top)
usually match high observation sensitivity
(right).
34
Atmospheric Adaptive Observing Programs

• FASTEX (1997)
• NORPEX (1998)
• Winter Storm Reconnaissance (ongoing)
• Atlantic THORPEX Regional Campaign ( 2003)
  dropsondes, off-time Radiosonde, additional
  commercial aircraft, etc.
• Hurricane Research Division (ongoing)
• DOTSTAR (Taiwan typhoon adaptive observing)

35
Adaptive Observing Products for Tropical Cyclones
Majumdar, Abrams, Bishop, Buizza, Peng, Reynolds
36
Adaptive Observing Products for Tropical Cyclones
Majumdar, Abrams, Bishop, Buizza, Peng, Reynolds
37
Adaptive Observing Products for Tropical Cyclones
Majumdar, Abrams, Bishop, Buizza, Peng, Reynolds
38
Issues Validation
How do we assess optimality of adaptive
deployment? Can we predict observation impact
on forecast error variance? Need large sample
for validation ef M e0 M Pa MT Pf
Langland et al, 1999
39
Validation Predicting Observation Impact with
ETKF
On average, larger ETKF predicted signal variance
does correspond to larger forecast signal. Would
spread-skill relationship improve if using
ensemble transform data assimilation system
(instead of 3DVAR?)
Majumdar et al. 2002
40
Validation Predicting Observation Impact with
ETKF
Evaluate ET KF signal variance prediction when
using data assimilation scheme with
flow-dependent covariances.
Bishop, Majumdar, and Toth
41
Validation Observation Sensitivity
December 2003 Rawinsondes, Dropsondes Impact on
42-h Forecast error
These results show the impact of the different
soundings on forecast error reduction during
Atlantic-THORPEX regional campaign.
Rolf Langland
42
Observation Impact Atlantic THORPEX Regional
Campaign
Cumulative observation impact on 48-h Forecast
Error (J kg-1) from observations assimilated in
NAVDAS at 1800 UTC in the NA-TReC domain
(10N-70N, 100W-40E) from 1 November to 31
December 2003.
Rolf Langland
43
Validation Consistency of target method and
DA-Forecast system
Ideally, observations would not just change
analysis, but would change construction of the
initial-time ensemble perturbations Validation
should encompass impact on ensemble spread
44
Issues Linearity Assumption
Nonlinear evolution of positive and negative SV
perturbations (solid). Sum of positive and
negative perturbations (dashed). At 12 (red), 24
(blue) and 48 (brown) hours.
Nonlinear perturbation growth far more
significant on smaller scales. At short forecast
times, nonlinearities are primarily due to
diabatic processes.
45
Issues Linearity Assumption
Gilmour and Smith
At 12 and 24 hours, larger perturbations have
smaller relative nonlinearities. Decrease in
relatively nonlinearities between 12 and 24 hours.
46
Issues Linearity Assumption
Nonlinearities increase with perturbation size
for adiabatic perturbations. Large nonlinearities
a function of diabatic processes.
47
Issues Model Error
48
Issues Model Error
Model errors and model differences behave quite
differently, especially in tropics.
49
Adaptive Observing and Observing System Design
Outstanding Issues

• Quantitative estimates of observation impact on
  forecast error variance (are methods consistent
  with DA and ensemble forecasting system?)
• Sampling issues
• Limitations of linear assumption
• Limitations of perfect model assumption
• Efficient ways to selectively thin satellite
  data? (will it matter?)

50
Issues Linearity Assumption
Hurricane Singular Vectors Positive and Negative
Perturbation 850-mb Vorticity
Linear Perturbations
Nonlinear Perturbations
In full nonlinear forecasts, perturbations alter
the basic state. Positive and negative
perturbations approximately symmetric but exhibit
phase shift.
51
Issues Linearity Assumption
Hurricane Singular Vectors Positive and Negative
Perturbation 850-mb Vorticity
Linear Perturbations
Nonlinear Perturbations
In full nonlinear forecasts, perturbations alter
the basic state. Positive and negative
perturbations approximately symmetric but exhibit
phase shift.
52
Quantifying Limits of Atmospheric
PredictabilityEnvironmental Processes
IMPACT OF DIFFERENT OBSERVING SYSTEM COMPONENTS
ON 72-h GLOBAL FORECAST ERROR TOTAL ENERGY JULY
2002
Number of observations
2,000,000
1,000,000
Accumulated reduction in error (J kg-1) due to
observations.
RAOB SATWIND SHIP
AUS ATOVS AIRCRAFT LAND
SSMI
The NAVDAS adjoint gives the impact of different
observing system components in a rigorous and
computationally feasible manner.
Encl (7)
53
Predicting Forecast Uncertainty
SVs large where forecast errors are large. This
type of diagnostic captures both spatial and
temporal variations in forecast error variance.
Providing information on flow-dependent
reliability of forecasts
54
72-h SV1 for 2002021400 Positive and Negative
Nonlinear Perturbations
Examine symmetry to quantify nonlinearities
? for 300 hPa V
55
Quantifying Limits of Atmospheric Predictability
Approach
Improving the Data Assimilation System with
NAVDAS Adjoint
Specified error variance (eob ) of ATOVS
temperature retrievals
Note large error variance below 700 mb (near
surface)
Negative values of sensitivity indicate
24h-forecast error (J) would be smaller if eob
was reduced in these levels of the model. Other
levels are not as sensitive.
Alternative (improved) profile of (eob )
Low-level ATOVS error variances appear to be too
large.
20S-20N (Tropics) 00UTC 31 Dec 2002
Explore impact of the specified observation error
variances on forecast Errors
56
200-hPa Meridional Wind
01/10/00
01/12/00
Forecast errors travel at the group speed, with
profound implications for predictability. The
Washington, D.C. blizzard of 25 January 2000 was
part of a group that traveled from the mid
Pacific to the US east coast in 72 hours.
Accurate analysis of this group in the
mid-Pacific would have been necessary for
accurate 72-hour forecasts of the blizzard.
UPPER-TROPOSPHERIC WAVE PACKET PROPAGATES WITH
GROUP VELOCITY OF 30 m s-1
01/15/00
01/18/00
R1
T1
01/21/00
R2
T2
01/24/00
BLIZZARD FORECAST VERIFICATION TIME
01/27/00
SYNOPTIC-SCALE TROUGH AND RIDGE FEATURES
PROPAGATE WITH PHASE VELOCITY OF 5-10 m s-1
01/30/00
0
60E
120E
180
120W
60W
0
Longitude
Adapted from Langland et al., 2001 (NRL/Monterey)
-60
-48
-36
-24
-12
0
12
24
36
48
m s-1
57
Optimal Perturbation Growth Day-to-Day Changes
0.17
0.30
Leading Lyapunov Vector and growth rate for
time-evolving basic state (T21L3 QG model)
0.33
0.24
0.10
0.28
0o 180o 0o
58
EFFECTIVE ROUTINE OBSERVATIONAL NETWORKS IN A
T21L3 QG MODELC. H. Bishop and C. A.
ReynoldsUCAR and Naval Research Lab, Monterey,
CAThanks to R. Gelaro (NASA/DAO) and M. Tippett
(IRI)

• Given high cost of developing and maintaining
  global observational networks, methods to aid in
  effective design of such networks are needed.
• Use simple (time-independent, perfect model)
  system to evaluate relative effectiveness of
  different observational network configurations
  using an optimal data assimilation scheme.

59
METHOD

• Use T21L3 QG model (Marshall and Molteni, 1993)
• Find quasi-steady states (time-mean and blocked
  flow)
• Mxo xt Find eo such that M(xoeo)xoeo
• Solve eo(M-I)-1 (xo-xt)
• Assume autonomous system
• Observations of QG streamfunction (equivalent to
  100 meter height error at 45N) at Gaussian grid
  points only
• 12-hour analysis cycle
• No model error
• Solution composed of 122 non-decaying
  eigenvectors of tangent forward propagator
  (compared to 1449 d.f. in the model).

60

• SUMMARY
• For time-independent system with negligible model
  error and optimal DA scheme, only growing normal
  modes needed to describe error covariances.
  Testing of many observing configurations becomes
  feasible.
• Error correlations
• Baroclinic tilt,
• Significant remote correlations
• Effective observing configurations
• Clustered in mid-latitude belts
• Several times more effective than other (e.g.
  land-based or equally-spaced) observing networks.
• For obs at one level only
• 200-mb obs more effective at reducing global
  forecast variance
• 500-mb obs more effective baroclinic regions.
• FUTURE WORK
• Test impact of sub-optimal correlations
• Test impact of model errors
• Evaluate Eigenvectors in more complex model

61
Targeted-observing applications
Perturbations and errors exhibit fast group-speed
propagation as well as slower phase-speed
propagation. Targeting trough of interest may
not be enough.
200-mb background PV (black contours) and SV
perturbation meridional wind (shading).
62
SV DEFINITION

• SVs Fastest growing (linear) perturbation to a
  given trajectory
• SVs are a function of metric. SVs maximize ratio
• ltPxtEPxtgt
• ltxoCxogt
• P Local projection operator
• E Final-time metric
• C Initial-time metric
• For TE SVs E C total energy norm
• For VAR SVs E total energy norm C inverse
  analysis error variance norm
• Satisfy Eigenvector problem
• LTPTEPL x ? Cx

Will VAR SVs explain more forecast error than TE
SVs?
63
Patterns can look similar and still have high
Nonlinearity Index (NI). Patterns often exhibit a
shift relative to one another, which can result
in a high NI index, especially for fine-scaled
fields.
NEG. ANOM
Adjoint and SV tools may be useful even if
nonlinearities are large, depending on the
application.
64
Linear tools may still be useful even when full
perturbation growth is significantly nonlinear.
Perturbations within dry SV subspace highly
linear.
65
SINGULAR VALUES (AMPLIFICATION FACTORS)
66
Vertically Averaged Total Energy for NH 48-h SVs
TE SVs
TE SVs
VAR SVs shifted toward oceans, high latitudes
VAR SVs
VAR SVs
INITIAL TIME
FINAL TIME
67
Nonlinear perturbation energy as a function of
total wave number (solid) Energy of the sum of
the positive and negative nonlinear perturbations
(dashed)
Synoptic scales far more linear than smaller
scales
68
DYNAMICS DA Plans

• Use Ensemble Transform Kalman Filter (ETKF) for
  quantitative estimate of forecast error variance
  reduction due to new observations
• ET KF solves KF estimation equations in ensemble
  subspace
• Ensembles used to estimate forward propagation of
  error covariance matrix
• Forecast perturbations transformed into analysis
  perturbations reflecting density and accuracy of
  observations
• P(tH)L(t,ta)PfL(t,ta)T Q - S(tH)

SIGNAL VARIANCE REDUCTION IN ERROR VARIANCE
Bishop et al. 2001.
69
Ensemble Spread vs. Forecast Error
Impact of dynamic conditioning of initial
perturbations (ensemble transform)? Impact of
model errors (sst perturbations, stochastic
physics, different physical parameterizations)?
Singular Values vs. Tropical Cyclone Track Error
What physical processes are necessary to capture
the predictability of tropical cyclone tracks
(moist process, high resolution)?
Providing information on flow-dependent
reliability of forecasts
70
Effective Observing Networks Function of
Forecast Time
How much improvement can come from better
observing network design?
Location of effective column observations
Global error variance reduction () as a function
of location
We find the exact solution to the Kalman filter
in this simple, time-independent, perfect model
system.
Effective placement of column obs yields global
error variances several times smaller than other
configurations, such as land-based or equally
spaced observations
Bishop, Reynolds, Tippet, 2003
71
Atmospheric Predictability and Sensitivity
Approach
Use the NAVDAS ADJOINT to design effective
observing strategies
These plots show the sensitivity of 72-hr
forecast error over a region of North America to
existing radiosondes supplemented by (A) a
hypothetical network of Pacific driftsondes, and
(B) dropsondes from the NOAA G-IV in Winter Storm
Reconnaissance.
Explore impact of actual and hypothetical
observing platforms/configurations
Langland, Baker and Daley
72
Atmospheric Predictability and Sensitivity
Singular Vectors and Lyapunov Vectors
Fraction of Variance of Leading Lyapunov Vector
Explained by Subset of Leading 1-day Singular
Vectors
What is the relationship between SVs (transient
growth) and Lyapunov Vectors (asymptotic
growth). Initial time SVs explain only small
fraction of Lyapunov vector. However, they
account for most of the growth of the Lyapunov
vector Growing part of LV contained within
leading SV subspace
Growth rate of Lyapunov Vector with Leading
Singular Vectors Removed
Gelaro, Reynolds, Errico 2002.
73
When linear and nonlinear corrections differ,
nonlinear corrections usually better
Although not optimal, SVs still relevant (i.e.
explain substantial part of large forecast
errors), even at 72 h
74
Stability Analysis of Linearized Dynamics
Operator M

• et M(0,t) e0 q Pf
  M Pa MT Q
• System is nonnormal, eigenvectors of M not
  orthogonal, transient (finite time) growth can be
  much faster than exponential normal-mode growth
• Optimal finite time growth given by leading
  singular vectors (SVs).
• M B-1/2 U D VTA1/2 Leading SV
  maximizes ltptBptgt/ltp0Ap0gt
• Average variance perturbation (error) growth
  given by the mean square of the singular values
  given that initial errors project equally onto
  all initial singular vectors (Lorenz 1965)
• For forecasting applications, appropriate
  initial metric is inverse of the analysis error
  covariance matrix, Pa-1 (Ehrendorfer and Tribbia
  1997)

75
Eigenvectors, Adjoint Eigenvectors and Singular
Vectors Time-independent M, T21L3 QG model
Eigenvector M v ? v, ?1.2
Adjoint Eigenvector MT v ? v, ?1.2
Singular Vector MTM v ?2 v, ?3.3
76
Eigenvectors, Adjoint Eigenvectors and Singular
Vectors Time-independent M, T21L3 QG model
Eigenvector M v ? v, ?1.2
Adjoint Eigenvector MT v ? v, ?1.2
Singular Vector MTM v ?2 v, ?3.3