Maturation of Data Assimilation Over the Last Two Decades

1
Maturation of Data Assimilation Over the Last Two
Decades

• John C. Derber
• Environmental Modeling Center
• NCEP/NWS/NOAA

2
Roger Daley (1996) ECMWF seminar on data
assimilation

• Fifteen years ago, data assimilation was a minor
  and often neglected sub-discipline of numerical
  weather prediction. The situation is very
  different today. Data assimilation in now felt
  to be important for all climate/environmental
  monitoring and estimating the ocean state. There
  has been great advances in both modelling and
  instrumentation for a variety of atmospheric
  phenomena and variables, and data assimilation
  provides the bridge between them.

3
Optimal Interpolation (1980s)

• With the advent of optimal interpolation,
  analysis schemes transitioned from empirical
  techniques to theoretically based techniques.
• With these techniques, one could begin to use
  information on
• Observations and observation errors
• Short term forecasts and forecast errors
• However, for most applications of optimal
  interpolation, many approximations had to be made.

4
Variational assimilation (1990s and 2000s)

• J Jb Jo Jc
• J (x-xb)TB-1(x-xb) (H(x)-y0)T(EF)-1(H(x)-y0)
  JC
• J Fit to background Fit to observations
  constraints
• x Analysis
• xb Background
• B Background error covariance
• H Forward model
• y0 Observations
• EF R Instrument error Representativeness
  error
• JC Constraint term

5
Variational assimilation

• Inclusion of observation operator (H),
  transforming the analysis variable into the form
  of the observation operator, which in turn
  allowed
• Inclusion of radiances and other indirect
  observations
• Definition of analysis variables different than
  the model variables or observations
• Inclusion of forecast model in interpolation
  operator (4DVAR)
• Use of all observations at once, eliminating many
  approximations/complex codes which were prone to
  failure
• Allowed inclusion of additional constraint terms

6
Variational Assimilation

• Background error covariances
• Background error variance now approximately ½
  radiosonde error variance (ECMWF)
• Fully non-separable covariance matrices
• Inclusion of constraints within background error
• Ongoing research in situation dependent
  background errors

7
Isotropic Error Correlation in ValleyPlotted
Over Utah Topography obs influence extends into
mountains indiscriminately
8
Anisotropic Error Correlation in ValleyPlotted
Over Utah Topographyobs influence restricted to
areas of similar elevation
9
Anisotropic Error Correlation on Slope Plotted
Over Utah Topographyobs influence restricted to
areas of similar elevation
10
Anisotropic Error Correlation on Mt Top Plotted
Over Utah Topographyobs influence restricted to
areas of similar elevation
11
Observations

• No large scale data voids.
• Number of observations used in assimilation
  increasing rapidly (but not as rapidly as number
  of observations).
• Both operational and non-operational satellite
  data being used operationally.
• Increased use of data assimilation systems in
  calibration/validation activities for satellites.
• Expected data impact from data producers always
  overoptimistic.

12
Observations

• Adequate fast forward models for observations
  still major problem, e.g.,
• Precipitation (satellite/radar)
• Clouds (IR and microwave)
• Lightning
• Biases in forward models/observations greatly
  impact the usefulness of data

13
(No Transcript)
14
Variational Assimilation Surprises

• Computational cost for 3DVAR similar to OI (even
  with all observations used together).
• Much of need for Nonlinear Normal Mode
  Initialization came from data selection.
• Direct use of radiances produces significant
  impacts in both hemispheres.
• In Southern Hemisphere, significantly stronger
  circulations were produced using radiances.
• Southern Hemisphere forecast skill has become
  similar to Northern Hemisphere skill. Since NH
  much better observed, is SH easier?
• Microwave instruments dominate impact.

15
(No Transcript)
16
Future?

• Extension of data assimilation techniques to
• Additional analysis variables (including cloud
  water/ice, etc.)
• Smaller scales
• Tropical disturbances
• Land/ice/ocean surfaces
• Inclusion of improved bias correction for
  background and all types of observations
• Inclusion of observation specific
  observational/representativeness errors

17
Future?

• Use of situation dependent background errors
• Trying to catch up with volume of data from new
  observing platforms
• Improved models and physics within analysis
  systems
• Ensembles?
• New systems judged on performance. With data
  assimilation you must do everything right!

18
Summary

• Assimilation is the integration of all knowledge
  of the atmosphere (observations, physics,
  statistics) to produce the best estimate of the
  real state of the atmosphere.
• Data assimilation systems have matured and become
  fairly good at large scales for the basic
  meteorological variables.
• Assimilation has advanced from a necessary evil
  to an essential scientifically-based component of
  numerical weather prediction.
• However, there are many research groups still
  using empirically based techniques and incomplete
  systems.

19
Summary

• Greatest potential improvement is in improved
  background error estimates (not increasing the
  number of observations).
• Unrealistic expectations from data providers in
  terms of data impacts.
• Operational community has lead the scientific
  development of modern data assimilation systems.
• In data assimilation, details are extremely
  important you must do everything right!

20
(No Transcript)
21
Analysis variables

• Wind, temperature and surface pressure no longer
  sufficient. Additional variables
• Cloud ice and water
• Ozone and other constituent gases
• Surface variables (soil moisture, surface
  temperature, snow, etc.
• Aerosols
• Oceans
• Etc.

22
Anisotropic Error Correlation on Mt Top Plotted
Over Utah Topographyobs influence restricted to
areas of similar elevation
x
23
Sample forecast error structure
24
Satellite observationsCurrent Instruments (used)

• Infrared (IR) sounders (HIRS, GOES)
• Microwave sounders (MSU, AMSU-A/B)
• Microwave imager (SSM/I (wind speed,
  precipitation))
• SBUV (ozone profiles)
• Winds inferred from imagery (GOES, GMS, Meteosat)
• Scatterometers (Quikscat)

25
Satellite observationsCurrent Instruments (not
used)

• Visible/IR imager (AVHRR, GOES, MODIS)
• Microwave sounders ( DMSP/T/T2, TMI)
• TRMM radar
• Total Ozone (TOMS)
• Etc.

26
Satellite observationsFuture Platforms

• EOS-PM (AIRS, AMSU-A, HSB, MODIS)
• GIFTS (IR sounder)
• DMSP (SSM/IS)
• NPP(CrIS, ATMS, VIIRS)
• NPOESS(CMIS, CrIS, OMPS, ATMS, VIIRS, ALT,
  SARSAT)
• METOP(AVHRR, AMSU, IASI, GOME, ASCATT)
• Cosmic (GPS radio-occultation)
• Etc.

27
(No Transcript)
28
Satellite observations

• Different observation and error characteristics
• Type of data (cloud track winds, radiances, etc.)
  
• Version of instrument type (e.g., IR sounders
  -AIRS, HIRS, IASI, GOES, GIFTS, etc.)
• Different models of same instrument (e.g.,
  NOAA-15 AMSU-A, NOAA-16 AMSU-A)