Thermospheric Data Assimilation

1
Thermospheric Data Assimilation

• Tim Fuller-Rowell, Cliff Minter, and
• Mihail Codrescu
• Specify and forecast neutral atmosphere
  composition for ionospheric models (GAIM)
• Specify and forecast neutral density and
  temperature for drag calculations

2
Relationship between composition and ion density
Ionospheric depletion
Increase in N2
Modeling
Observations Paxton et al. 2001
3
Future DMSP/NPOESS Data Sources

• Special Sensor Ultraviolet Limb Imager (SSULI).

• Special Sensor Ultraviolet Spectrographic Imager
  (SSUSI).

4
GUVI-TIMED dataApril 2002 Storm
Paxton et al
5
The Challenge
How do we optimally combine the understanding
captured in our physical models with the
available data to specify and forecast the state
of the system
CTIP
GUVI-TIMED
6
Kalman Filter Data Assimilation

• A Kalman Filter has been developed to test the
  utility of neutral density or temperature data
  from low Earth-orbiting satellites. Kalman
  filter data assimilation is an optimal method
  combining data and models for specification and
  forecast of the space environment.

x - Model State Vector M - State Transition
Matrix ? - Transition Model Error P - Model
Error Covariance Q - Transition Model Error
Covariance y - Data Vector H - Measurement
Matrix ? - Observation Error R - Observation
Error Covariance K - Kalman Gain
C. Minter, T.J. Fuller-Rowell, and M.V. Codrescu
7
Whats behind the Kalman Filter?

• Model propagated state x
• Standard deviation of the error in the state sx
• Observation y
• Standard deviation of the error in the
  observation sy
• New estimated state z
• Standard deviation of the error in the new state
  sz

8
Whats behind the Kalman Filter?

9
Covariance Matrix
x1
x2
? - uncertainty in estimate of state element ? -
how one state element is related to another
10
Problems in Covariance Matrix

• Size n x n computationally intensive, often use
  diagonal or sparse covariance matrix
• Need to propagate forward in time with the state
• Usually dont know ?, assumes linear
  relationship between state elements, and
  instantaneous
• Observations dont automatically improve
  estimate of ?

x1
x2
11
The Ensemble Kalman Filter
3 Monte Carlo Example
Take Measurement
Calculate the Kalman Gain
Randomize the State
Propagate the State
Return to the Model
Calculate the Covariance
Updated States


Model
1
1
X
1
X
k
k1
__

P
K

2
Model
2
2
X
k1
X
k
k1


3
Model
3
3
X
X
k
k1


Main
Model
K
X
X
k
k1
12
Propagating the State Covariance
To get the propagated state covariance,
instead of using the usual two Kalman equations,
solve directly using...
need to run model many times 10, 20 .maybe 100
time every time step!
13
Model Types

• Persistence
• Polynomial
• Advanced Physical Model Coupled
  Thermospheric-Ionospheric Model (CTIM)
• Gauss-Markov - relax to climatology
• Empirical Model Mass Spectrometer and Incoherent
  Scatter (MSIS) or neural network

14
The H Matrix

• Relates observations to the state

15
Example of Gauss-Markov Kalman Filter

• Simulate real storm interval 15-19th April 2001
• Truth file generated from CTIP
• Using Weimer electric field driven by solar wind
  data
• 3-days driven by time-dependent ACE solar wind
  data
• Realistic observation scenario
• Relaxation to MSIS, ? 15 hours
• 10 minute update cycle
• 2-D Kalman state, 2 latitude x 6 longitude

16
Observation Scenario

• 2 sun-synchronous satellite observing systems.
• 0930/2130 and 1230/0030 LT equatorial crossing.
• SSUSI horizon-to-horizon scan O/N2 at 1021 N2
  column depth, observation error 0.1 units.
• SSULI vertical scan O, N2 vertical profiles.
• Measurement every 1 sec., sunlit hemisphere only.

17
April Storm 2001 - Gauss-Markov
Definition of State Ratio of height integrated O
and N2

• .

18

Storm changes
Truth

• .

G-M Filter
MSIS storm changes
Filter storm changes
Error
19
Neutral Composition Physics-Based Kalman Filter

• Can we use a physics-based model to improve on
  the accuracy, particularly during storms

20
Best Case Scenario Physics-based Kalman filter
compared with Gauss-Markov and MSIS

• .

Best case scenario driver of model in propagator
of Kalman state same as in truth file
Using CTIM
21
Realistic Case April Storm Physics-based Kalman
filter

• .

MSIS
CTIM
G-M
22
Challenge of physical model Kalman filter
Joule heating modeled
Joule heating reality

• .

Need to put the model drivers in the Kalman state
23
Similar challenges for neutral density capturing
wave propagation
24
Neutral Density Wave
Kalman Filter Observations 4 satellites 4 orbit
planes State Propagator physical model with
known phase velocity
25
SETA - Neutral Density Waves and
Holes compared with CTIM - Coupled
Thermosphere Ionosphere Model
26
Lessons Learned

• Dont treat the Kalman Filter as a black box,
  its a powerful tool but it has its limitations
• Dont underestimate the time to develop a data
  assimilation model or the time to acquire the
  same level of intuition - it takes the same time
  to develop a data assimilation model and
  understand how it works it as it does a physical
  model
• The devil is in the details - covariance
  matrices, process noise, correlation lengths,
  computational constraints, etc...
• Set up a test-bed to try out different techniques
  using combination of metrics and common sense to
  evaluate
• Draw on the GAIM experiences and Good Luck!