Dennis McLaughlin, Parsons Lab', Civil

1
Dennis McLaughlin, Parsons Lab., Civil
Environmental Engineering, MIT Dara Entekhabi,
Parsons Lab., Civil Environmental Engineering,
MIT Rolf Reichle, NASA Goddard Space Flight Center

• Problem context - Mapping continental-scale soil
  moisture from satellite passive microwave
  measurements. Problem is spatially distributed,
  nonlinear, and has many degrees of freedom
  O(106). Available models of hydrologic system
  and measurement process are highly uncertain.
• The ensemble Kalman filter
• Results from a synthetic experiment (OSSE)
• An opportunity for multi-scale estimation?

2
Soil Moisture
Soil moisture is important because it controls
the partitioning of water and energy fluxes at
the land surface. This effects runoff (flooding),
vegetation, chemical cycles (e.g. carbon and
nitrogen), and climate.
Soil moisture varies greatly over time and space.
Measurements are sparse and apply only over very
small scales.
3
Microwave Measurement of Soil Moisture
L-band (1.4 GHz) microwave emissivity is
sensitive to soil saturation in upper 5 cm.
Brightness temperature decreases for wetter
soils. Objective is to map soil moisture in real
time by combining microwave meas. and other data
with model predictions (data assimilation).
4
SGP97 Experiment - Soil Moisture Campaign
5
Relevant Time and Space Scales
Plan View Estimation pixels (small) Microwave
pixels (large)
Vertical Section Soil layers differ in
thickness Note large horizontal-to-vertical scale
disparity
For problems of continental scale we have 105
est. pixels, 105 meas, 106 states,
6
State equations are derived from mass and energy
conservation
Soil moisture is governed by a 1D (vertical)
nonlinear diffusion eq (PDE). Soil temperature
and canopy moisture are linear ODEs.
7
The Estimation (Data Assimilation) Problem
What is the best estimate of the state y(t),
given the vector Zi z1, ..., zi of all
measurements taken through ti? Posterior
probability density py(t) Zi is the ideal
estimate since it contains everything we know
about y(t) (given Zi )
In practice, we must settle for partial
information about this density

• Some options
• Variational Approaches Derive mode of py(t)
  Zi by solving least-squares problem.
• Extended Kalman Filtering Uses Gaussian
  assumption to approximate mean and covariance of
  py(t) Zi.

Both have serious limitations Is there a more
efficient and complete way to characterize
py(t) Zi ?
8
Divide filtering problem into two steps
propagation and update. Characterize random
states with an ensemble (j 1, , J) of random
replicates
Evolution of posterior probability density
Evolution of random replicates in ensemble
Ensemble filtering propagates only replicates (no
statistics). But how should update be performed?
It is not practical to construct and update
complete multivariate probability density.
9
The updating problem simplifies greatly if we
assume py(ti1) Zi1 is Gaussian. Then update
for each replicate is
K Kalman gain derived from propagated ensemble
sample covariance Covy(ti1) Zi. After each
replicate is updated it is propagated to next
measurement time. No need to update
covariance. This is the ensemble Kalman filter
(EnKF).

• Potential Pitfalls
• Appropriateness of the Kalman update for
  non-Gaussian density functions?
• Need to construct, store, and manipulate large
  covariance matrices (e.g. 5000 X 5000 for our
  example)

10
Observing System Simulation Experiment (OSSE)
Mean land-atmosphere boundary fluxes
Random model error
True soil, canopy moisture and temperature
Soil properties and land use
Land surface model
Radiative transfer model
True microwave radiobrightness
Random meas. error
Mean initial conditions
Measured microwave radiobrightness
Random initial condition error
Soil properties and land use, mean fluxes and
initial conditions, error covariances
Estimated microwave radiobrightness and soil
moisture
OSSE generates synthetic measurements which are
then processed by the data assimilation
algorithm. These measurements reflect the effect
of random model and measurement errors.
Performance can be measured in terms of
estimation error.
11
Synthetic experiment uses real soil, landcover,
and precipitation data from SGP97 (Oklahoma).
Radiobrightness measurements are generated from
our land surface and radiative transfer models,
with space/time correlated model error (process
noise) and measurement error added.
SGP97 study area, showing principal inputs to
data assimilation algorithm
12
Normalized error for open-loop prediction (no
microwave meas.) 1.0 Compare jumps in EnKF
estimates at measurement time to variational
benchmark (smoothing solution). EnKF error
generally increases between measurements.
Increasing ensemble size
13
Scattered errors decrease as number of replicates
increases
14
Error decreases faster than J0.5 up to 500
replicates but then levels off. Does this
reflect impact of non-Gaussian density (good
covariance estimate is not sufficient)?
15
Filter consistently underestimates rms error,
even when input statistics are specified
perfectly. Non-Gaussian behavior?
16
Errors appear to be more Gaussian at intermediate
moisture values and more skewed at high or low
values. Uncertainty is small just after a storm,
grows with drydown, and decreases again when soil
is very dry.
17
EnKF provides discontinuous but generally
reasonable estimates of model error but sample
problem. Compare to smoothed error estimate from
variational benchmark. Specified error
statistics are perfect.
18
EnKF is severely limited by need to compute large
covariance at update step. Can this covariance be
replaced by a smaller tree model which can then
be used to condition ensemble replicates?
Fine scale replicates at time ti
. . .
19

• Ensemble filtering provides an efficient reduced
  rank method to reduce size of large distributed
  systems (replicates are comparable to reduced
  rank basis functions).
• Ensemble forecasting/propagation characterizes
  distribution of system states (e.g. soil
  moisture) while making relatively few
  assumptions. Approach accommodates very general
  descriptions of model error.
• Most ensemble filter updates are based on
  Gaussian assumption. Validity of this assumption
  is problem-dependent.
• Updates can be performed using a classical
  covariance-based Optimal Interpolation/Kriging
  procedure. Alternative covariance-free
  (multi-scale or variational) updates may provide
  similar results with much less computational
  effort.
• Ensemble filtering is a very flexible, efficient,
  and easy-to-use data assimilation method that may
  greatly improve our ability to interpret large
  amounts of remotely sensed hydrologic data