Use of time-dependent parameters for improvement and uncertainty estimation of dynamic models

1
Use of time-dependent parameters for improvement
and uncertainty estimation of dynamic models
Peter Reichert Eawag Dübendorf and ETH Zürich
2
Contents
Motivation References Approach Preliminary
Results Problems/Challenges

• Motivation
• References
• Approach
• Concept
• Implementation
• Preliminary Results for a Simple Hydrologic Model
• Problems / Challenges

3
Motivation
Motivation References Approach Preliminary
Results Problems/Challenges
Motivation
4
Motivation

• Fundamental Objectives
• Improve understanding of mechanisms governing the
  behaviour of the system described by the model.
• Estimate realistic uncertainty bounds / decrease
  the width of uncertainty bounds of model
  predictions.
• Technical Objectives
• Improve the formulation of the deterministic
  model component.
• Make the stochastic component of the model more
  realistic.

Motivation References Approach Preliminary
Results Problems/Challenges
5
Motivation

• Achieve these objectives by
• Improving the input error model.
• Allowing model parameters to vary (e.g. in time)
  to address model structure error.
• Improving the output error model (by addressing
  bias explicitly).
• In particular
• Search for statistical model components that
  cannot be rejected by the data.
• Try to explain the bias by input and/or model
  structure error (trace the causes of the bias).

Motivation References Approach Preliminary
Results Problems/Challenges
6
References
Motivation References Approach Preliminary
Results Problems/Challenges
References
7
References
The idea of using time-dependent parameters for
model structure deficit evaluation is very old
(e.g. review by Beck, 1987). Our work applies
this idea to continuous time models and provides
algorithms to apply it to nonlinear dynamic
systems.
Motivation References Approach Preliminary
Results Problems/Challenges

• This talk is based on
• Brun, PhD dissertation, 2002 First trials with
  filtering algorithm.
• Buser, Masters thesis, 2003 Smoothing, MCMC
  algorithm.
• Tomassini, Reichert, Künsch, Buser, Borsuk,
  2007Estimation of process parameters,
  cross-validation.

8
Approach
Motivation References Approach Preliminary
Results Problems/Challenges
Approach
9
Notation (according to Bayarri et al. 2005)
Input Field data reality plus measurement
error
Motivation References Approach Preliminary
Results Problems/Challenges
Output Field data reality plus measurement
error
An ideal model describes reality
A realistic model approximates an ideal model
All terms together
inputerror
meas.error
effect of model structure error
10
Problem
Motivation References Approach Preliminary
Results Problems/Challenges
inputerror
meas.error
effect of model structure error
Problem The bias term describes the effect, but
not the cause of the model structure error. This
leads to a satisfying statistical description of
the past, but is hard to extra-polate into the
future. For uncertainty reduction and
extrapolation it would be better to reduce the
bias by improving the mechanistic description of
the system. In particular, trends must be
described by the model, not by the bias term. How
can statistical procedures support this?
11
Concept
Motivation References Approach Preliminary
Results Problems/Challenges
inputerror
meas.error
effect of model structure error

• Concept
• Allow model parameters to vary. Add parameters
  where appropriate (input, output).
• Try to reduce the bias by finding an adequate
  behaviour of these parameters.
• Explore dependency of parameter variability on
  external or model variables. If successful (from
  a statistical and physical point of view), modify
  the model structure to reflect this dependency.
• Redo the analysis with improved model structure
  and reduced bias.

12
Use for dynamic models
Motivation References Approach Preliminary
Results Problems/Challenges
inputerror
meas.error
effect of model structure error
Formulation for time dependent models
?xt Correction accounting for input error. ??t
Model-internal correction of model structure
error. ?yt Model-external correction for
remaining effect of model structure error.In the
ideal case, this error could be neglected as it
would be accounted for by the internal correction.
13
Approach
Motivation References Approach Preliminary
Results Problems/Challenges

• Fit model with constant parameters, identify
  presence of bias. If bias exists
• Identify, separately or jointly, time-dependent
• input variation (?xt )
• parameter variation (??t )
• output variation (?yt )
• Identify dependences of time-dependent parameters
  on external or model variables.
• Improve the model structure by deterministic or
  sto-chastic elements (according to statistical
  and physi-cal considerations), try to avoid
  output error (?yt ).
• Use the extended model for understanding and
  prediction.

14
Implementation
The time dependent parameter is modelled by a
mean-reverting Ornstein Uhlenbeck process
Motivation References Approach Preliminary
Results Problems/Challenges
This has the advantage that we can use the
analytical solution
or, after reparameterization
15
Implementation

• We combine the estimation of
• constant model parameters, ?, with
• state estimation of the time-dependent
  parameter(s), ?t, and with
• the estimation of (constant) parameters of the
  Ornstein-Uhlenbeck process(es) of the time
  dependent parameter(s), ?(?,?, ,?to).

Motivation References Approach Preliminary
Results Problems/Challenges
16
Conceptual Framework
Motivation References Approach Preliminary
Results Problems/Challenges
Original deterministic model
Model extended by input- and output-parameter and
measurement error
17
Simplified Framework

• Simplifications
• Omit representation of given measured input, xF.
• Add parameter to input to represent input
  uncertainty by parameter uncertainty.
• Add parameter to output to represent output
  uncertainty by parameter uncertainty.

Motivation References Approach Preliminary
Results Problems/Challenges
18
Numerical Implementation (1)
Gibbs sampling for the three different types of
parameters. Conditional distributions
Motivation References Approach Preliminary
Results Problems/Challenges
simulation model (expensive)
Ornstein-Uhlenbeck process (cheap)
Ornstein-Uhlenbeck process (cheap)
simulation model (expensive)
19
Numerical Implementation (2)
Metropolis-Hastings sampling for each type of
parameter
Motivation References Approach Preliminary
Results Problems/Challenges
Multivariate normal jump distributions for the
parameters q and x. This requires one simulation
to be performed per suggested new value of q.
The discretized Ornstein-Uhlenbeck parameter, ft,
is split into subintervals for which OU-process
realizations conditional on initial and end
points are sampled. This requires the number of
subintervals simulations per complete new time
series of ft.
20
Estimation of Hyperparametersby Cross -
Validation
Motivation References Approach Preliminary
Results Problems/Challenges
Due to identifiability problems we selected the
hyperparameters, x, in a previous application
(Tomassini et al., 2006) alternatively by
cross-validation
21
Preliminary Results
Motivation References Approach Preliminary
Results Problems/Challenges
Preliminary Results for a Simple Hydrologic Model

• Model
• Model Application
• Preliminary Results(based on Markov chains of
  insufficient length)

22
Model
A Simple Hydrologic Watershed Model (1)
Motivation References Approach Preliminary
Results Problems/Challenges
Kuczera et al. 2006
23
Model
A Simple Hydrologic Watershed Model (2)
Motivation References Approach Preliminary
Results Problems/Challenges
7 model parameters 3 initial conditions 1
standard dev. of meas. err. 3 modification
parameters
Kuczera et al. 2006
24
Model
A Simple Hydrologic Watershed Model (3)
Motivation References Approach Preliminary
Results Problems/Challenges
Kuczera et al. 2006
25
Model Application

• Model application
• Data set of Abercrombie watershed, New South
  Wales, Australia (2770 km2), kindly provided by
  George Kuczera (Kuczera et al. 2006).
• Box-Cox transformation applied to model and data
  to decrease heteroscedasticity of residuals.
• Step function input to account for input data in
  the form of daily sums of precipitation and
  potential evapotranspiration.
• Daily averaged output to account for output data
  in the form of daily average discharge.

Motivation References Approach Preliminary
Results Problems/Challenges
26
Model Application
Prior distribution Estimation of constant
parameters Independent uniform distributions for
the loga-rithms of all parameters (73111),
keeping correction factors (frain, fpet) equal to
unity and bias (bQ) equal to zero. Estimation of
time-dependent parameters Ornstein-Uhlenbeck
process applied to log of the parameter (with the
exception of bQ). Hyper-parameters t 5d, s
fixed, only estimation of initial value and mean
(0 for frain, fpet, bQ). Constant parameters as
above.
Motivation References Approach Preliminary
Results Problems/Challenges
27
Preliminary Results (MC of insufficient length)
Posterior marginals
Motivation References Approach Preliminary
Results Problems/Challenges
28
Preliminary Results (MC of insufficient length)
Max. post. simulation with constant parameters
Motivation References Approach Preliminary
Results Problems/Challenges
29
Preliminary Results (MC of insufficient length)
Residuals of max. post. sim. with const. pars.
Motivation References Approach Preliminary
Results Problems/Challenges
30
Preliminary Results (MC of insufficient length)
Residual analysis, max. post., constant
parameters
Motivation References Approach Preliminary
Results Problems/Challenges
Residual analysis, max. post., q_gw_max
time-dependent
31
Preliminary Results (MC of insufficient length)
Residual analysis, max. post., s_F time-dependent

Motivation References Approach Preliminary
Results Problems/Challenges
Residual analysis, max. post., f_rain
time-dependent
32
Preliminary Results (MC of insufficient length)
Time-dependent parameters
Motivation References Approach Preliminary
Results Problems/Challenges
33
Problems / Challenges
Motivation References Approach Preliminary
Results Problems/Challenges
Problems / Challenges ( Working Group
Opportunities)
34
Problems / Challenges

• Problems / Challenges
• Other formulations of time-dependent parameters?
• Dependence on other factors than time.
• How to estimate hyperparameters? (Reduction in
  correlation time always improves the fit.)
• How to avoid modelling physical processes with
  the bias term?
• Learn from more applications.
• Compare results with methodology by Bayarri et
  al. (2005). Combine/extend the two methodologies?
• ?

Motivation References Approach Preliminary
Results Problems/Challenges
35
Problems / Challenges
Motivation References Approach Preliminary
Results Problems/Challenges
Problems / Challenges ( Working Group
Opportunities) Discussion slides from talk at
Oct. 16.
36
Problems / Challenges

• Research Questions / Options for Projects (1)
• Compare results when making different model
  parameters stochastic and time-dependent.
  (Ongoing with a postdoc in Switzerland extending
  earlier work with continuous-time stochastic
  parameters.)
• Develop a better statistical description of
  rainfall uncertainty.(Option for a collaboration
  with climate/weather working groups.)
• Explore alternative options for making parameters
  time-dependent.(Suggestions so far
  storm-dependent parameters, time-dependent
  parameter as an Ornstein-Uhlenbeck process.)

Motivation References Approach Preliminary
Results Problems/Challenges
37
Problems / Challenges

• Research Questions / Options for Projects (2)
• Investigate how to learn from state estimation of
  stochastic hydrological models.(Can the pattern
  of state adaptations lead to insights of model
  structure deficits or input errors?)
• Develop uncertainty estimates when using
  multi-objective optimization.(How to use
  information on Pareto set for uncertainty
  estimation of parameters and results?)
• Analyse differences in results of suggested
  approaches when using different models.(Is there
  a generic behaviour of different techniques when
  they are applied to different models/data sets?)

Motivation References Approach Preliminary
Results Problems/Challenges
38
Problems / Challenges

• Research Questions / Options for Projects (3)
• Improve the efficientcy of posterior maximisation
  and posterior sampling.(Efficiency becomes
  important when having complex watershed models in
  mind. Efficient global optimizers and sampling
  from multi-modal posterior distributions becomes
  then important.)
• More questions will come up during discussions.

Motivation References Approach Preliminary
Results Problems/Challenges
39
Thank you for your attention