Title: Kevin (Minxue) He
1An Integrated Uncertainty and Ensemble Data
Assimilation Approach for Improved Operational
Streamflow Predictions
NOAA/NWS Office of Hydrologic Development
(OHD) Riverside Technology, Inc.
Acknowledgements Haksu Lee (OHD), Yuqiong Liu
(NASA GSFC) Andy Wood and Stacie Bender (CBRFC),
Terri Hogue and Steven Margulis (UCLA)
NOAA/NWS/NCEP/EMC, Camp Springs, MD September
16, 2011
2Outline
- Introduction
- Hydrologic forecasting in US
- Hydrologic Ensemble Forecast System
- Focus of this presentation
- Methodology
- Integrated unCertainty and Ensemble data
Assimilation (ICEA) - Study area and experimental design
- Verification metrics
- Results
- Simulation results and parameter uncertainty
- Prediction results
- Conclusions and ongoing work
2/24
3Hydrologic Forecasting in US
- Focus streamflow
- Two components
- Forecast model
- Non-snow basins rainfall-runoff model (SAC-SMA)
- Snow-covered basins snow model (SNOW17)
SAC-SMA - Forcing numerical weather prediction models
(NCEP) - Products
I. Deterministic (4,925 sites, short-term)
II. Ensemble (climatology forcing, seasonal)
3
4Hydrologic Ensemble Forecast System (HEFS)
Hydrologic Ensemble Forecast System (HEFS)
Atmospheric Ensemble Pre-Processor
GFS/GEFS, CFS/CFSv2, NAEFS, SREF
Data Assimilator
SNOW17 SAC-SMA
Hydrology and Water Resources Models
Hydrologic Ensemble Post-Processor
Hydrology and Water Resources Ensemble Product
Generator
Forecasters
Users
5Data Assimilator Prototypes of the HEFS
- Deterministic techniques (Variational method
(VAR)) - 1D-VAR, for routing model
- 2D-VAR, for lumped SAC-SMA model
- 4D-VAR, for distributed SAC-SMA model (4 km
resolution) - Ensemble techniques (for SNOW17 SAC-SMA)
- Ensemble Kalman Filter (EnKF)
- Ensemble Kalman Smoother (EnKS)
- Integrated unCertainty and Ensemble data
Assimilation (ICEA) - Hybrid deterministic and ensemble technique
- Maximum Likelihood Ensemble Filter (MLEF) for
SAC-SMA
5
6Outline
- Introduction
- Hydrologic forecasting in US
- Hydrologic Ensemble Forecast System
- Focus of this presentation
- Methodology
- Integrated unCertainty and Ensemble data
Assimilation (ICEA) - Study area, modeling procedure, and scenarios
- Verification metrics
- Results
- Simulation results and parameter uncertainty
- Prediction results
- Conclusions and ongoing work
6/24
7ICEA
- Model in a systematic view
- ICEA Uncertainty Analysis Ensemble DA
(ISURFEnKF) - Part 1 Uncertainty analysis (He, 2010 He et
al., 2011a, b) - Integrated Sensitivity and UnceRtainty analysis
Framework (ISURF) - Sensitivity analysis screening tool
- Uncertainty analysis Markov Chain Monte Carlo
technique - ? Parameter uncertainty info. the optimal
parameter set
7
8ICEA
- ICEA Uncertainty Analysis Ensemble DA
(ISURFEnKF) - Part 2 EnKF (He, 2010 He et al., 2011c)
I. Basis Bayes theorem
II. EnKF approximates the Bayesian updating
scheme using a Monte Carlo approach
8
9Study Area
SNOw TELemetry (SNOTEL) network 800 sites,
Natural Resources Conservation Service (NRCS),
daily snow observations (e.g., snow water
equivalent (SWE))
9
10Experimental Design
- Areal SWE for upper sub-basin
- Non-negative least-squares algorithm
- Study period Training (water year 1979-1984)
Prediction (1991-1996) - Modeling procedure
(obs. MAT/P other than FMAT/P used)
Steps to implement ICEA
Step 1 ISURF ?SNOW17/SAC-SMA, upper ? Para. Unc.
(training period) Step 2 EnKF ? SNOW17 model,
upper ? assimilate areal SWE (prediction
period) Step 3 Lower, RFC para. ? flow upper
flow? UH routing? outlet flow
10
11Experimental Design
- Scenarios
- S1 RFC parameters S2 ISURF optimal parameters
(S1 S2 deterministic) - S3 Stand-alone EnKF S4 ICEA
(S3 S4 ensemble) - Similarities between S3 and S4 (sensitivity tests
conducted in He, 2010) - Precipitation uncertainty
- Temperature uncertainty
- Measurement uncertainty
- Ensemble size 100
- Assimilation frequency every week
- No uncertainty assumed in initial condition
(start date Oct. 1, no snowpack) - Difference between S3 and S4
- Parameter uncertainty ranges
- S3 entire feasible para. range S4
ISURF-derived optimal para. range
11
12Verification Metrics
- Deterministic metrics
- Correlation (R), Percent Bias, RMSE,
Nash-Sutcliffe Efficiency (NSE) - (for S3 S4, ensemble mean is used when
calculating above metrics) - Ensemble metrics
- Normalized RMSE Ratio (NRR)
- 95th Percentile Uncertainty Ratio (UR95)
- Measure of ensemble dispersion
- Value 1 (perfect)
- gt 1 (little spread)
- lt 1 (much spread)
- (Anderson, 2002)
Aggregated variability of prediction relative to
observation Range 0-100, perfect value 0
(Hossain and Anagnostou, 2005)
12
13Outline
- Introduction
- Hydrologic forecasting in US
- Hydrologic Ensemble Forecast System
- Focus of this presentation
- Methodology
- Integrated unCertainty and Ensemble data
Assimilation (ICEA) - Study area, modeling procedure, and scenarios
- Verification metrics
- Results
- Simulation results and parameter uncertainty
- Prediction results
- Conclusions and ongoing work
13/24
14Simulation Results
- Annual statistics of simulated and observed
streamflow during the training period 1979-1984
(S1 S2)
- ISURF-derived optimal parameters outperform RFC
parameters -
- ISURF-derived parameter uncertainty information ?
trustable
14
15Parameter Uncertainty
- ISURF identifies four sensitive parameters, their
marginal distributions (in bars) and correlation
structure (in dots)
15
16Prediction Results
- Overall performance (entire prediction period)
Bias
RMSE
- RFC prediction can be improved via advanced
calibration (e.g., ISURF) -
- DA has added value ? RFC/advanced calibration
methods - ICEA outperforms EnKF
R
NSE
16
17Prediction Results
- Performance on high flow (gt95th percentile)
RFC ISURF EnKF ICEA
R 0.80 0.85 0.83 0.87
Bias () -19.58 -13.70 -14.91 -10.02
RMSE (m3/s) 65.37 56.47 58.33 49.59
NSE 0.50 0.62 0.60 0.71
- Scatter Plot ICEA mean (best), RFC (worst) DA
methods provide ensemble info. -
- Statistics ICEA (best) RFC (worst) ISURF
EnKF comparable
17
18Prediction Results
- EnKF vs. ICEA ensemble statistics (annual)
NRR measure of ensemble dispersion (perfect
value 1 too little
spread when gt1)
- comparable, but not enough spread
UR95 variability relative to observations
(perfect value 0)
- Overall, ICEA has less variability but not in
1993, 1995, and 1996
18
19Prediction Results
- EnKF vs. ICEA finer resolution (daily)
I. Selection of the wettest year,1995, for
demonstration
II. EnKF and ICEA flow predictions in this year
- RFC misses peak/recession
- Both ens. capture peak flow high flows spread
is narrow - EnKF ens. wide in early melting period, but
underestimate later ? melting parameter samples
19
20Prediction Results
- EnKF vs. ICEA performance at various lead times
- Deterministic metrics (a-d) overall, ICEA
outperforms EnKF in all lead days - ICEA ensemble less variability (e) all lead
days comparable dispersion (f)
20
21Outline
- Introduction
- Hydrologic forecasting in US
- Hydrologic Ensemble Forecast System
- Focus of this presentation
- Methodology
- Integrated unCertainty and Ensemble data
Assimilation (ICEA) - Study area, modeling procedure, and scenarios
- Verification metrics
- Results
- Simulation results and parameter uncertainty
- Prediction results
- Conclusions and ongoing work
21/24
22Conclusions
- Simulation ISURF optimal para. outperform RFC
para. - Parameter uncertainty 4 sensitive para. 3
normal, 1 uniform - Prediction
- DA methods (EnKF/ICEA) provide improved flow
predictions (vs. RFC/ISURF) and ensemble
predictions - ICEA ensemble mean prediction ? best in overall
performance high flow prediction in 4
scenarios better in all lead days vs. EnKF mean
prediction - ICEA ensemble predictions generally have less
variability comparable dispersion vs. EnkF
ones, both on annual basis and at various lead
days - ICEA and EnKF ensembles capture high flows, but
too narrow
Take Home Message
ICEA has the
potential to supplement the current operational
method in 1) providing improved single-valued
(ens. mean) forecasts 2) meaningful ensemble
forecasts.
22
23Ongoing and Future Work
- Enhance the experimental prototype (ICEA) by
- Investigating ensemble initialization
(uncertainty in I.C.) - Verifying ensembles via other metrics
(reliability, resolution) - Considering model structural uncertainty (He et
al., 2011a) - Evaluating it against the operational snow
updating system used at CBRFC across multiple
watersheds - Evaluate the enhanced prototype in real-time
forecasting - To digest forecasted ensemble forcing (e.g.
processed GFS/PQPF) with educated perturbations
of I.C. ? predictions with wider spread
23
24Thank youQuestions?
Contact Kevin.He_at_noaa.gov
- References
- He, M. (2010) Data assimilation in watershed
models for improved hydrologic forecasting, Ph.D.
Dissertation, Civil and Environmental
Engineering, University of California, Los
Angeles, 173 pp. - He, M., Hogue, T. S., Franz, K. J., Margulis,
S. A., and Vrugt, J. A. (2011a) Corruption of
parameter behavior and regionalization by model
and forcing data errors A Bayesian example using
the SNOW17 model, Water Resour. Res., 47,
10.1029/2010WR009753. - He, M., Hogue, T. S., Franz, K. J., Margulis,
S. A., and Vrugt, J. A. (2011b) Characterizing
parameter sensitivity and uncertainty for a snow
model across hydroclimatic regimes, Adv. Water
Resour., 34, 114-127. - He, M., Hogue, T. S., Margulis, S. A, and
Franz, K. J. (2011c) An integrated uncertainty
and ensemble-based data assimilation approach for
improved operational streamflow predictions,
Hydrol. Earth Syst. Sci. Discuss., 8, 7709-7755,
10.5194/hessd-8-7709-2011.
24
25Extra Slides
26ISURF
ISURF a step-wise framework (He, 2010 He et
al., 2011b) Step 1 Generalized sensitivity
analysis (GSA) (Spear and Hornberger,
1980) screening tool ? sensitive
parameters Step 2 Differential Evolution
Adaptive Metropolis (DREAM) (Jasper et al.,
2008) ? parameter uncertainty
27ISURF Methodology
- GSA
- Identify feasible parameter ranges
- Monte Carlo sampling Latin Hypercube Sampling
(sample size ) - Behavioral /non-behavioral classification
- Nash-Sutcliffe efficiency (NSE)0.3
(Garbrecht, 2006) ? - Bin division and CDF calculation
- - ? bins
- - CDF of NSE for each bin
- Kolmogorov-Smirnov test
- ? KS value
- (Kottegoda and Rosso, 1997)
28ISURF Methodology
DREAM
Candidate point
Modify proposal
Metropolis acceptance Prob.
(Jasper et al., 2008 He et al., 2011a, b)
29SNOW17 Model
Key notes 1. Ppt forcing SCFPptPpt forcing
TairgtPXTEMP rain TairltPXTEMP snow 2.
Non-rain melt M N(Tair-MBASE) 3.
Rain-on-snow melt M UADJ(Tair-32)
Ppt precipitation Tair air temperature
30SAC-SMA Model
RIVA
Upper zone
Lower zone