Anthony Barnston, Lisa Goddard, Simon Mason and Andrew Robertson

1
Consolidation of Predictions of Seasonal Climate
by Several Atmospheric General Circulation Models
at IRI

• Anthony Barnston, Lisa Goddard, Simon Mason and
  Andrew Robertson
• International Research Institute
• for Climate and Society (IRI)

2
IRI DYNAMICAL CLIMATE FORECAST SYSTEM
2-tiered OCEAN
ATMOSPHERE
GLOBAL ATMOSPHERIC MODELS ECPC(Scripps)
ECHAM4.5(MPI) CCM3.6(NCAR)
NCEP(MRF9) NSIPP(NASA)
COLA2 GFDL
PERSISTED GLOBAL SST ANOMALY
Persisted SST Ensembles 3 Mo. lead
10
POST PROCESSING MULTIMODEL ENSEMBLING -Bayesian -
Caninical variate
24
24
FORECAST SST TROP. PACIFIC THREE
scenarios (multi-models, dynamical
and statistical) TROP. ATL, INDIAN
(ONE statistical) EXTRATROPICAL (damped
persistence)
12
Forecast SST Ensembles 3/6 Mo. lead
24
24
30
12
30
30
3
IRI DYNAMICAL CLIMATE FORECAST SYSTEM

2-tiered OCEAN
ATMOSPHERE
GLOBAL ATMOSPHERIC MODELS ECPC(Scripps)
ECHAM4.5(MPI) CCM3.6(NCAR)
NCEP(MRF9) NSIPP(NASA)
COLA2 GFDL
PERSISTED GLOBAL SST ANOMALY
FORECAST SST TROP.
PACIFIC THREE scenarios 1) CFS prediction
2) LDEO prediction 3) Constructed
Analog prediction TROP. ATL, and INDIAN
oceans CCA, or slowly damped persistence
EXTRATROPICAL damped persistence
4
Six GCM Precip. Forecasts, JAS 2000
5
RPSS Skill of Individual Models JAS 1950-97
6
Goals To combine the probability forecasts of
several models, with relative weights based on
the past performance of the individual
models To assign appropriate forecast
probability distribution e.g. damp
overconfident forecasts toward climatology
7
Probabilities and Uncertainty
Climatological Probabilities
GCM Probabilities
k tercile number t forecast time m no. ens
members
Above Normal
1/3
6/24
Tercile boundaries are identified for the models
own climatology, by aggregating all years and
ensemble members. This corrects overall bias.
1/3
Near-Normal
8/24
Below Normal
1/3
10/24
8
Bayesian Model Combination
Combine climatology forecast (prior) and an
AGCM forecast, with its evidence of historical
skill, to produce weighted (posterior) forecast
probabilities, by maximizing the historical
likelihood score.
9
Aim to maximize the likelihood score
ktercile category tyear number
The multi-year product of the probabilities that
were hindcast for the category that was
observed. (Could maximize other scores, such as
RPSS) Prescribed, observed SST used to force
AGCMs. Such simulations used in absence of ones
using truly forecasted SST for at least half of
AGCMs.
10
1. Calibration of each model, individually,
against climatology
Optimize likelihood score
ktercile category (1,2, or
3) tyear number jmodel number (1
to 7) wweight
for climo (c) or for model j
PMMkt weighted linear comb of Pjkt over all
j, normalized by S(wj)
2. Calibration of the weighted model combination
against climatol
Optimize likelihood score
where wMM uses wj proportional to results of the
first step above
11
Algorithm used to maximize the designated
score Feasible Sequential Quadratic Programming
(FSQP) Nonmonotone line search for minimax
problems
C M TOTAL NUMBER OF CONSTRAINTS. C
ME NUMBER OF EQUALITY CONSTRAINTS. C
MMAX ROW DIMENSION OF A. MMAX MUST BE AT
LEAST ONE AND GREATER C THAN M. C
N NUMBER OF VARIABLES. C NMAX
ROW DIMENSION OF C. NMAX MUST BE GREATER OR EQUAL
TO N. C MNN MUST BE EQUAL TO M N
N. C C(NMAX,NMAX) OBJECTIVE FUNCTION MATRIX
WHICH SHOULD BE SYMMETRIC AND C
POSITIVE DEFINITE. IF IWAR(1) 0, C IS SUPPOSED
TO BE THE C CHOLESKEY-FACTOR OF
ANOTHER MATRIX, I.E. C IS UPPER C
TRIANGULAR. C D(NMAX) CONTAINS THE CONSTANT
VECTOR OF THE OBJECTIVE FUNCTION. C
A(MMAX,NMAX) CONTAINS THE DATA MATRIX OF THE
LINEAR CONSTRAINTS. C B(MMAX) CONTAINS THE
CONSTANT DATA OF THE LINEAR CONSTRAINTS. C
XL(N),XU(N) CONTAIN THE LOWER AND UPPER BOUNDS
FOR THE VARIABLES. C X(N) ON RETURN, X
CONTAINS THE OPTIMAL SOLUTION VECTOR. C U(MNN)
ON RETURN, U CONTAINS THE LAGRANGE
MULTIPLIERS. THE FIRST C M POSITIONS
ARE RESERVED FOR THE MULTIPLIERS OF THE M C
LINEAR CONSTRAINTS AND THE SUBSEQUENT ONES
FOR THE C MULTIPLIERS OF THE LOWER
AND UPPER BOUNDS. ON SUCCESSFUL C
TERMINATION, ALL VALUES OF U WITH RESPECT TO
INEQUALITIES C AND BOUNDS SHOULD BE
GREATER OR EQUAL TO ZERO.
12
Circumventing the effects of sampling variability

• Sampling variability appears to be an issue
  noisy weight distribution with large number of
  zero weights and some unity weights
• Bootstrap the optimization, omitting contiguous
  6-year blocks of the 48-yr time series
• yields 43 samples of 42 years
• shows the sampling variability of the likelihood
  over subsets of years
• We average the weights across the samples

13
Example

• Six GCMs Jul-Aug-Sep precipitation simulations
• Training period 195097
• Ensembles of between 9 and 24 members

14
Model Weights initially, by individual model
15
Climatological Weights Multi-model
16
Model Weights after second (damping) step
17
Model Weights step 2, and Averaged over
Subsamples
18
For more spatially smooth results, the weighting
of each grid point is averaged with that of its 8
neighbors, using binomial weighting. X X X X
X X X X X
19
Climatological Weights
20
Combination Forecasts of July-Sept Precipitation
After first stage only
After second (damping) stage
After spatial smoothing
After sampling subperiods
21
ReliabilityJAS Precip., 30S-30N
Above-Normal
Below-Normal
Bayesian
Pooled
Observed relative Freq.
Observed relative Freq.
Individual AGCM
Forecast probability
Forecast probability
(3-model)
from Goddard et al. 2003
22
RPSS Precip.
from Roberson et al. (2004) Mon. Wea. Rev.,
132, 2732-2744
23
RPSS 2-m Temp.
from Roberson et al. (2004) Mon. Wea. Rev.,
132, 2732-2744
24
Conclusions - Bayesian

• The climatological (equal-odds) forecast
  provides a useful prior for combining multiple
  ensemble forecasts
• Sampling problems become severe when attempting
  to combine many models from a short training
  period (noisy weights)
• A two-stage process combines the models together
  according to a pre-assessment of each against
  climatology
• Smoothing of the weights across data sub-samples
  and spatially appears beneficial

25
IRIs forecasts use also a second
consolidation scheme, whose result is averaged
with the result of the Bayesian scheme. 1.
Bayesian scheme 2. Canonical Variate scheme
26

• Canonical Variate Analysis (CVA)
• A number of statistical techniques involve
  calculating linear combinations (weighted sums)
  of variables. The weights are defined to achieve
  specific objectives
• PCA weighted sums maximize variance
• CCA weighted sums maximize correlation
• CVA weighted sums maximize discrimination

27
Canonical Variate Analysis
28
Canonical Variate Analysis
The canonical variates are defined to maximize
the ratio of the between-category (separation
between the crosses) to the within-category
(separation of dots from like-colored crosses)
variance.
29
Conclusion IRI presently using a 2-tiered
prediction system. It is interested in using
fully coupled systems also, and is looking into
incorporating those. Within its 2-tiered system
it uses 4 SST prediction scenarios, and combines
the predictions of 7 AGCMs. The merging of 7
predictions into a single one uses two
multi-model ensemble systems Bayesian
and canonical variate. These give somewhat
differing solutions, and are presently given
equal weight.