Autoregressive AR Models

1
Autoregressive (AR) Models

• The ACF of an AR(p) model decays towards zero as
  the lag k increases
• The autocorrelations can either alternate signs
  or be all positive, but in all cases they show a
  decline towards zero as the lag increases
• Implication We can identify an AR(p) model as
  the underlying model of a time series variable
  when the sample autocorrelations decay towards
  zero as the lag k increases

2
Partial Autocorrelations

• If we identify an AR(p) model as the underlying
  model of a time series variable from the behavior
  of sample autocorrelations, how can we tell what
  the order p is?
• The sample partial autocorrelations provide an
  easy way to recognize the order of an AR(p) model
• The partial autocorrelation shows the correlation
  between observations of the variable that are k
  periods apart without including the effect of the
  correlations between shorter lags

3
Partial Autocorrelations

• Result The partial autocorrelation of an AR(p)
  model is non-zero up to order p, but it is zero
  for orders higher than p
• Example If we examine the partial
  autocorrelations of a variable and we observe
  that they are non-zero up to lag 3, but then
  become zero, then we can say that the AR(3) model
  is an appropriate model for this variable

4
Two Examples of Partial Autocorrelations
5
ACF and PACF for Oil Prices AR(2) Model
6
Moving Average (MA) Models

• In a moving average model, the dependent (time
  series) variable Yt is explained by current and
  past values of the stochastic errors
• An MA(1) model is given by
• A moving average model of order q, or MA(q)
  model, will include q lags of the stochastic
  errors as explanatory variables of the time
  series variable Yt

7
Moving Average (MA) Models

• How can we determine the order q in a moving
  average model?
• Result In a MA(q) model, all autocorrelations of
  order (or lag) higher than q are equal to zero
• The partial autocorrelations of a moving average
  process do not abruptly cut off, but decay
  towards zero
• Example If from the ACF of variable Yt we
  observe that the autocorrelations up to lag 4 are
  non-zero, but those beyond lag 4 are zero, then a
  MA(4) model is an appropriate model

8
ACF and PACF of First Difference in Market
Returns MA(1) Model
9
Summarizing Results

• Results of AR(p) model
• The autocorrelations of an AR model decay towards
  zero as the lag k increases
• The partial autocorrelations of an AR(p) process
  cut off abruptly, meaning they are all zero for
  lag k greater than p
• Results of MA(q) model
• The autocorrelations of a MA(q) model cut off
  abruptly, meaning they are all zero for k greater
  than q
• The partial autocorrelations of a moving average
  process decay toward zero as the lag k increases

10
Autoregressive Moving Average (ARMA) Models

• ARMA models include lagged values of both the
  dependent time series variable and the stochastic
  errors on the right-hand side
• These models are very good representations of
  stationary variables using relatively few unknown
  parameters (meaning p and q) to be estimated
• An ARMA (p,q) model includes p lagged dependent
  variables and q lagged error terms

11
Autoregressive Moving Average (ARMA) Models

• Example The ARMA(2,3) model is given by
• Pure autoregressive and pure moving average
  models can be viewed as special cases of ARMA
  models
• Example Setting q 0 in an ARMA (p,q) model
  gives us the AR(p) model alternatively, setting
  p 0 gives us the MA(q) model

12
Autoregressive Moving Average (ARMA) Models

• Given that the ARMA(p,q) models have both lagged
  values of the dependent variable and lagged error
  terms, it is more difficult to identify them by
  examining the ACF and PACF
• For instance, if ARMA(p,0) then the ACF decays
  towards zero rather than cutting off abruptly
• However, if ARMA(0,q) then the PACF decays
  towards zero rather than cutting off abruptly

13
Example of ARMA Model Oil Price Changes

• If we observe the ACF of the variable that shows
  oil price changes, we note that the
  autocorrelations are not different from zero
  after lag 3
• This is an indication of a MA(3) model that would
  best describe the variable

14
Example of ARMA Model Oil Price Changes

• If we also examine the PACF of the oil price
  change variable, we note that the partial
  autocorrelations are not different from zero
  after lag 2 or 3
• That would indicate that an AR(2) or AR(3) model
  would best describe this variable

15
Example of ARMA Model Oil Price Changes

• Given the above two potential model
  specifications, it seems reasonable to also
  consider two or three mixed models, for example
  ARMA(2,1) and ARMA(1,2)
• More generally, we could consider a number of
  models and see which one works better based on
  some model evaluation criteria that we will
  discuss later

16
Nonstationary Time Series VariablesARIMA Models

• Many financial and economic time series exhibit
  nonstationarity stock market indices, exchange
  rates, interest rates, commodity prices
• However, many of those variables can be converted
  to stationary variables by taking first or second
  differences
• Example If variable Yt is nonstationary it is
  highly likely that either ?YtYt Yt-1 or the
  first difference of ?Yt will be stationary

17
Nonstationary Time Series VariablesARIMA Models

• ARIMA models can be used to describe
  nonstationary time series variables that can be
  converted into stationary ones after some
  differencing (usually first or second
  differences)
• An ARIMA(p, d, q) is a model with p lagged values
  of the dependent variable and q lagged values of
  the error term for a variable that becomes
  stationary after d degrees of differencing
• Example An ARIMA(1, 1, 0) is a model with one
  lagged value of the variable Yt (as in the case
  of an AR(1) model), no lagged values of the error
  term and the variable Yt becomes stationary after
  taking first differences

18
Selecting a Model Fitting ARIMA Models to the
Data

• In selecting a model to represent the time series
  data, we should follow the principle of parsimony
• Principle of parsimony In searching for a model,
  we should not try to look for a very elaborate
  model, but for the simplest model that appears to
  adequately represent the data
• Implication In making our initial choice of
  model, we should attempt to proceed, if possible,
  with a model containing just a very small number
  of parameters (lowest possible number of p and q)
  

19
Selecting a Model Fitting ARIMA Models to the
Data

• The steps to follow in selecting an ARIMA model
  are
• Is our variable stationary or not? How many
  differences does it take to convert it into a
  stationary variable? This determines the degree
  of differencing (d)
• Examine the sample autocorrelations of the
  variable and if they decay slowly, then the
  variable is nonstationary
• Select the autoregressive and moving average
  orders (p) and (q)
• Use the summary results for the two models(AR(p)
  and MA(q)) based on the pattern of the ACF and
  PACF to determine these orders (see notes a few
  slides back)
• If there are clear indications of an AR(p) or a
  MA(q) model, use either of those models

20
Selecting a Model Fitting ARIMA Models to the
Data

• For ARMA(p, q) models, identification is
  trickier, but very often good representations of
  time series are available with small values of p
  or q or both
• In general, we would like to estimate a number of
  models with various combinations of p and q while
  keeping pq lt 5 (based on the principle of
  parsimony, we want to keep the model simple)
• Evaluate the performance of the models based on
  the Akaike Information Criterion (AIC) and the
  Schwartz Bayesian Criterion (SBC) (preference
  will be on the SBC criterion)

21
Selecting a Model Fitting ARIMA Models to the
Data

• Select the model with the lowest value of the
  criterion
• If there is no clear answer based on the model,
  then we examine if any of the 2-3 models with the
  lowest SBC values have insignificant
  coefficient(s), which means that they will be
  eliminated
• Use the estimated model for forecasting future
  values of the dependent time series variable
• Evaluate the forecasts based on a forecasting
  confidence interval
• Note that selecting the right model can be more
  of an art rather than a straightforward process

22
Example Fitting a Model of Oil Prices

• Examining first the ACF, we observe that there is
  a gradual decay
• This implies nonstationarity, so we can take
  first differences and examine the ACF again to
  see if there is now evidence of stationarity

23
Example Fitting a Model of Oil Prices

• The ACF of the first differences shows that
  autocorrelations become zero very quickly
• This is an indication of stationarity
• So, in the ARIMA model, we use d 1
• We next examine the ACF and PACF of the variable
  of first differences in oil prices

24
ACF and PACF of First Differences of Oil Prices
25
Example Fitting a Model of Oil Prices

• As noted previously, the ACF and PACF indicate
  the possibility of a MA(3) and an AR(2) model
• Thus, we could estimate a number of combinations
  of ARIMA models and select the best based on the
  SBC criterion
• For example, estimate the following models
• ARIMA(1,1,0)
• ARIMA(0,1,1)
• ARIMA(2,1,0)
• ARIMA(0,1,2)
• ARIMA(0,1,3)
• ARIMA(1,1,1)

26
Fitting a Model of Oil PricesForecasts from
ARIMA (2, 1, 0)