Exponential smoothing: The state of the art

1
Exponential smoothingThe state of the art
Part IIEverette S. Gardner, Jr.
2
Exponential smoothingThe state of the art
Part II

• History
• Methods
• Properties
• Method selection
• Model-fitting
• Inventory control
• Conclusions

3
Timeline of Operations Research (Gass, 2002)

• 1654 Expected value, B. Pascal
• 1733 Normal distribution, A. de Moivre
• 1763 Bayes Rule, T. Bayes
• 1788 Lagrangian multipliers, J. Lagrange
• 1795 Method of Least Squares, C. Gauss, A.
  Legendre
• 1826 Solution of linear equations, C. Gauss
• 1907 Markov chains, A. Markov
• 1909 Queuing theory, A. Erlang
• 1936 The term OR first used in British military
  applications
• 1941 Transportation model, F. Hitchcock
• 1942 U.K. Naval Operational Research, P.
  Blackett
• 1943 Neural networks, W. McCulloch, W. Pitts
• 1944 Game theory, J. von Neumann, O.
  Morgenstern
• 1944 Exponential smoothing, R. Brown

4
Exponential smoothing at work

• A depth charge has a magnificent laxative
  effect on a submariner.
• Lt. Sheldon H. Kinney,
• Commander,
• USS Bronstein (DE 189)

5
Forecast Profiles

• N A
  M
• 
  None Additive
  Multiplicative
• N
• None
• A
• Additive
• DA
• Damped Additive
• M
• Multiplicative
• DM
• Damped Multiplicative

6
Damped multiplicative trends (Taylor, 2002)
Damping parameter
7
Variations on the standard methods

• Multivariate series (Pfefferman Allen, 1989)
• Missing or irregular observations (Wright,1986)
• Irregular update intervals (Johnston, 1993)
• Planned discontinuities (Williams Miller, 1999)
• Combined level/seasonal component (Snyder
  Shami, 2001)
• Multiple seasonal cycles (Taylor, 2003)
• Fixed drift (Hyndman Billah, 2003)
• Smooth transition exponential smoothing (Taylor,
  2004)
• Renormalized seasonals (Archibald Koehler,
  2003)
• SSOE state-space equivalent methods (Hyndman et
  al., 2002)

8
Smoothing with a fixed drift (Hyndman Billah,
2003)

• Equivalent to the Theta method?
• (Assimakopoulos and Nikolopoulos, 2000)
• How to do it
• Set drift equal to half the slope of a regression
  on time
• Then add a fixed drift to simple smoothing, or
• Set the trend parameter to zero in Holts linear
  trend
• When to do it
• Unknown

9
Adaptive simple smoothing (Taylor, 2004)

• Smooth transition exponential smoothing (STES) is
  the only adaptive method to demonstrate credible
  improved forecast accuracy
• The adaptive parameter changes according to a
  logistic function of the errors
• Model-fitting is necessary

10
Renormalization of seasonals

• Additive (Lawton, 1998)
• Without renormalization
• Level and seasonals are biased
• Trend and forecasts are unbiased
• Renormalization of seasonals alone
• Forecasts are biased unless renormalization is
  done every period
• Multiplicative (Archibald Koehler, 2003)
• Competing renormalization methods give forecasts
  different from each other and from unnormalized
  forecasts

11
Archibald Koehler (2003) solution

• Additive and multiplicative renormalization
  equations that give the same forecasts as
  standard equations
• Cumulative renormalization correction factors for
  those who wish to keep the standard equations

12
Continental Airlines Domestic Yields
Model Restarted
13
Standard vs. state-space methods

• Trend damping
• Standard Immediate
• State-space Starting at 2 steps ahead
• Multiplicative seasonality
• Standard Seasonal component depends on level
• State-space Independent components
• Model fitting
• Standard Minimize squared errors
• State-space Minimize squared relative errors if
  multiplicative errors are assumed.

14
Properties

• Equivalent models
• Prediction intervals
• Robustness

15
Equivalent models

• Linear methods
• ARIMA
• DLS regression
• Kernel regression (Gijbels et al.,1999 Taylor,
  2004)
• MSOE state-space models (Harvey, 1984)
• All methods
• SSOE state-space models (Ord et al.,1997)

16
Analytical prediction intervals

• Options
• SSOE models (Hyndman et al., 2005)
• Model-free (Chatfield Yar, 1991)
• Empirical evidence
• None

17
Empirical prediction intervals

• Options
• Chebyshev distribution (fitted errors) (Gardner,
  1988)
• Quantile regression (fitted errors) (Taylor
  Bunn, 1999)
• Parametric bootstrap (Snyder et al., 2002)
• Simulation from assumed model (Bowerman,
  OConnell, Koehler, 2005)
• Empirical evidence
• Limited, but encouraging

18
Robustness

• Many equivalent models for each method (Chatfield
  et al., 2001 Koehler et al., 2001)
• Simple ES performs well in many series that are
  not ARIMA (0,1,1) (Cogger,1973)
• Aggregated series can often be approximated by
  ARIMA (0,1,1) (Rosanna Seater, 1995)

19
Robustness (continued)

• Exponentially declining weights are robust (Muth,
  1960 Satchell Timmerman, 1995)
• Additive seasonal methods are not sensitive to
  the generating process (Chen,1997)
• The damped trend includes numerous special cases
  (Gardner McKenzie,1988)

20
Automatic forecasting with the damped additive
trend
? .84
? .38
? 1.00
21
Summary of 66 empirical studies,1985-2005

• Seasonal methods rarely used
• Damped trend rarely used
• Multiplicative trend never used
• Little attention to method selection
• But exponential smoothing was robust, performing
  well in at least 58 studies

22
Method selection

• Benchmarking
• Time series characteristics
• Expert systems
• Information criteria
• Operational benefits
• Identification vs. selection

23
Benchmarking in method selection

• Methods should be compared to reasonable
  alternatives
• Competing methods should use exactly the same
  information
• Forecast comparisons should be genuinely out of
  sample

24
Method selection Time series characteristics

• Variances of differences (Gardner
  McKenzie,1988)
• Seemed a good idea at the time
• Discriminant analysis (Shah,1997)
• Considered only simple smoothing and a linear
  trend
• Should be tested with an exponential smoothing
  framework
• Regression-based performance index (Meade, 2000)
• Considered every feasible time series model
• Should be tested with an exponential smoothing
  framework

25
Method selection Expert systems

• Rule-based forecasting
• Original version (Collopy Armstrong, 1992)
• Automatic version (Vokurka et al., 1996)
• Streamlined version (Adya et al., 2001)
• Other rule-induction systems
• (Arinze,1994 Flores Pearce, 2000)
• Expert systems are no better than aggregate
  selection of the damped trend alone (Gardner,
  1999)

26
Method selection AIC

• Damped trend vs. state-space models selected by
  AIC
• Average of all forecast horizons

MAPE
Asymmetric MAPE
27
Method selectionEmpirical information criteria
(EIC)

• Strategy Penalize the likelihood by linear and
  nonlinear functions of the number of parameters
  (Billah et al., 2005)
• Evaluation EIC superior to other information
  criteria, but results are not benchmarked

28
Method selection Operational benefits

• Forecasting determines inventory costs, service
  levels, and scheduling and staffing efficiency.
• Research is limited because a model of the
  operating system is needed to project performance
  measures.

29
Method selection Operational benefits (cont.)

• Manufacturing (Adshead Price, 1987)
• Producer of industrial fasteners (4 million
  annual sales)
• Costs holding, stockout, overtime
• U.S. Navy repair parts (Gardner, 1990)
• 50,000 inventory items
• Tradeoffs Backorder delays vs. investment
• Savings 30 million (7) in investment

30
Average delay in filling backorders
31
Inventory analysis Packaging materials for
snack-food manufacturer
Actual Inventory from subjective forecasts
Month
Target maximum inventory based on damped trend
Month
Monthly Usage
32
Method selection Operational benefits (cont.)

• Electronics components (Flores et al., 1993)
• 967 inventory items
• Costs holding cost vs. margin on lost sales
• RAF repair parts (Eaves Kingsman, 2004)
• 11,203 inventory items
• Tradeoffs inventory investment vs. stockouts
• Savings 285 million (14) in investment

33
Forecasting for inventory controlCumulative
lead-time demand

• SSOE models yield standard deviations of
  cumulative lead-time demand (Snyder et al., 2004)
• Differences from traditional expressions (such as
  ) are significant

34
Standard deviation multipliers, a 0.30
Lead time
35
Forecasting for inventory controlCumulative
lead-time demand (cont.)

• The parametric bootstrap (Snyder et al., 2002)
  can estimate variances for
• Any seasonal model
• Non-normal demands
• Intermittent demands
• Stochastic lead times

36
Forecasting for inventory controlIntermittent
demand

• Crostons method (Croston, 1972)
• Smoothed nonzero demand
• Mean demand
• Smoothed inter-arrival time
• Bias correction (Eaves Kingsman, 2004
  Syntetos Boylan, 2001, 2005)
• Mean demand x (1 a / 2)

37
Forecasting for inventory control Intermittent
demand (continued)

• There is no stochastic model for Crostons method
  (Shenstone Hyndman, 2005)
• Many questionable variance expressions in the
  literature
• The state-space model for intermittent series
  requires a constant mean inter-arrival time
  (Snyder, 2002)
• Why not aggregate the data to eliminate zeroes?

38
Progress in the state of the art, 1985-2005

• Analytical variances are available for most
  methods through SSOE models.
• Robust methods are available for multiplicative
  trends and adaptive simple smoothing.
• Crostons method has been corrected for bias.
• Confusion about renormalization of seasonals has
  finally been resolved.
• There has been little progress in method
  selection.
• Much empirical work remains to be done.

39
Suggestions for research

• Refine the state-space framework
• Add the damped multiplicative trend
• Damp all trends immediately
• Test alternative method selection procedures
• Validate and compare method selection procedures
• Information criteria Benchmark the EIC
• Discriminant analysis
• Regression-based performance index

40
Suggestions for research (continued)

• Develop guidelines for the following choices
• Damped additive vs. damped multiplicative trend
• Fixed vs. adaptive parameters in simple smoothing
• Fixed vs. smoothed trend in additive trend model
• Standard vs. state-space seasonal components
• Additive vs. multiplicative errors
• Analytical vs. empirical prediction intervals

41
Conclusion

• The challenge for future research is to
  establish some basis for choosing among these and
  other approaches to time series forecasting.
  (Gardner,1985)