Josefina L

1

Time Series Prediction Using Inductive Reasoning
Techniques

• Josefina López Herrera
• Advisors
• François Cellier
• Gabriela Cembrano
• IOC - UA - IRI

2
Table of Contents

• Contributions principales.
• Antecedents.
• Time Series Analysis Techniques.
• Fuzzy Inductive Reasoning (FIR) for Time Series
  Analysis.
• Time Series Characteristics.
• Conclusions and Future Research.

3
Contributions

• Evaluation of Prediction Error.
• Confidence Measures for Prediction in FIR.
• Dynamic Mask Allocation.
• Estimation of Horizon of Predictability.
• Applications
• Early Warning Using Smart Sensors.
• Signal Predictive Control Using FIR

4
Antecedents

• George Klir at the State University of New York
  Uyttenhove 1978, Klir 1985

• François Cellier at the University of Arizona
  Cellier and Yandell 1987, D. Li and Cellier
  1990,Cellier 1991,Cellier et al. 1996, Cellier et
  al. 1998

• Rafael Huber and Gabriela Cembrano at the IRI
  Institute (UPC-CSIC)

5
PhD. Dissertations UPC-UA

• Angela Nebot Castells (1994)
• Qualitative Modeling and Simulation of
  Biomedical Systems using FIR

• Francisco Múgica (1995)
• Diseño Sistemático de Controladores Difusos
  Usando Razonamiento Inductivo

• Alvaro de Albornoz Bueno (1996)
• Inductive Reasoning and Reconstruction
  Analysis Two Complementary Tools for Qualitative
  Fault Monitoring of Large-Scale Systems

6
Time Series Analysis Techniques
Pattern-Based Approaches
Linear Models
Fuzzy Logic
Non-Linear Models
FIR
7
Linear Models

• Stationarity will be assumed.
• Prefiltering of data may be necessary.
• Probabilistic Reasoning.
• Ljung 1999, Brockwell and David 1991, 1996 ,Box
  Jenkins 1994.
• Stochastic Time Series.

8
Non-Linear Models

• Parametric Models, Learning Techniques
• At least Quasi-stationary
• Deterministic Elements
• State Space Models (Casdagli and Eubank 1992)
• Neural Networks (Weigend and Gershenfeld 1994)
• Hybrid Models (Delgado 1998, Telecom 1994)

9

Fuzzy Logic

• Non-parametric Models, Synthesized Techniques
• At least Quasi-stationary, Deterministic Elements
• Fuzzy Neural Networks (Jang 1997)
• FIR (López et al. 1996)
• Mixed Models Burr 1998, Takagi and Sugeno 1991

10
FIR

• Fuzzification Conversion to qualitative
  variables (Fuzzy Recoding)
• Qualitative ModelingFind the best qualitative
  relationship between inputs and outputs (Fuzzy
  Modeling)
• Qualitative Simulation Forecasting of future
  qualitative outputs (Fuzzy Simulation)
• Defuzzification Conversion to quantitative
  variables (Regeneration)

11
(No Transcript)
12
Qualitative Modeling
13
Qualitative Simulation
Behavior Matrix
Raw Data Matrix
Input Pattern
Optimal Matrix
3
2
Matched Input Pattern
1
1
?
Distance Computation Euclidean dj
5-Nearest Neighbors
Output Forecast Computation fiF(W5-NN-out)
Forecast Value
Class
Member
Side
14
Time Series Forecasting

• In univariate time series, only a single variable
  has been observed, the future values of which are
  to be predicted on the basis of their own past.

• In this case, the mask candidate matrix has
  n-rows and one column. In order to decide the
  depth of the mask, the autocorrelation function
  is used.

15
Characteristics of Time Series
B-Barcelona water demand time series
L- chaotic intensity pulsation of a single-mode
far infrared NH3 laser beam
V-Van-der-Pol oscillator time series
Weigend and Gershenfeld 1994
16
Water Demand Prediction

• Data Daily Demand in Barcelona. Jan 1985 - Nov
  1986.

• The process is quasi-stationary, and its variance
  is roughly constant.

17
Water Demand Prediction

• The water demand on any given day is strongly
  correlated with the demand seven days earlier.

• Autocorrelation function of daily demand series.

18
Water Demand Prediction

• The result of prediction was

19
(No Transcript)
20
Prediction Error
21
Prediction Error
22
Qualitative Simulation with FIR
real data
predicted data
using k steps
prediction for time
23
Comparison of FIR with other Methodologies for
the Barcelona Water Demand Time Series
) with intervention analysis Related
Investigation
without intervention analysis
24
Comparison of FIR with other Methodologies
25
Confidence Measures
Crisp
Fuzzy Logic
Proximity
Similarity
26
Sources of Uncertainty in Predictions

• Dispersion among neighbors in input space.
• Uncertainty related to quantity of measurements.

• Dispersion among neighbors in output space.
• Uncertainty related to quality of measurements.

27
Proximity Measure

• This measure is related to establishing the
  distance between the testing input state and the
  training input states of its five nearest
  neighbors in the experience data base and to
  establishing distance measures between the output
  states of the five nearest neighbors among
  themselves.

28
Similarity Measure

• This measure is defined without the explicit use
  of a distance function, the similarity measure
  presented is based on intersection, union and
  cardinality.

AB then S1(A,B) 1.0
A disjoint B then S1(A,B) 0.0

• (Dubois and Pradé 1980).

29
Similarity Measure

• The similarity of the ith m-input of the jth
  nearest neighbor to the testing m-input based on
  intersection can be defined as follows

where qi are normalized values in the range from
0 to 1.

• The overall similarity of the jth neighbor is
  defined as the average similarity of all its
  m-inputs in the input space

30
Similarity Measure

• The similarity of the jth neighbor to the
  estimated testing m-output based on intersection
  can be defined as follows

• A confidence value based on similarity measures
  can thus be defined

31
FIR Confidence Measures for NH3 Time Series

• Deterministic process

• Similarity and Proximity

32
FIR Confidence Measures for Barcelona Time Series

• Stochastic Process with deterministic elements.

• The relationship between the prediction error and
  the confidence measures is less evident.

• The two are positively correlated.

33
Evaluation of Confidence Measures

• The similarity measure is more sensitive to the
  prediction error because the similarity measure
  preserves the qualitative difference between a
  new input state and its neighbors in the
  experience data base.

• The confidence measures are indicators of how
  well the series may be fitted by an
  autoregressive or deterministic model.

34
Dynamic Mask Allocation in Fuzzy Inductive
Reasoning (DMAFIR)
c1
FIR Mask 1
y1
Mask Selector
c2
y2
FIR Mask 2
Best mask
Ts
Switch Selector
y
cn
yi predicted output using mask mi ci estimated
confidence
FIR Mask n
yn
35
Optimal and Suboptimal Mask for Barcelona Time
Series
36
Dynamic Mask Allocation Applied to Barcelona Time
Series
37
Prediction and Simulation

• FIR Predictions use different masks to predict
  future values n-steps into the future, avoiding
  the use of already predicted (contamined) data in
  the predictions.
• FIR Simulations use the optimal mask of the
  single step prediction recursively, minimizing
  the distance of extrapolation at the expense of
  recursively using already contamined data.

38
Qualitative Prediction
Optimal Mask
Mask candidate matrix
1-step prediction
2-step prediction
3-step prediction
39
Simulation and Prediction

• Without dynamic mask allocation for Barcelona
  time series.

• Comparison of FIR qualitative simulation and
  prediction with dynamic mask allocation for
  Barcelona time series.

40
DMAFIR Algorithm to Predict Time Series with
Multiple Regimes

• The behavioral patterns change between segments.
• Van-der-Pol oscillator series is introduced. This
  oscillator is described by the following
  second-order differential equation

• By choosing the outputs of the two integrators as
  two state variables

• The following state-space model is obtained

Output Time Series
41
DMAFIR Algorithm to Predict Time Series with
Multiple Regimes
the input/output behaviors will be different
because of the different training data used by
the two models
42
Prediction Errors for Van-der-Pol Series

• The values along the diagonal are smallest and
  the values in the two remaining corners are
  largest.

• FIR during the prediction looks for five good
  neighbors, it only encounters four that are truly
  pertinent.

43
One-day Predictions of the Van-der-Pol Multiple
Regimes Series.

• A time series was constructed in which the
  variable ?

assumes a value of 1.5 during one segment,
followed by a value of 2.5 during the second time
segment, followed by 3.5 .
The multiple regimes series consists of 553
samples.
44
Prediction Errors for Multiple Regimes
Van-der-Pol Series

• The model obtained for ?

1.5 cannot predict the higher peaks of the
second and third time segment very well.

• The DMAFIR error demostrates that this new
  technique can indeed be successfully applied to
  the problem of predicting time series that
  operate in multiple regimes.

45
Variable Structure System Prediction with DMAFIR

• A time-varying system exhibits an entire
  spectrum of different behavioral patterns. To
  demonstrate DMAFIRs ability of dealing with
  time-varying systems, the Van-der-Pol oscillator
  is used. A series was generated, in which

changes its value continuously in the range from
1.0 to 3.5. The time series contains 953 records
sampled using a sampling interval of 0.05.

46
One-day Predictions of the Van-der-Pol
Time-varying Series Using DMAFIR with the
Similarity Confidence Measure

• Predictions Errors for Time-varying Van-der-Pol
  Series.

47
Predicting the Predictability Horizon

• The errors are likely to accumulate during
  iterative predictions of future values of a time
  series.
• It is thus of much interest to the user of such a
  tool to be able to assess the quality of
  predictions made not only locally, but as a
  function of time.
• When the predictions depend on previously
  predicted data points these are by themselves
  associated with a degree of uncertainty already.
• In the first step of a multiple-step prediction,
  the predicted value depends entirely on
  measurement data.
• The local error can be indirectly estimated using
  the proximity or similarity measure.
• Either measure can easily be extended to become
  an estimator of accumulated confidence

48
Water Demand of the City of Barcelona Multiple
Step simulation using FIR
49
Conclusions

• The prediction made by CIR (Causal Inductive
  Reasoning) were not significantly better.
• The confidence measure of FIR are an indirect
  prediction error estimate.
• A new formula to assess the error of predictions
  of a univariate time series, the FIR filters out
  what it considers to be a noise.
• FIR provides the model automatically, not
  requires a significant development effort as well
  as knowledge about the nature of the process form
  wich the series was derived.
• The confidence measures provide at least a
  statistical estimate for the quality of the
  prediction.
• Several suboptimal mask are used to make, in
  parallel forecast of the same time series. Each
  of the forecast is accompanied by an estimate of
  its quality. In each step, the one forecast is
  kept as the true forecast to be reported back to
  the user that shows the highest confidence value.
• A set of formulae has been devised to estimate
  the effects of data contamination on the
  accunulated confidence over multiple prediction
  steps.
• The FIR is a robust methodology, after López et
  al. 96 some UPC groups use FIR like Prediction
  Module in an Optimation Tool for Water
  Distribution Networks, Quevedo et al. 1999.

50
Publications

• Cellier, F. And J. López (1995). Causal Inductive
  Reasoning. A new paradigm for data-driven
  qualitative simulation of continuous-time
  dynamical systems. Systems Analysis Modelling
  Simulation 18(1), pp.26-43.
• Cellier F., J. López, A. Nebot, G. Cembrano
  (1996), Means for estimating the forecasting
  error in Fuzzy Inductive Reasoning,
  ESM96European Simulation Multiconference,
  Budapest, Hungary, June 2-6, pp.654-660.
• López J., G. Cembrano, F, Cellier (1996), Time
  series prediction using Fuzzy Inductive
  Reasoning, ESM96European Simulation
  Multiconference, Budapest, Hungary, June 2-6,
  pp.765-770.
• Cellier F., J. López, A. Nebot, G. Cembrano
  (1998), Confidence measures in Fuzzy Inductive
  Reasoning, International Journal of General
  Systems, in print.
• López J., F. Cellier (1999), Improving the
  Forecasting Capability of Fuzzy Inductive
  Reasoning by Means of Dynamic Mask Allocation,
  ESM99European Simulation Multiconference, in
  print.
• López J., F. Cellier, G. Cembrano, L. Ljung,
  (1999), Estimating the horizon of predictability
  in time series predictions using inductive
  modeling tools, International Journal of General
  Systems, submitted for publication.

51
Future Research

• Use of time-series predictors in the design of
  smart sensors with look-ahead capabilities. If a
  sensor with look-ahead capability can anticipate
  the crossing of a critical threshold, it may
  issue an early warning that might enable the
  plant operator to do something about the problem
  before it ever occurs. (Appendix A)
• The design of signal predictive controllers that
  make use of smart sensors of the class introduced
  in Appendix A, to improve the control performance
  of feedback control systems.