A1256673179zrJlD

1
(No Transcript)
2
MIMO Models and Combinationsof Neural
Networksin Prediction of Gas Consumption
Institute of Chemical Technology PragueFaculty
of Chemical EngineeringDepartment Computing and
Control Engineering
Process Control 2004
Aleš Pavelka
9th June 2004
3
Introduction
3/21
MIMO Models and Combinations of Neural
Networksin Prediction of Gas consumption
Aleš Pavelka, Aleš Procházka DCCE ICT
Introduction

• Architectures Algorithms

• Source Data

• Results

• Conclusion What To Do More?

4
4/21
MIMO Models and Combinations of Neural
Networksin Prediction of Gas consumption
Aleš Pavelka, Aleš Procházka DCCE ICT
Architectures Algorithms
Parametric Model
5
5/21
MIMO Models and Combinations of Neural
Networksin Prediction of Gas consumption
Aleš Pavelka, Aleš Procházka DCCE ICT
Architectures Algorithms
Parametric Model
6
Linear Neural Network
6/21
MIMO Models and Combinations of Neural
Networksin Prediction of Gas consumption
Aleš Pavelka, Aleš Procházka DCCE ICT
Architectures Algorithms
Processing layerof output neurons
Weights
Input layer
7
Feed-Forward Neural Network
7/21
MIMO Models and Combinations of Neural
Networksin Prediction of Gas consumption
Aleš Pavelka, Aleš Procházka DCCE ICT
Architectures Algorithms
newff Create a feed-forward backpropagation
network trainlm Levenberg-Marquardt
backpropagation
Processing layer ofoutput neurons
Processing layerof hidden neurons
Input layer
8
Elman Recurrent Neural Network
8/21
MIMO Models and Combinations of Neural
Networksin Prediction of Gas consumption
Aleš Pavelka, Aleš Procházka DCCE ICT
Architectures Algorithms
External Input
9
Recurrent Neural Network
9/21
MIMO Models and Combinations of Neural
Networksin Prediction of Gas consumption
Aleš Pavelka, Aleš Procházka DCCE ICT
Architectures Algorithms
Processing layer of hidden and output neurons
Forward and feedback connections
Concatenated input - output layer
Source 1, S. Haykin (1994).
10
Recurrent Network - Architecture and Dynamics
10/21
MIMO Models and Combinations of Neural
Networksin Prediction of Gas consumption
Aleš Pavelka, Aleš Procházka DCCE ICT
Training algorithms
Source 2, Williams and Peng (1990).
11
Recurrent Network - BPTT
11/21
MIMO Models and Combinations of Neural
Networksin Prediction of Gas consumption
Aleš Pavelka, Aleš Procházka DCCE ICT
Training algorithms
BackPropagation Trought Time
Source 2, Williams and Peng (1990).
12
Linear Neural Network
12/21
MIMO Models and Combinations of Neural
Networksin Prediction of Gas consumption
Aleš Pavelka, Aleš Procházka DCCE ICT
Architectures Algorithms
13
Source Data
13/21
MIMO Models and Combinations of Neural
Networksin Prediction of Gas consumption
Aleš Pavelka, Aleš Procházka DCCE ICT
Data
Gas consumption and daily temperature in Czech
Republic
14
Source Data - Properties
14/21
MIMO Models and Combinations of Neural
Networksin Prediction of Gas consumption
Aleš Pavelka, Aleš Procházka DCCE ICT
Data
15
Results - parametric models
15/21
MIMO Models and Combinations of Neural
Networksin Prediction of Gas consumption
Aleš Pavelka, Aleš Procházka DCCE ICT
Results
input data - daily temperatureoutput data - gas
consumption
Learning data 14.9.1999 30.4.2000Testing
data 14.9.2000 1.5.2001Calculation of 3 521
826 parametric models (one months of
calculation) Selection condition AIC lt 1.55
FPE lt 4.68 fit gt 71Box - Jenkins models
(na 0 and mostly nk 0 too)!
AIC FPE fit na nb nc nd nf nk
1.4770 4.4109 72.2824 0 5 3 8 13 0
1.4671 4.3559 71.4417 0 9 6 0 11 0
1.4858 4.4626 71.4334 0 10 3 0 20 0
1.4722 4.4025 71.9078 0 10 4 6 15 0
16
Parametric Model
16/21
MIMO Models and Combinations of Neural
Networksin Prediction of Gas consumption
Aleš Pavelka, Aleš Procházka DCCE ICT
Graphs
BJ Model 0 5 3 8 13 0 - Testing
BJ Model 0 5 3 8 13 0 - Learning
17
Results neural networks
17/21
MIMO Models and Combinations of Neural
Networksin Prediction of Gas consumption
Aleš Pavelka, Aleš Procházka DCCE ICT
Results
Learning data 1.10.1998 1.5.1999L. and T.
data 1.10.1999 30.4.2000Testing data
1.10.2000 1.5.2001
gas temperature day info
Model Base Models Base Models Base Models Base Models Final
Architecture 27170-2-1 27170-2-1 27170-2-1 27170-1 4-1
Network Elman Feed Forward BPTT Linear Linear
ME 0.81956 -0.18288 -0.90519 0.41221 -0.60557
STD 3.0707 3.4996 11.228 3.3003 2.9924
MIN -7.9024 -11.22 -20.6589 -10.3363 -7.8115
MAX 7.7218 7.4217 27.3787 7.5286 9.6296
SSE 1869.326 2271.975 23474.99 2046.598 1724.774
MSE 10.0501 12.2149 126.2096 11.0032 9.273
in5 40.3226 48.3871 12.3656 40.3226 47.8495
18
Neural Networks
18/21
MIMO Models and Combinations of Neural
Networksin Prediction of Gas consumption
Aleš Pavelka, Aleš Procházka DCCE ICT
Graphs
Base Models - Testing No.1Final Network - Input
Data Learning 1.10.1999 30.4.2000
Base Models - Learning 1.10.1998 1.5.1999
Final Model - Learning 1.10.1999 30.4.2000
19
Neural Networks
19/21
MIMO Models and Combinations of Neural
Networksin Prediction of Gas consumption
Aleš Pavelka, Aleš Procházka DCCE ICT
Graphs
Base Models - Testing No.2Final Network - Input
Data Testing 1.10.2000 1.5.2001
Final Network - Testing 1.10.2000 1.5.2001
20
Conclusion
20/21
MIMO Models and Combinations of Neural
Networksin Prediction of Gas consumption
Aleš Pavelka, Aleš Procházka DCCE ICT
Final

• found suitable parametric model Box -
  Jenkins model
• has been created and practically tested a neural
  network combining prediction results
• How to decide the best algorithm?

21
What To Do For Better Results ?
21/20
MIMO Models and Combinations of Neural
Networksin Prediction of Gas consumption
Aleš Pavelka, Aleš Procházka DCCE ICT
Final

• use of different types of data preprocessing
• prediction using further preprocessing tools
• selection of different data sets
• multi-step prediction by modified one-step
  prediction models