Network Weather Forecasting MAGGIE (NWF)

1
Network Weather ForecastingMAGGIE(NWF)
Advisor Dr Arshad Ali Co-Advisor Umar
kalim Committee Members Aatif Kamal Kamran
hussain

• Fareena Saqib
• BIT-4 A
• (195)
• 37fareena_at_niit.edu.pk
• fareenas_at_gmail.com

2
Contents

• Problem statement
• Motivation
• Project Aim
• Introduction
• Scope
• Literature Review
• Proposed Solution
• Methodology
• Project Modules
• Comparative Analysis
• Time Line
• Conclusion
• Research Accomplishments
• Future Recommendations

3
Problem Statement
Forecasting the performance of network
using technique that better conserves the varying
patterns in the data using historical data
collected by different active monitoring tools
Content
4
Motivation

• GRID Management System
• Allocation of task
• Parallel processing
• Storage

Content
5
Project Aim
The aim of project is to develop a module that
forecasts the performance of different networks
based on historical data. So that efficiency
of the system can be increased.
Content
6
Introduction

• Forecasting techniques
• Holt Winters
• ARMA/ARIMA
• EWMA
• Regression Analysis
• Why ARMA/ARIMA?
• Varying trends in network data

Content
7
ARMA/ARIMA

• Auto Regressive Integrated Moving Average
• Box and Jenkins approach
• Merger of techniques
• Auto Regression (AR)
• Moving Average (MA)

Benefits of AR
Benefits of MA
ARIMA Approach
Better Results
Content
8
ARMA/ARIMA

• Approach followed
• Box and Jenkins approach is followed

1. Identification of the model (Choosing
tentative p,d,q)
2. Parameter estimation of the chosen model
3. Forecasting
4. Diagnostic checking (are the estimated
residuals white noise?)
No (Return to step 1)
Content
9
ARMA/ARIMA

• Identification
• Through Correlogram
• Autocorrelation Function (ACF)
• Partial Auto Correlation Function (PACF)

Content
10
ARMA/ARIMA

• Estimation
• Estimation of order
• Estimation of equation
• Estimation of coefficients
• Forecasting of data
• Diagnostic Checking
• To check that model is fit to the data.
• Obtain residual
• Obtain ACF and PACF of residual

Content
11
Use of ARIMA Approach
Content
12
Use of ARMA/ARIMA

• Sales of dates contains seasonal effect.
• Month of Ramadan
• Sales of products
• Summer
• Winter
• Spring
• USA economic forecasts
• Weather forecasts

Content
13
Network Weather Forecasting

• Use of ARIMA in network forecast

Network Weather Forecasting
Computer Science Field
Statistics
Network
Econometrics Field
GRID System
Economics Field
Content
14
Scope

• Study of different forecasting techniques
• Pros and cons
• Selection of Technique
• Development of methodology
• Verification of the algorithm
• Modules
• Data Processing module
• Forecasting module
• Visualization module
• Testing Module
• Comparative module
• Development of user Interface

Content
15
Research Issues

• Research Issues
• Development of algorithm using ARIMA approach
• Estimation of the coefficients.
• Diagnostic Checking tests.

Content
16
Literature Review

• Development of algorithm using ARIMA approach
• Basic Econometrics by Damodar N.Gujarati
• Basic concepts
• Time Series Analysis
• ARMA and ARIMA approach introduction.
• Time Series Analysis Forecasting and Control by
  George E.P Box, Gwilym M.Jenkins,Gregory
  C.Reinsel
• Study of ARMA/ARIMA in detail.
• Box and Jenkins Approach
• Basics of statistics
• To understand and revise basic concepts of
  statistics involved in the project.

• Research Issues

17
Literature Review

• Estimation of the coefficients.
• Estimation of coefficient
• http//www.qmw.ac.uk/ugte133/courses/tseries/8idn
  tify.pdf
• Non-linear approaches
• http//www.ece.cmu.edu/moura/papers/icassp88-ribe
  iro-ieeexplore.pdf
• Other approaches
• http//www.cs.cmu.edu/afs/cs/project/cmcl/archive/
  Remulac-papers/tech-report.pdf

• Research Issues

18
Literature Review

• Diagnostic Checking tests.
• Basic Econometrics by Damodar N.Gujarati
• Basic concepts
• Time Series Analysis
• ARMA and ARIMA approach introduction.
• Basics of statistics
• To understand and revise basic concepts of
  statistics involved in the project.

• Research Issues

19
Literature Review

• Algorithms Involved

• Data Processing
• Selection
• of Parameter
• Trim Operation
• Regularization
• Algorithm
• Moving Average for
• Interpolation

• Forecasting
• Stationarity
• Order Estimation
• Coefficient
• Estimation
• Formulation of
• equation

• Verification
• Calculation of
• Residuals
• Trend Analysis
• Portmanteau tests

Content
20
Proposed Solution
Content
21
ARIMA Modeling
Postulate General Class of Models
Identify Model to be Tentatively Entertained
Estimate Parameters in Tentatively Entertained
Model
Diagnostic Checking
yes
No
Use Model for Forecasting
Content
22
Methodology
Content
23
Methodology
Content
24
Methodology
Decision making based on results
Content
25
Network Weather Forecasting

Interpolation
Data Trim
User Interface
Regularization
Access Data
Identification
Covariance
correlogram
Estimation
Integration
Estimation Of order
Estimation of Coefficient
Forecasting
Diagnostic Checking
Content
26
Architecture Diagram
Data files

Data Cleaning
Docs
Docs
Docs
Estimation
GUI
User
Forecasting
Visualization through graphs
Content
27
Use Case Diagram
Content
28
Project Module
Content
29
Data Processing Module
Content
30
Data Cleaning Module

• Data files
• Define format of the data
• Trim operation
• Regularization operation
• Interpolation operation

Content
31
Data Processing
Content
32
Identification Module
Content
33
Identification Module

• Calculated autocorrelations.
• Number of lags
• Trend analysis of autocorrelation coefficients
• Correlogram

Content
34
Identification Module
Content
35
Estimation Module
Content
36
Estimation Module

• Estimation Issues
• Random variable generation with normal
  distribution.
• Estimation of the Model
• Estimation of the order
• Estimation of coefficient
• Estimation of the equation
• Testing the t-test of the coefficient

Content
37
Order Estimation
Content
38
Forecasting Module
Content
39
Forecasting Module

• Processed data with equal time intervals.
• Estimation of order.
• Formulate Equation.
• Estimation of coefficients.
• Forecast parameter values.
• Plot Graph of forecasted values.

Content
40
Forecasting
Content
41
Residual test Module
Content
42
Residual Module

• Graph of the residuals to check white noise
• Check if the forecasting is valid or not.
• Correlogram of residuals.
• Portmanteau tests
• To test the Q-values

Content
43
Residual test
Content
44
Visualization Module
Content
45
Visualization Module

• Visualization of steps carried by algorithm.
• Visualization of the tool developed.
• Plots of data at different processing stages.

Content
46
Demo Snapshot of tool
Content
47
Results
Content
48
Content
49
Correlogram
Content
50
Correlation between yt and yt-1
Content
51
Order Estimation

• Using t-TEST

Content
52
AR Coefficients
Content
53
ARMA/ARIMA results
Content
54
Comparison Module
Content
55
Data plot of xtr
Content
56
Comparison of Results
Content
57
Time Line
Content
58
Conclusion

• The basic requirement of Network Weather
  Forecasting has been achieved by resorting to two
  prong efforts.
• Firstly the technique of ARMA/ARIMA was followed
  and secondly an Algorithm was developed, both of
  which converged in dynamic Network data
  forecasting.
• The methodology adopted was
• Available data on the subject was gathered and
  processed to be used as an input to the
  forecasting module.
• After studying ARMA/ARIMA and ascertaining its
  suitability, algorithm was developed.
• Based on the adopted approach and developed
  algorithm, experiments on forecasting were
  conducted employing the duly processed data.
• The results obtained through different
  experiments were computed, compared and
  characteristics were plotted to come out with a
  fair idea of the final accomplishment.
• Analysis of the results, their comparisons and
  other details were carried out before preparation
  of the report.
• Documentation was undertaken to compile the
  project report.

Content
59
Research Accomplishments

• Developed an algorithm for network weather
  forecasting.
• A research paper on using ARIMA approach in
  network weather forecasting.
• A Journal on results of different forecasting
  techniques on network data and their comparative
  analysis.

Content
60
Future Recommendations

• The forecasting carried out is based on a single
  approach which could be explored for new
  dimensions.
• The forecasting was carried out by employing
  three different tools of data collection which
  could be expanded to more numbers of tools in the
  future.
• Efforts maybe initiated to apply new techniques
  like neural networks and others which are bound
  to come up in the fast developing field of
  information technology.
• Forecasting of data should be made universal and
  the present form of retaining it on a single
  machine could be transformed as a web based tool
  serving all surfers of the web.

Content
61
Demo
Content
62
Thank You!
63
Appendix
64
Trim
65
Regularization
66
Interpolation
67
Parameter estimation
68
Identification Module
69
Correlogram
70
Order Estimation
71
Forecast
72
Residual tests
73
Trim
74
Regularization
75
Interpolate
76
Data Processing
77
Parameter selection
78
Order estimation
79
Forecasting
80
Residual test
81
Thank you!!!