Traffic Matrix Estimation for Traffic Engineering

1
Traffic Matrix Estimation for Traffic Engineering

• Mehmet Umut Demircin

2
Traffic Engineering (TE)

• Tasks
• Load balancing
• Routing protocols configuration
• Dimensioning
• Provisioning
• Failover strategies

3
Particular TE Problem

• Optimizing routes in a backbone network in order
  to avoid congestions and failures.
• Minimize the max-utilization.
• MPLS (Multi-Protocol Label Switching)
• Linear programming solution to a multi-commodity
  flow problem.
• Traditional shortest path routing (OSPF, IS-IS)
• Compute set of link weights that minimize
  congestion.

4
Traffic Matrix (TM)

• A traffic matrix provides, for every ingress
  point i into the network and every egress point j
  out of the network, the volume of traffic Ti,j
  from i to j over a given time interval.
• TE utilizes traffic matrices in diagnosis and
  management of network congestion.
• Traffic matrices are critical inputs to network
  design, capacity planning and business planning.

5
Traffic Matrix (contd)

• Ingress and egress points can be routers or PoPs.

6
Determining the Traffic Matrix

• Direct Measurement
• TM is computed directly by collecting flow-level
  measurements at ingress points.
• Additional infrastructure needed at routers.
  (Expensive!)
• May reduce forwarding performance at routers.
• Terabytes of data per day.
• Solution Estimation

7
TM Estimation

• Available information
• Link counts from SNMP data.
• Routing information. (Weights of links)
• Additional topological information. ( Peerings,
  access links)
• Assumption on the distribution of demands.

8
Traffic Matrix EstimationExisting Techniques
and New DirectionsA. Madina, N. Taft, K.
Salamatian, S. Bhattacharyya, C. DiotSigcomm
2003
9
Three Existing Techniques

• Linear Programming (LP) approach.
• O. Goldschmidt - ISMA Workshop 2000
• Bayesian estimation.
• C. Tebaldi, M. West - J. of American Statistical
  Association, June 1998.
• Expectation Maximization (EM) approach.
• J. Cao, D. Davis, S. Vander Weil, B. Yu - J. of
  American Statistical Association, 2000.

10
Terminology

• cn(n-1) origin-destination (OD) pairs.
• X Traffic matrix. (Xj data transmitted by OD
  pair j)
• Y(y1,y2,,yr ) vector of link counts.
• A r-by-c routing matrix (aij1, if link i
  belongs to the path associated to OD pair j)
• YAX
• rltltc gt Infinitely many solutions!

11
Linear Programming

• Objective
• Constraints

12
Statistical Approaches
13
Bayesian Approach

• Assumes P(Xj) follows a Poisson distribution with
  mean ?j. (independently dist.)
• needs to be
  estimated. (a prior is needed)
• Conditioning on link counts P(X,?Y)
• Uses Markov Chain Monte Carlo (MCMC) simulation
  method to get posterior distributions.
• Ultimate goal compute P(XY)

14
Expectation Maximization (EM)

• Assumes Xj are ind. dist. Gaussian.
• YAX implies
• Requires a prior for initialization.
• Incorporates multiple sets of link measurements.
• Uses EM algorithm to compute MLE.

15
Comparison of Methodologies

• Considers PoP-PoP traffic demands.
• Two different topologies (4-node, 14-node).
• Synthetic TMs. (constant, Poisson, Gaussian,
  Uniform, Bimodal)
• Comparison criteria
• Estimation errors yielded.
• Sensitivity to prior.
• Sensitivity to distribution assumptions.

16
4-node topology
17
4-node topology results
18
14-node topology
19
14-node topology results
20
Marginal Gains of Known Rows
21
New Directions

• Lessons learned
• Model assumptions do not reflect the true nature
  of traffic. (multimodal behavior)
• Dependence on priors
• Link count is not sufficient (Generally more data
  is available to network operators.)
• Proposed Solutions
• Use choice models to incorporate additional
  information.
• Generate a good prior solution.

22
New statement of the problem

• Xij Oi.aij
• Oi outflow from node (PoP) i.
• aij fraction Oi going to PoP j.
• Equivalent problem estimating aij .
• Solution via Discrete Choice Models (DCM).
• User choices.
• ISP choices.

23
Choice Models

• Decision makers PoPs
• Set of alternatives egress PoPs.
• Attributes of decision makers and alternatives
  attractiveness (capacity, number of attached
  customers, peering links).
• Utility maximization with random utility models.

24
Random Utility Model

• Uij Vij eij Utility of PoP i choosing to
  send packet to PoP j.
• Choice problem
• Deterministic component
• Random component mlogit model used.
  

25
Results

• Two different models (Model 1attractiveness,
• Model 2 attractiveness repulsion )

26
Fast Accurate Computation of Large-Scale IP
Traffic Matrices from Link LoadsY. Zhang, M.
Roughan, N. Duffield, A. GreenbergSigmetrics
2003
27
Highlights

• Router to router traffic matrix is computed
  instead of PoP to PoP.
• Performance evaluation with real traffic
  matrices.
• Tomogravity method (Gravity Tomography)

28
Tomogravity

• Two step modeling.
• Gravity Model Initial solution obtained using
  edge link load data and ISP routing policy.
• Tomographic Estimation Initial solution is
  refined by applying quadratic programming to
  minimize distance to initial solution subject to
  tomographic constraints (link counts).

29
Gravity Modeling

• General formula
• Simple gravity model Try to estimate the amount
  of traffic between edge links.

30
Generalized Gravity Model

• Four traffic categories
• Transit
• Outbound
• Inbound
• Internal
• Peers P1, P2,
• Access links a1, a2, ...
• Peering links p1,p2,

31
Generalized Gravity Model
32
Generalized Gravity Model
33
Tomography

• Solution should be consistent with the link
  counts.

34
Reducing the computational complexity

• Hundreds of backbone routers, ten thousands of
  unknowns.
• Observations
• Some elements of the BR to BR matrix are empty.
  (Multiple BRs in each PoP, shortest paths)
• Topological equivalence. (Reduce the number of
  IGP simulations)

35
Quadratic Programming

• Problem Definition
• Use SVD to solve the inverse problem.
• Use Iterative Proportional Fitting (IPF) to
  ensure non-negativity.

36
Evaluation of Gravity Models
37
Performance of proposed algorithm
38
Comparison
39
Robustness

• Measurement errors
• xAte
• exN(0,s)

40
Questions?