DiffServ/MPLS Network Design and Management

1
DiffServ/MPLS Network Design and Management

• Doctoral Dissertation
• Tricha Anjali
• Broadband and Wireless Networking Laboratory
• Advisor Dr. Ian F. Akyildiz

2
Contents

• Introduction
• Network Management
• TEAM Structure
• LSP/lSP Setup
• Traffic Routing
• Available Bandwidth Estimation
• End-to-end Available Bandwidth Measurement
• Inter-domain Management
• TEAM Implementation
• Conclusions
• Future Work

3
Goals

• Two-fold which are complementary
• Guarantee Quality of Service for the required
  applications.
• Use the network resources efficiently.

4
MultiProtocol Label Switching

• Explicitly routed point-to-point paths called
  Label Switched Paths (LSPs)
• Support for traffic engineering and fast reroute
• Simpler switching operations

5
Generalized MPLS

• GMPLS is a set of protocols for a common control
  of packet and wavelength domains
• Reserve a wavelength on a path (Lambda Switched
  Path or lSP) for an aggregation of flows

6
DiffServ GMPLS

• DiffServ
• Scalable service differentiation
• DiffServ GMPLS
• Class differentiation for QoS provisioning
• Traffic Engineering for DiffServ classes for
  efficient use of resources

7
Network Model
MPLS Networks Link Label Switched Path (LSP)
Class Type 0 (BE)
Class Type 1 (AF)
Class Type 2 (EF)
Wavelength Network Link lambda Switched Path
(lSP)
Optical Network Link fiber
8
MPLS Network Management

• Existing MPLS network management tools
• RATES (Bell Labs, 2000)
• Sets up bandwidth guaranteed LSPs
• Does not support DiffServ
• No performance measurement and analysis
• DISCMAN (EURESCOM, 2000)
• Provides test and analysis results of DiffServ
  and MPLS-based DiffServ
• Does not provide its own management system
  functionality

9
MPLS Network Management

• Other existing MPLS network management tools
• MATE (Bell Labs, Univ. Michigan, Caltech,
  Fujitsu, 2001)
• The goal is to distribute the traffic across
  several LSPs established between a given ingress
  and egress node pair
• Not for traffic that requires bandwidth
  reservation
• TEQUILA (European Union Project, 2002)
• Global and integrated approach to network design
  and management
• No network management methods developed and
  implemented
• No evaluation of performances

10
A New Network Management Tool

• Traffic Engineering Automated Manager (TEAM)
• Automated
• Monitors the network performance
• Implements various algorithms for handling events
  in MPLS and optical network
• Allows efficient use of resources and prompt
  responses

11
Big Picture of TEAM

• Traffic Engineering Automated Manager

Simulation Tool (ST)
Management Plane
DiffServ/ GMPLS Domain
Traffic Engineering Tool (TET)
LSP/lSP Setup/ Dimensioning
Resource
LSP Preemption
Route
LSP Routing
Measurement/ Performance Evaluation Tool (MPET)
Traffic Routing
Network Dimensioning and Topology Design
TEAM
To neighboring TEAM
12
LSP and lSP Setup Problem

• Optimal Policy for LSP Setup in MPLS Networks,
  Computer Networks Journal, June 2002
• LSP and lSP Setup in GMPLS Networks,
  Proceedings of IEEE INFOCOM, March 2004

• Find an adaptive traffic driven policy for
  dynamic setup and tear-down of LSPs and ?SPs.
• Why not the fully connected topology?
• Too many LSPs for increasing number of routers N
  (N2 problem)
• Why not a fixed topology?
• Because traffic is unpredictable

13
LSP and lSP Setup Problem

• Arrival of bandwidth request
• Decision among
• Option 1 no action
• Option 2 setup a direct LSP
• Option 3 setup a direct lSP and LSP

1
dest
2
src
3
14
LSP and lSP Setup

• Optical network virtual topology design
  algorithms
• Chen 1995, Davis 2001, Krishnaswamy 2001 Design
  the network off-line with a given traffic matrix
• Gençata 2003 On-line virtual topology
  adaptation approach for optical networks
• Does not combine optical and MPLS layers

15
Assumptions

• Routing Assumption
• Default topologies
• Packets are routed either on
• the direct LSP(i,j) or
• the min-hop path P(i,j) over the default MPLS
  network
• LSPs are routed either on
• the direct lSP or
• the min-hop path Plij over the default optical
  network
• a new LSP can not be routed on a previously
  established non-default lSP

16
Model Formulation

• Events and Decision Instants
• MPLS network
• Arrival/Departure of bandwidth requests between
  (i, j)
• Optical network
• Arrival of LSP(i, j) capacity increment/decrement
  requests

17
Model Formulation

• State vector (local)
• MPLS network s (A, Bl, Bp)
• Available capacity (A)
• Bandwidth requests on direct LSP (Bl) or on
  min-hop path (Bp)
• Optical network s (A, Bl, Bp, k)
• Available capacity (A)
• Capacity requests on direct lSP (Bl) or on
  min-hop path (Bp)
• Number of lSPs between the node pair (k)

18
Model Formulation (Contd.)

• Action Variables
• MPLS network
• Optical network

19
Cost Model

• Incremental cost
• W Wb Wsw Wsign
• Wb(s,a) Bandwidth cost
• Wsw(s,a) Switching cost
• Wsign(s,a) Signaling cost if LSP/lSP is set-up
  or re-dimensioned
• Wb and Wsw are linear with respect to the
  bandwidth request and time
• Wsign is incurred only if the decision is a 1

20
Optimal Setup Policy

• Based on Markov Decision Process Theory
• Minimize expected infinite-horizon discounted
  total cost
• Determine transition probabilities and optimality
  equations
• Solve the optimality equations with value
  iteration algorithm
• Optimal policy stationary
  control-limit

21
Optimization (MPLS network)
Optimal policy ? such that
Optimality equations
where
22
Optimal Policy (MPLS Network)
where
23
Optimization (Optical Network)
Optimal policy ? such that
Optimality equations
where
24
Optimal Policy (Optical Network)
where
25
Sub-optimal Policy

• Optimal policy is difficult to pre-calculate
  because of large number of possible system states
• Sub-optimal policy that is fast and easy to
  calculate
• Minimizes the cost incurred between two decision
  instants
• Maintains the threshold structure of the optimal
  policy

26
Sub-optimal Policy (MPLS)
where
where
27
Sub-optimal Policy (Optical)
where
28
Performance Evaluation

• Example network
• Network has 10 nodes and 17 links
• Cph 1000 Mbps
• Diameter length of longest shortest path 3

29
Comparison
Discounted total cost vs. Initial state
Discount factor0.5
Discount factor0.1
30
Experimental Results

• What happens when we homogeneously increase
  traffic on selected node pairs
• LSPs with larger number of default LSPs in
  their path are established first
• lSPs with larger number of default lSPs that
  need re-dimensioning in their path are
  established first

31
Heuristics for Comparison

• Heuristic 1 Fully connected LSP network
• Heuristic 2 LSP re-dimensioned exactly
• Heuristic 3 LSP re-dimensioned with extra
  capacity
• In each heuristic, lSP network is fully
  connected

32
Total Expected Cost
33
Bandwidth Wastage in MPLS Network
34
Big Picture of TEAM

• Traffic Engineering Automated Manager

Simulation Tool (ST)
Management Plane
DiffServ/ GMPLS Domain
Traffic Engineering Tool (TET)
LSP/lSP Setup/ Dimensioning
Resource
LSP Preemption
Route
LSP Routing
Measurement/ Performance Evaluation Tool (MPET)
Traffic Routing
Network Dimensioning and Topology Design
TEAM
To neighboring TEAM
35
QoS Routing

• A New Path Selection Algorithm for MPLS Networks
  Based on Available Bandwidth Estimation,
  Proceedings of QoFIS, October 2002
• Traffic Routing in MPLS Networks Based on QoS
  Estimation and Forecast, submitted

• Find a low cost feasible path for routing traffic
  flows in MPLS networks adaptively.
• Why adaptive?
• Because MPLS network topology is changing
• Existing routing algorithms
• Heuristic solutions of the delay constrained
  least cost problem
• LSP routing algorithms (MIRA, PBR)

36
Routing Algorithm

• Notations
• puv path in the MPLS network
• puv (lux, , lzv)
• Alij/dlij Available capacity/delay on lij
• npuv Number of LSPs in puv

37
Cost Model

• LSP cost
• W Wb Wsw WsignWABWd
• Wb and Wsw linear with respect to the bandwidth
  request and duration of request
• Wsign is instantaneous
• WAB is inversely related to LSP available
  bandwidth
• Wd linear with respect to delay on the LSP
• Path cost
• Wp ? LSP costs (n-1) ( Relay node cost )

38
Routing Problem

• Find the path such that
• subject to feasibility constraints

39
Routing Algorithm

• Heuristic of the exact problem
• Path set size restricted to F
• Set populated by paths with increasing length
• Feasibility check
• Cost comparison

40
Partial Information

• Estimation algorithm for accurate state
  information
• Linear prediction
• Dynamically change the number of past samples
  based on prediction performance

41
Performance Evaluation
Popular ISP topology with link capacity 155 c.u.
42
Rejection Ratio
43
Minimum Available Bandwidth
44
Paths with Relay Nodes
45
Big Picture of TEAM

• Traffic Engineering Automated Manager

Simulation Tool (ST)
Management Plane
DiffServ/ GMPLS Domain
Traffic Engineering Tool (TET)
LSP/lSP Setup/ Dimensioning
Resource
LSP Preemption
Route
LSP Routing
Measurement/ Performance Evaluation Tool (MPET)
Traffic Routing
Network Dimensioning and Topology Design
TEAM
To neighboring TEAM
46
Available Bandwidth Measurement

• ABEst An Available Bandwidth Estimator within
  an Autonomous System, Proceedings of IEEE
  Globecom, November 2002
• MABE A New Method for Available Bandwidth
  Estimation in an MPLS Network, Proceedings of
  IEEE NETWORKS, August 2002

• Measure/estimate the available bandwidth in a
  link/path to analyze the performance of the
  network
• Various existing tools to measure narrow link
  capacity
• Pathchar based (Jacobson 1997) link-by-link
  measurement
• Packet pair based (Keshav 1991) end-to-end
  capacity
• Nettimer (Lai 2001) end-to-end capacity
• AMP (NLANR 2002) active link-by-link
  measurement
• OCXmon (NLANR 2002) passive link-by-link
  measurement
• MRTG (Oetiker 2000) 5 min averages of link
  utilization
• Pathload (Jain 2002) end-to-end available
  bandwidth measurement

47
Available Bandwidth Estimator

• Assumptions
• SNMP is enabled in the domain
• MRTG is used to poll the network devices with
  10 sec granularity
• Notations
• L(t) Traffic load at time t
• ? Length of averaging interval of MRTG
• L?k Average load in (k-1)?, k?
• p Number of past measurements in prediction
• h Number of future samples reliably predicted
• Ahk Available bandwidth estimate for (k1)?,
  (kh)?

48
ABEst (Contd.)

• We use the past p samples to predict the
  utilization for the next h samples
• Utilize the covariance method for prediction
• Values of p and h varied according to the
  estimation error

49
ABEst (Contd.)

1. At time instant k, available bandwidth
   measurement is desired.
2. Find the vectors wa, a?1,h using covariance
   method given p and the previous measurements.
3. Find and
4. Predict Ahk for (k1)?, (kh)t.
5. At time (kh)t, get
6. Find the error vector
7. Set k kh.
8. Obtain new values for p and h.
9. Go to step 1.

50
ABEst (Contd.)

• Covariance estimated as
• Covariance normal equations
• Ahk estimated
• Either C maxpredicted utilization vector
• Or C Effective bandwidth from the utilization
  vector

51
ABEst (Contd.)

• Algorithm for h and p
• If s/m gt Th1, decrease h until hmin and increase
  p till pmax multiplicatively
• If Th1 gt s/m gt Th2, decrease h until hmin and
  increase p till pmax additively
• If s/m lt Th2, then
• If mgt Th3M2E, decrease h until hmin and increase
  p till pmax additively
• If Th3M2E gt m gt Th4M2E, keep h and p constant
• If m lt Th4M2E, increase h and decrease p till
  pmin additively

52
Performance Evaluation

• hmin10

53
Performance Evaluation (Contd.)
hmin20
54
End-to-end AB Measurement

• TEMB Tool for End-to-End Measurement of
  Available Bandwidth, Proceedings of IEEE ELMAR,
  June 2003

• Motivation
• Combine active and passive approaches
• Most tools estimate narrow link capacity
• Accuracy
• Scalability
• Statistical robustness
• Not intrusive

55
Tight Link Identification

• Measurement packets
• 10 measurement packets sent in a second, to make
  the tool non-intrusive

56
Data Record

• Data record
• Inserted/modified by the hops of the path
• Counter information from MIB-II in router

57
Example of Auto-detection
D.1.1.1
B.1.1.1
A.1.1.1
C.1.1.1
S
D
58
Example of Non-min-hop Path
D.1.1.1
B.1.1.1
A.1.1.1
C.1.1.1
S
D
59
Tight Link Identification

• 10 packets in one second
• N packets back at source for analysis
• Utilization of I-th interface at time tk
• Available bandwidth
• At least agreelink of the estimates should concur
  about the tight link identity.

60
Tight Link Identification (Contd.)

• All (N-1) estimates should be within 100,
  agreeavail of the minimum estimate
• Otherwise the next batch of 10 packets is sent.
• Average available bandwidth of interface I is
• where n attempts have been made at measurement

61
MRTG-based Measurement

• More accurate estimation of tight link available
  bandwidth
• MRTG-based passive approach similar to ABEst
• Reliably predicts the utilization of the link for
  a future interval, that varies in size

62
Big Picture of TEAM

• Traffic Engineering Automated Manager

Simulation Tool (ST)
Management Plane
DiffServ/ GMPLS Domain
Traffic Engineering Tool (TET)
LSP/lSP Setup/ Dimensioning
Resource
LSP Preemption
Route
LSP Routing
Measurement/ Performance Evaluation Tool (MPET)
Traffic Routing
Network Dimensioning and Topology Design
TEAM
To neighboring TEAM
63
Inter-domain Resource Management

• A New Scheme for Traffic Estimation and Resource
  Allocation for Bandwidth Brokers, Computer
  Networks Journal, April 2003
• Filtering and Forecasting Problems for Aggregate
  Traffic in Internet Links, Performance
  Evaluation Journal, 2004

• Inter-domain resource reservation agreements
• Estimate the traffic on an inter-domain link and
  forecast its capacity requirement, based on a
  measurement of the current usage
• Efficient resource utilization while keeping the
  number of reservation modifications to low
  values.
• Two approaches for resource allocation
• Off-line simple and predictable but lead to
  resource wastage
• On-line Cushion scheme (Terzis 2001) wherein
  extra bandwidth is reserved over the current
  usage.
• large number of re-negotiations to satisfy the
  QoS.

64
Resource Reservation Problem

• Assumptions
• Estimate traffic for one traffic class
• Number of established sessions is N and stays
  constant during analysis
• For each session, flows are defined as active
  periods
• Each flow has a constant rate of b bits per
  second
• Flows are assumed to be Poissonian with
  exponential inter-arrival times and durations

65
Model Formulation

• Notations
• y(m) aggregate traffic on link at time m
• x(m) number of active flows on link at time m
• ? y(m) noisy measure of the aggregate traffic
  on link at time m
• x(m) estimate of x(m)
• pk(t) probability that number of active flows
  at time t is k

66
Traffic Estimation

• Generating function G(z,t), with the initial
  condition G(z,mT)zx(m)

where
67
Allocation Forecasting

• x(m) to forecast R(m1)
• Define and Q as the transition
  probability matrix

68
Performance Evaluation

• N20, lm0.005

69
Performance Evaluation (Contd.)
70
Big Picture of TEAM

• Traffic Engineering Automated Manager

Simulation Tool (ST)
Management Plane
DiffServ/ GMPLS Domain
Traffic Engineering Tool (TET)
LSP/lSP Setup/ Dimensioning
Resource
LSP Preemption
Route
LSP Routing
Measurement/ Performance Evaluation Tool (MPET)
Traffic Routing
Network Dimensioning and Topology Design
TEAM
To neighboring TEAM
71
TEAM Implementation

• TEAM has been implemented to run on a computer
  with the Linux OS.
• This testbed has been used as the platform to
  implement and test the operation of TEAM.

72
TEAM Top-level Design
Measurements
Routers
Topology updates
Scheduler
MRTG
Interface
Configuration
Trigger receiver
Configurerouters
Topology change
Update topology
Create/Destroy/Resize LSP
User interfaceserver
Commands
Label, path, priority, bandwidth
Label, path
New bandwidth request
Path, priority, bandwidth
New bandwidthrequest
LSP Setup
Create/Resize LSP
Path, priority,bandwidth
Preemption
Reroute
LSPs to be destroyed
Route
LSPs to be re-routed
Path, priority,bandwidth
Route
Route
73
TEAM Module Hierarchy
GRAPH
NET_SNMP
LSP_DB
REA
TOPOLOGY
PREEMPT
ROUTING
SNMP
RRDTOOL
LSP_SETUP
REQUEST_DB
MRTG
GSL
REQUEST
RE-ROUTE
ABEST
EVENTS
UI-PROTOCOL
SCHEDULER
CONFIG
UI-SERVER
COMMAND
MPET
TET
74
Performance Evaluation

• Topology with 40 nodes and 64 links of capacity
  600 Mbps
• Comparison with a traditional manager
• Shortest path routing for LSPs
• Shortest path routing for traffic
• LSP setup based on service level agreements
• No LSP preemption
• No on-line network measurements

75
Generalized Medium Traffic Load
Rejection Ratio
76
Generalized Medium Traffic Load
Minimum AB Average AB
77
Focused High Traffic Load
Priority 0 Rejection Priority 1 Rejection
78
Conclusions

• Development of TEAM, an automated manager for
  MPLS networks, that performs network design and
  adaptive network management including LSP and
  traffic routing, LSP setup and capacity
  allocation, etc. based on network measurements.

79
Future Work

• Heterogeneous large network management
• MPLS in Wireless Networks
• Network Tomography

80
Publications

1. Building an IP Differentiated Services Testbed,
   Proceedings of IEEE ICT, June 2001
2. A New Threshold-Based Policy for Label Switched
   Path Setup in MPLS Networks, Proceedings of
   17th ITC, September 2001
3. Optimal Policy for LSP Setup in MPLS Networks,
   Computer Networks Journal, June 2002
4. Design and Management Tools for an MPLS Domain
   QoS Manager, Proceedings of SPIE ITCOM, July
   2002
5. MABE A New Method for Available Bandwidth
   Estimation in an MPLS Network, Proceedings of
   IEEE NETWORKS, August 2002
6. A New Path Selection Algorithm for MPLS Networks
   Based on Available Bandwidth Estimation,
   Proceedings of QoFIS, October 2002
7. ABEst An Available Bandwidth Estimator within
   an Autonomous System, Proceedings of IEEE
   GLOBECOM, November 2002
8. A New Traffic Engineering Manager for
   DiffServ/MPLS Networks Design and Implementation
   on an IP QoS Testbed, Computer Communications
   Journal, March 2003
9. A New Scheme for Traffic Estimation and Resource
   Allocation for Bandwidth Brokers, Computer
   Networks Journal, April 2003
10. Adding QoS Protection in Order to Enhance MPLS
    QoS Routing,Proceedings of IEEE ICC, May 2003

81
Publications (Contd.)

• TEMB Tool for End-to-End Measurement of
  Available Bandwidth, Proceedings of IEEE ELMAR,
  June 2003
• QoS On-line Routing and MPLS Multilevel
  Protection A Survey,IEEE Communications
  Magazine, October 2003
• Optimal Filtering in Traffic Estimation for
  Bandwidth Brokers, Proceedings of IEEE
  GLOBECOM, December 2003
• LSP and lSP Setup in GMPLS Networks,Proceedings
  of IEEE INFOCOM, March 2004
• Threshold-Based Policy for LSP and ?SP Setup in
  GMPLS Networks, Proceedings of IEEE ICC, June
  2004
• New MPLS Network Management Techniques Based on
  Adaptive Learning,IEEE Transactions on Neural
  Networks, 2004
• Filtering and Forecasting Problems for Aggregate
  Traffic in Internet Links,Performance
  Evaluation Journal, 2004
• Traffic Routing in MPLS Networks Based on QoS
  Estimation and Forecast, submitted for
  publication
• TEAM A Traffic Engineering Automated Manager
  for DiffServ-based MPLS Networks,submitted for
  publication