Traffic characterization and analysis

1
Traffic characterization and analysis

• Nicolas Larrieu, Philippe Owezarski
• LAAS-CNRS
• Toulouse, France
• nlarrieu, owe_at_laas.fr

2
Outline

• Introduction to Internet monitoring
• Needs, principles and applications
• State of the art on active and passive technics
• Internet traffic characterisitics and related
  performance problems
• Traffic mutation P2P impact
• Treats for the Internet oscillations, LRD
• How to use monitoring results?
• Application to traffic engineering a new
  congestion control mechanism

3
Part 1

• Introduction to Internet monitoring
• Needs, principles and applications
• State of the art on active and passive technics
• Internet traffic characterisitics and related
  performance problems
• Traffic mutation P2P impact
• Treats for the Internet oscillations, LRD
• How to use monitoring results?
• Application to traffic engineering a new
  congestion control mechanism

4
Introduction

• Deals with both monitoring results and effects on
  network design, research and management
• Topic under the spotlight
• Definition  science of measurements 
• To apply to networks in general, to the Internet
  in particular

5
Monitoring and networks

• Explosion of the Internet (in size)
• New services
• QoS
• Increasing complexity of the Internet
• No more control of the global Internet

6
Monitoring and networks

•  Defeat  statement
• For instance on QoS aspects
• Bad knowledge of the traffic
• Because of the current Internet complexity, its
  evolution requires a good knowledge of the
   real  traffic
• Real  traffic exists and is full of
  information

7
Example of a provisioning problem
8
What to use for network monitoring?

• Administration / operation tools based on SNMP
• Topology of networks / configuration
• Some statistics measurements
• Granularity is too coarse min 5 s (but can be
  1 hour, 1 day, 1 week or whatever)
• Measured parameters are more or less the amount
  of traffic sent and received

9
Some examples of SNMP results (1)
RAP ? RENATER interconnection
Per hour trace
Input traffic
Output traffic
10
Some examples of SNMP results (2)
Per Month trace
Per Week trace
Input traffic
Output traffic
11
Problems for monitoring networks

• Impossible to monitor traffic dynamics (second
  order values as variability auto-covariance for
  instance)
• Impossible to monitor traffic QoS (user point of
  view goodput)
• Impossible to get a (formal) traffic model

12
Example on network provisioning

• Common beliefs tell us traffic is Poisson
• EXl
• VXl
• Provisioning should be 2l
• Actually, provisioning has to be at least 13
  (i.e. 3l)
• RENATER 13
• Sprint 13
• WorldCom 15
• ATT 110

13
Questions on the example

• How explaining this over-provisioning requirement
  ?
• How to predict the traffic that will be supported
  by a new network to design ?

?
14
Part 2

• Introduction to Internet monitoring
• Needs, principles and applications
• State of the art on active and passive technics
• Internet traffic characterisitics and related
  performance problems
• Traffic mutation P2P impact
• Treats for the Internet oscillations, LRD
• How to use monitoring results?
• Application to traffic engineering a new
  congestion control mechanism

15
Active measurements

• Principles
• Consists in sending packets on a network and
  observing results (Delay, RTT, Throughput, etc.)
• User point of view
• Best solution to evaluate the service you can get
  from the network youre connected to
• Drawbacks
• Probe packets change the state of the network
• ?IETF IPPM WG is working on the definition of
  probing scenarios minimizing the effects on the
  network state

16
Some active measurement tools

• Ping
• Traceroute
• MGEN
• RIPE equipments
• Etc.
• Importance of clock synchronization most of the
  time GPS is required

17
Passive measurements

• Principles
• Capture packets (or headers)
• Not intrusive at all
• Carrier / ISP point of view
• Best solution for a carrier to measure traffic
• Drawbacks
• Sampling issues
• Difficult to get a user point of view
• Technical limits (speed of components, capacity)

18
On line vs. Off line measurements

• On line
• Packets are analyzed in real-time
• Analysis on very long periods
• But complexity of analysis is quite limited
• Off line
• Packets are stored on hard drives / SAN for later
  analysis
• Possibilities of analysis are endless
• Possibility of correlating several traces
• But amount of stored data is really huge (small
  periods only)

19
Some passive measurement tools

• TSTAT
• LIBCAP
• Tcpdump
• Tcptrace
• QoSMOS
• IPANEMA
• CISCOs Netflow
• DAG cards
• Etc.

20
Part 3

• Introduction to Internet monitoring
• Needs, principles and applications
• State of the art on active and passive technics
• Internet traffic characterisitics and related
  performance problems
• Traffic mutation P2P impact
• Treats for the Internet oscillations, LRD
• How to use monitoring results?
• Application to traffic engineering a new
  congestion control mechanism

21
Current Internet traffic

• New applications with various and changing
  requirements in terms of QoS appear
• New P2P applications make traffic characteristics
  change

22
Internet traffic evolution (May 2000)
23
Internet traffic evolution (August 2000)
24
Internet traffic evolution (May 2003)
Throughput (kbits/s)
Time
25
TCP flow size
90
80
70
60
50
Percentage
40
30
20
10
0
0-10
10-100
gt100
Number of packets per flow
26
TCP flow size vs. total bandwidth
90
?  Mice vs. elephants 
80
70
60
50
Percentage
40
30
20
10
0
0-10
10-100
gt100
Number of packets per flow
27
Impact of P2P on traffic

• Thousands of mice
• A large number of elephants
• ? Change flow size distribution

28
Flow size distribution
29
Traffic oscillation issues
30
Illustration LRD and losses
31
Illustration LRD and losses
32
Illustration LRD and losses
33
Illustration LRD and losses
34
Illustration LRD and losses
35
Wavelett analysis of the traffic
Range
Time
36
Log-log scale variance diagram

• 2 different power laws
• 2 different rating for scale invariance
• High scales
• H 0.5
• LRD characteristics

37
Oscillations persistence characterization
H 0.741
38
Traffic characteristics bibliography

• Actual Internet traffic has self-similarity and
  long range dependancy properties
• Due to
• Heavy-tailed distribution of flow size
• TCP-like congestion control mechanism
• Routers
• Human and application behaviors

39
Self-similarity
40
Illustration of self-similarity

• Worst case with a highly self-similar traffic
• EX l
• VX ? O(l2)
• Example
• If EX 100 Mbps ? Provisioning for a reliable
  network with low delays is in the order of 10
  Gbps
• Reach the technological limits of networks with
  an average traffic of few hundreds of Mbps

41
Conclusion on traffic evolution

• Actual Internet traffic is not Poisson
• Well, it is probably not self-similar at all
  scales?
• But it is really LRD (and this is a major issue!)

42
Part 4

• Introduction to Internet monitoring
• Needs, principles and applications
• State of the art on active and passive technics
• Internet traffic characterisitics and related
  performance problems
• Traffic mutation P2P impact
• Treats for the Internet oscillations, LRD
• How to use monitoring results?
• Application to traffic engineering a new
  congestion control mechanism

43
Measurement Based

• Monitoring/measurements become the corner stone
  of research in networking
• Many topics are addressed
• New protocols and architectures for
• Traffic characterization and modeling
• Multi-domains QoS guaranty
• Service and network utilization optimization
• Network or VPN or CoS provisioning
• QoS routing
• Network security, etc.
• Techniques and mechanisms for
• Pricing, etc.

44
Multiple causes for Internet oscillations

• TCP like congestion control mechanisms (Slow
  Start and Congestion Avoidance mechanisms /
  Closed control loop)
• Increase of transmitted file size
• Increase of network capacities (and
  over-provisioning)
• ? Increase of oscillations
• Amplitude
• Range

traffic oscillations limit network
performance  High variability  paper of
Willinger (IEEE ToN 96)
45
Smoothing flow behavior

• Disturbances are mainly due to elephants
• We need to increase elephant flows regularity
• ? We are going to use the TFRC mechanism to
  transmit elephant flows
• TFRC proposes a smooth sending rate with very
  soft increases and decreases

46
TFRC impact on flow QoS throughput analysis
47
TFRC impact on flow QoS LRD analysis
Real traffic
Simulated traffic
48
Conclusion

• Monitoring give us very important information on
  traffic characteristics and related performance
  problems
• Traffic engineering can be really improve with
  monitoring results
• Lots of network research fields can be
  investigated and improved
• QoS, security, pricing, networking

49
Thank you for your attentionAny questions?
Contact nlarrieu_at_laas.fr