EndtoEnd Internet Packet Dynamics

1
Endto-End Internet Packet Dynamics

• Author Vern Paxson
• In Proc. of ACM SIGCOMM, September 1997

Presenter Vishu Gupta
2
Studying the internet

• Internet is difficult to model
• Increasingly so
• Internet is heterogeneous
• No representative subset
• Previous studies
• No broad measurements
• Mostly fixed rate UDP or ICMP packets

3
Issues

• TCP difficult because effects of transport
  protocol and network get intertwined
• tcpanaly
• Unusual network behavior can skew results
• Out of order delivery
• Replication
• Packet corruption

4
Measurements

• 2 sets of measurements 35 sites
• N1 Dec 1994
• 2800 pairs of traces
• N2 Dec 1995
• 18000 pairs of traces
• Bigger window to potentially utilize all
  bandwidth
• 100 kb TCP bulk transfers
• Measurements occurred at Poisson intervals
• Done on both sender and receiver sides
• Differences give indication of change in packet
  dynamics over an year

5
Network Pathologies

• Unusual or unexpected behavior
• Doesnt adhere to the semantics
• Out-of-order delivery
• Packet replication
• Packet corruption

6
Out-of-order delivery

• Paxson considers late arrivals rather than
  premature arrivals
• Violates FIFO network abstraction
• Reordering is asymmetric sender cant infer
• So receiver side measurements help
• Route changes
• Most occurrences coincide with oscillations
• More data packet reordering
• Delayed ack
• Important because affects the duplicate ack
  threshold

7
No route change

• Huge reordering due to router forwarding lulls

8
Improving Fast Retransmit

• Ack reordering doesnt affect
• Trade-off between unnecessary retransmits and
  retransmit opportunities
• Rgb f(Nd, W)
• Delaying generation of dupacks (W)
• Small delay helps
• Altering the retransmission threshold (Nd)

9
Packet Replication

• Due to spurious retransmission
• Effect is spread-out across different sites
• Sender replicates packets
• True replication
• Site-specific

10
Packet Corruption

• network delivers to the receiver an imperfect
  copy of the original packet
• In this study most corruption was caused by T1
  links
• 1 in 5000 Internet data packets corrupted in
  transit
• TCP uses 16 bit checksum which means 1 in 65,536
  packets is accepted erroneously
• The result of which is 1 in 300 million packets
  sent will be accepted with corruption
• Paxson suggests increasing the TCP checksum to 32
  bits

11
Packet Corruption (contd..)

• Data checksum error rate of 0.02 is much higher
  than the pure ack rate
• one possibility - tcpanaly counts a packet as
  corrupted when it was actually never received
  (inadequate buffer space)
• Data packets may be more likely to be corrupted
  than acks because the corruption could occur
  inside routers and go undetected by link layer
  checksums
• Mechanism effective for packets greater than a
  particular size only
• Summary no definitive answer to overall Internet
  packet corruption rates

12
Measuring Bandwidth

• Available and bottleneck
• Packet pair

13
Bandwidth

• Bottleneck
• the upper limit
• Available
• Stability
• Self-interference time-constant

Qb b / ?B
14
Packet Pair Approach

• A pair of packets is sent
• Time difference between arrivals representative
  of the bottleneck (Qb)
• The two approaches
• Bolot directly
• Carter Crovella bprobe

15
Limitations Packet Pair

• Cant distinguish between forward and reverse
  path
• Noise delays due to generating the acks

16
Receiver Based Packet Pair

• Receiver has timing information
• Looks at time difference between arrivals
• Still..
• Packets taking different routes
• Clock resolution (k packets)
• Multichannel links

17
Robust Bottleneck Estimation

• Packet bunch modes
• Range of bunch sizes
• Accept multiple bottleneck estimates
• More than one mode
• Bottleneck change if spaced
• M-channel bottleneck if not
• Can have both

18
Explanation

• Plausible explanations for various peaks
• corresponding to different speed links
• Combination of parallel links
• Bottleneck bandwidth is asymmetric
• Differ by more than 20 more than 20 of the
  times
• Question mark on effectiveness of echo
  measurement techniques

19
Comparison

• SBPP agrees with PBM only 60 times
• RBPP estimates almost always (97-98) agree with
  PBM
• Packet Pair effective when
• PBMs clustering algorithms
• Packet-pair estimation at receiver
• Timing information available to receiver
• multi-channel effects not considered

20
Packet Loss

• Don't confuse measurement drops (??) with genuine
  losses
• Solution in tcpanaly is to determine whether
  traces suffer from measurement drops, and
  excluded those that do from packet loss analysis

21
Packet Loss

• 2.7 (N1) to 5.2 (N2)
• Due to bigger window?
• Data packet forward transmission is adapting, ack
  is not unless a whole flight of acks is lost -
  thus ack losses give a better picture of Internet
  loss patterns, while data loss shows pattern as
  perceived by TCP
• Ack loss 2.88 (N1) to 5.14 (N2)
• Conclusion - loss rates doubled during 1995

22
Packet Loss

• Two network states
• Quiescent no ack loss
• Busy ack loss
• Analysis by partitioning the geography into 4
  regions

23
Packet Loss

• Loss rate increase are primarily due to higher
  loss rates during busy periods
• Site to site variation within regions
• Current loss is a good predicator of loss over
  time (hours and perhaps days or weeks)
• Caching loss information should be beneficial
• This is just predicting loss or no loss
• The loss rate can not be predicted

24
Loaded and Unloaded packets

• Loaded data packet is one that had to queue at
  bottleneck link behind its predecessor
• Tis lt Ti-1s ?i-1
• Others called unloaded
• N1 N2 showed 2/3 of packets unloaded

25
Data Vs Ack packet loss

• Likelihood of loss Loaded packets gt acks gt
  unloaded packets
• Acks smaller in size
• Difference is that ack stream is not adapting to
  network conditions

26
Data Vs Ack packet loss

• Loss rates appear exponential
• They make measurements using a mechanism whose
  goal is to lower the value of what is being
  measured
• A better approach to measuring overall Internet
  packet loss rates would be non-adaptive sampling
• Link to exponential distribution?

27
Loss Bursts

• Packet loss in TCP can not be modeled as
  independent events (plu and plc)
• For example, ack packets loss rate jumps by a
  factor of seven if the previous ack packet was
  lost
• Paxson groups successive packet losses into
  outages (majority being longer than 200msec)

28
TCP Retransmissions

• 3 types of redundant retransmissions
• Unavoidable all acks lost
• Coarse feedback fixable using SACK
• Bad RTO
• rare in standard implementation
• Is its estimation overly conservative?

29
Packet Delay

• Analyze only one-way transit time(OTT)
• Packet delay modeled using gamma distribution
  Mukherjee
• Clock accuracy affects accurate assessment
• OTT cant be estimated by half of RTT because
  asymmetric

30
Timing compression

• Timing compression occurs when a flight of
  packets sent over an interval Ts arrive at the
  receiver over an interval lt Ts
• Later packets catch up a stalled predecessor
• Ack packet timing compression is rare
• Lead to faster data transmission rate
• Affects SBPPs estimation of bottleneck
• Data packet timing compression is rarer

31
Queuing time scales

• By removing suspect clocks, compressed timing,
  reordered packets, and traces with route changes,
  then remaining OTT variation reflect queuing
  delays
• ?Q? avg of ml - mr
• where ? timescale which maximizes ?Q?
• Internet delay variations are usually on the
  scale of 0.1 to 1.0 seconds, but can be larger

32
Available bandwidth

• How fast the connection should transmit to
  preserve network stability

? measured OTT min OTT ? ? - ? ? ?(? ?)
/ ?(? ?)

• ? reflects available bandwidth
• Current AB is a pretty good predicator of
  AB in the future (up to several hours)

33
Conclusion

• On removing filtering errors and TCP effects,
  TCP-based measurements can give an accurate look
  at end-to-end packet dynamics.
• Wide range of behavior. Be careful in trying to
  characterize typical dynamics
• Pathological cases do occur sometimes frequently
• TCP retransmission strategies work in a
  conservative fashion
• Sender only measurements are inferior to those
  that include receiver-cooperation