WEBRC using MRTT

1
WEBRC using MRTT
Vivek K Goyal and Michael Luby Digital
Fountain Fremont, CA With thanks to Robert
Chapman, Armin Haken, Diane Hernek, Gavin Horn,
and Simon Skaria
2
Goals of Talk

• What is WEBRC?
• What is MRTT?
• Reliability can be separated from rate control
• WEBRC
• Is scalable to large audiences
• Uses bandwidth efficiently
• Does not strain network elements
• Is fair to competing TCP flows
• (Is independent of reliability)

3
Outline

• Reliable Delivery
• Digital Fountain Approach
• WEBRC
• MRTT
• Simulations
• Summary

4
One-to-One Reliable Delivery

• UDP transport is best effort
• TCP transport
• Provides reliability and congestion control
• Entangled functionality

Content at server
A
G
F
E
D
C
B
F
C
E
D
Packet flow
D
C
D
A
B
Content at client
5
One-to-Many Reliable Delivery

• IP multicast is massively scalable
• IP multicast is best effort
• Question how can we add
• Reliability
• Congestion control
• Maintain massive scalability?

6
Outline

• Reliable Delivery
• Digital Fountain Approach
• WEBRC
• MRTT
• Simulations
• Summary

7
Digital Fountain Approach

• Reliability
• LT codes (beyond FEC)
• Send encoded data in packets instead of original
  data
• Doesnt matter which packets are received
• Only matters that enough packets are received
• Congestion control
• WEBRC
• Send packets to multiple channels
• Receivers join and leave channels to adjust
  reception rate

8
LT Codes

• It doesnt matter which droplets fill the cup.

• It only matters that the cup is filled.

9
LT Codes
10
LT Codes
11
Fountains Can Be Pooled

• Concurrent reception from multiple locations
• Fast recovery from network or server failures

Client regenerates content by combining packets
from multiple Fountains
12
Fountains Can Use Multiple Channels
13
Digital Fountain Advantages

• Completely separated functionality
• Reliability protocol (LT codes)
• Congestion control protocol (WEBRC)
• Massively scalable reliable content delivery
• A single sender is adequate
• No feedback from receivers to the sender
• A single sender for millions of concurrent
  receivers
• Minimizes delivery time for each receiver
• Receivers can start asynchronously
• Efficiently uses bandwidth between each receiver
  and sender
• Minimal amount of data needs to arrive at each
  receiver to recover content
• Friendly to competing flows

14
Outline

• Reliable Delivery
• Digital Fountain Approach
• WEBRC
• MRTT
• Simulations
• Summary

15
Related Techniques
RLM
equation-based unicast
TFRC
RLC
TFMCC
FLID-DL
multiple rate multicast
WEBRC
equation-based multicast
16
Related Techniques

• Receiver-driven Layered Multicast
• McCanne, Jacobson Vetterli (1996)
• for packet video
• hierarchical layers
• leave when congestion experienced
• join experiments to increase rate
• Weaknesses
• leave latency causes prolonged
• periods of oversubscription
• join experiments harm other
• receivers sharing
• bottleneck
• coarse rate control
• poor TCP fairness

RLM
RLC
FLID-DL
WEBRC
17
Related Techniques

• Receiver-driven Layered Congestion
• Control Vicisano, Rizzo Crowcroft (1998)
• Main innovations
• join experiments synchronized
• simulated joins reduce
• oversubscription
• Weaknesses
• oversubscription still prolonged
• rate control is coarse
• TCP fairness not great

RLM
RLC
FLID-DL
WEBRC
18
Related Techniques

• Fair Layered Increase/Decrease
• with Dynamic Layering Byers et al. (2000)
• Main innovations
• layers not hierarchical
• dynamic layers make leave
• latencies less important
• Weaknesses
• rate control somewhat coarse
• TCP fairness not great
• no slow start
• mechanism
• high packet loss
• high number of channels, join frequency

RLM
RLC
FLID-DL
WEBRC
19
Related Techniques

• TCP Friendly Rate Control
• Floyd et al. (2000)
• equation-based unicast
• rate control
• RTT measured by sender
• loss event rate measured by
• receiver and fed back to sender
• average throughput matches
• TCP very closely

TFRC
TFMCC
WEBRC
20
Related Techniques

• TCP Friendly Multicast Congestion
• Control Widmer Handley (2001)
• TFRC for single rate multicast
• RTT measured by receiver
• less relevant feedback inhibited
• to prevent implosion

TFRC
TFMCC
WEBRC
21
WEBRC Overview

• Wave and Equation Based Rate Control
• Wave
• Packets are transmitted in waves
• Wave channels carry periodically spaced waves
• Equation
• Receivers target reception rate is computed with
  the TCP equation
• MRTT plays role of RTT
• MRTT and packet loss event rate measured by
  receiver

22
Making WavesThe Sender
Wave channels
...
...
0
1
2
MWCR
BCR
P BCR
Base channel
TSD
T TSD
Q TSD
...
...
TSI
T-1
0
1
2
23
Making WavesThe Sender
rate
0
time
24
Catching WavesThe Receiver
rate
The receiver catches the end of every wave
Areas packets on a channel Height cumulative
rate
BCR
0
time
25
Catching WavesThe Receiver
rate
Joining waves slightly earlier increases the rate
BCR
0
time
26
Catching WavesThe Receiver
rate
Joining waves slightly later decreases the rate
BCR
0
time
27
Catching WavesThe Receiver
rate
Sudden increase in competing traffic may delay
joining, and a new equilibrium is found
BCR
0
time
28
Catching WavesThe Receiver
rate
Slow start mechanism the rate ramps
up exponentiallylike TCP, but less aggressively
BCR
0
time
29
WEBRC Receiver Behavior
To join or not to join?
30
WEBRC Receiver Behavior

• Always joined to the base channel
• Time slot indices carried in all packets
• Relative ordering of channels changes at time
  slot boundaries (every 10 seconds)
• Receiver tracks
• Anticipated reception rate (ARR)
• Loss event rate (p) and multicast round trip time
  (MRTT) to calculate target rate (TR)
• Use sequence numbers per channel to detect losses
• Measure MRTT each time a wave channel is joined
• At each epoch boundary (every 0.5 seconds), join
  if doing so doesnt cause ARR to exceed TR
• Facilitated by waves
• Leave wave channels when waves end

31
WEBRC Receiver Behavior
80
target
anticipated
70
true (smoothed)
60
50
Packets/second
40
30
20
10
0
150
200
250
time
32
The Target Rate

• Equation-based rate
• p Loss event rate
• MRTT Multicast round trip time (averaged)
• REQN Rate in packets per second
• Target rate
• SSR slow start rate
• MINRATE configured minimum rate
• MAXRATE configured maximum rate

33
WEBRC Loss Events

• A loss event lasts for a period of MRTT
• Loss within loss event ignored
• Loss could be ECN flag reception
• Count interloss packets between starts of loss
  events
• Loss event rate p 1/(weighted ave. of interloss
  counts)
• Computations adjusted for wave structure

9
6
6
8
loss event
loss event
loss event
loss event
loss event
Received packet
Lost packet
34
Before there is a target rate

• Cant have a meaningful target rate at the
  beginning of a session
• Increase rate quickly by joining waves (roughly)
  every other epoch, but look out for indications
  to slow down!
• Check for increases in MRTT measurements
• Check for lag in reception rate compared to ARR
• Initialize loss event rate to match postulated
  bandwidth
• Can be much less overshoot than a slow start
  mechanism based only on packet losses

35
And finally, the fine print

• Because of the shapes of the waves, rate should
  decrease between joins
• If rate doesnt decrease, buffers may be emptying
• Dont join unless rate has decreased since the
  last join
• Can stabilize the reception rate without packet
  losses
• Can keep queues emptier
• Most likely invoked when WEBRC is only flow over
  a bottleneck

36
Outline

• Reliable Delivery
• Digital Fountain Approach
• WEBRC
• MRTT
• Simulations
• Summary

37
MRTT

• MRTT is a natural multicast extension of RTT
• Measured individually by each receiver
• Time between join and receipt of packet flow
• Exponentially weighted average used in equation
• MRTT value may differ for different receivers

38
MRTT Measurement
sender S

• Raw MRTT measurements depend on when the
  receivers join the wave
• MRTT can be as small as RTT
• to/from the nearest router
• MRTT can be as large as RTT
• to/from the sender

X
R
Y
Z
receiver A
receiver B
39
MRTT Terminology
sender S

• Wave time
• At any point in the network, the time relative to
  the passage of a wave
• Join time
• Wave time at join
• Corresponds to rate
• Equal join time means joining at same point in
  descent of wave

0.1
0.1
R
0.2
0.2
0.3
Rec. A
0.3
Rec. B
40
MRTT Equilibrium
sender S

• S X Y Z
• If Y-Z lt X then at equilibrium
• MRTT(A) MRTT(B)
• MRTT(A)MRTT(B) S
• i.e., MRTT(A)MRTT(B)S/2

X
R
Y
Z
receiver A
receiver B
41
MRTT Example 1
Equal JTs lead to equal MRTTs
Receiver with the earliest JT has largest MRTT
this lowers the target rate and increases JT for
later waves.
sender
sender
1.0
JT 0.8 MRTT 0.6
JT 0.8 MRTT 1.2
A
A
router
router
0.1
0.1
JT 0.8 MRTT 0.6
JT 0.8 MRTT 1.2
B
B
router
router
JT 0.8 MRTT 0.6
JT 0.8 MRTT 1.2
C
C
router
router
JT 0.0 MRTT 1.4
JT 0.8 MRTT 1.2
D
D
router
router
JT 0.8 MRTT 0.6
JT 0.8 MRTT 1.2
E
E
router
router
All links except those marked have 0.1 second
latency
42
MRTT Example 2
Receivers with later JTs have smaller MRTTs.
This increases the target rate and decreases JT
for later waves. JTs converge, receivers
synchronize
sender
X
switch
Y
A
B
C
JT e2 MRTT XY-e2
JT e1 MRTT XY-e1
JT 0 MRTT XY
Assumes 0 lt e1, e2 lt 2X
43
TCP session utility
sender
T1
Cost time to deliver packet RTT T1
T2 Tn Benefit 1 receiver TCP session
utility Benefit/Cost 1/RTT Bottleneck share ?
TCP session utility
router
T2
router
T3
router

router
Tn
receiver
44
WEBRC session utility
Cost time to deliver packet S T1
Tn Benefit k receivers At equilibrium,
MRTT S/k WEBRC session utility Benefit/Cost
k/S 1/MRTT Bottleneck share ? WEBRC session
utility
sender
T1
router
T2
router
T8
T3
router
router
T9
T6
T4
router
router
router
Tn
T5
T7
receiver 1
receiver 2
receiver k
45
MRTT Summary

• Natural definition
• MRTT is the RTT between receiver and sender for a
  multicast session with one receiver
• Measures additional time for delivery unique to
  receiver
• Fair sharing of bottleneck links
• WEBRC/WEBRC and WEBRC/TCP
• Proportional to session utility
• Stabilizes reception rates
• Similar to RTT for TCP
• Reception rates decrease in response to buffer
  filling
• Proximate receiver reception rates coordinate and
  equalize
• Maximizes utilization of packets that flow
  through link
• Reception rates increase with receiver density
• More receivers implies higher throughput for all
  receivers

46
Outline

• Reliable Delivery
• Digital Fountain Approach
• WEBRC
• MRTT
• Simulations
• Summary

47
Simulations

• Sender and receiver implemented in ns-2
• Packet data payloads 1024 bytes
• Base channel rate 8192 bps
• Time slot duration 10 sec
• Multiplicative rate decrease per time slot P
  0.75
• Epoch length (receiver decision interval) 0.5 sec
• Number of channels depends on the maximum rate
  and number of quiescent channels
• Ex max. rate of 10 Mbps requires 20 active waves

48
Simulations with Random Loss
0.03 packet loss
0.2 sec MRTT
49
Simulations with Random Loss
ns simulations (8 simulations for each parameter
pair)
Predicted by equation (adjusted for peak vs. mean)
50
Simulations with Random Loss - TCP
TCP has more variance
ns simulations (8 simulations for each parameter
pair)
Predicted by equation
51
WEBRC and TCP session share 1 Mbps bottleneck
475 kbps
422 kbps
52
WEBRC and TCP session share 1 Mbps bottleneck
482 kbps
434 kbps
53
Coordination of Receivers
1
sender
0.5
Throughput (Mbps)
200 ms
1 loss
0
0
100
200
300
400
500
R
0.02
50 ms
LOSSP
0.01
Rec. A
0
0
100
200
300
400
500
100 ms
1
ARTT
0.5
Rec. B
0
0
100
200
300
400
500
Time (sec)
54
Coordination of Receivers
1
sender
0.5
Throughput (Mbps)
200 ms
1 loss
0
0
100
200
300
400
500
R
0.02
50 ms
LOSSP
0.01
Rec. A
0
0
100
200
300
400
500
100 ms
1
ARTT
0.5
Rec. B
0
0
100
200
300
400
500
Time (sec)
55
Efficient Multicast Replication
10
sender
8
6
198 ms
2 loss
4
R
2
2 ms
0


Rec. 1
Rec. n

Bottleneck utilization fraction
Number of receivers
n
56
Outline

• Reliable Delivery
• Digital Fountain Approach
• WEBRC
• MRTT
• Simulations
• Summary

57
WEBRC Summary

• Fewer channels
• Factor of 20 fewer quiescent channels than
  FLID-DL
• Granularity of rate control
• Fine rate control, almost all rates equally
  natural
• Slow start mechanism
• Rate can ramp up quickly
• Large overshoot avoided
• Each receiver can individually adjust
• Maximum reception rate
• Minimum reception rate

58
WEBRC Summary

• Applies to multicast and unicast
• Unicast only one feedback packet each 10
  seconds
• Multicast only one join and leave each 10
  seconds
• Fair to TCP
• Uses newly introduced concept of MRTT
• WEBRC receiver is fair to TCP receiver with same
  RTT
• Network friendly
• Factor of 20 fewer IGMP join/leave messages than
  FLID-DL
• Causes least amount of packet loss of all
  proposed receiver-driven solutions
• Packet loss comparable to that for TCP

59
Further Information

• Paper at ACM SIGCOMM02 (August 2002)
• See http//lcavwww.epfl.ch/goyal/Papers/sigcomm02
  .pdf
• WEBRC simulation website
• WEBRC ns code and simulation post-processing
  tools
• See http//www.digitalfountain.com/technology/
  later this month
• Reliable Multicast Transport building block
• Submitted 02 revision to working group June 2002
• http//www.ietf.org/internet-drafts/draft-ietf-rmt
  -bb-webrc-02.txt
• Integration into Digital Fountain products
• Transporter Fountain 2.0 available 15 Aug 2002
• Both multicast and unicast