Bimodal Multicast

1
Bimodal Multicast

• Kenneth P. Birman, Mark Hayden, Oznur Ozkasap,
  Zhen Xiao,
• Mihai Budiu, Yaron Minsky
• Presented by Bruce Chin

2
Reliable Multicast

• Strong. Provide atomicity (all or none delivery),
  virtual synchrony, security, real-time
  guarantees. The downside is unpredictable
  performance under stress and limited scalability.
• Best effort. Offers better scalability but
  degrades guarantees for delivery.
• This paper presents bimodal multicast or
  probabilistic multicast (pbcast). Said to provide
  both scalability and predictable reliability.

3
Throughput Stability

• Needed for critical real-time applications
• Perturbations occur when participants are busy.
  Buffer space begins to fill up and messages have
  to be delayed.

4
Throughput Stability
5
Bimodal Multicast Features

• Atomicity - "almost all or almost none" instead
  of "all or nothing"
• Throughput stability - can tolerate some amount
  of perturbation
• Ordering - FIFO order on a per-sender basis
• Multicast stability - when a message is detected
  to satisfy the bimodal delivery guarantee, it can
  be garbage collected
• Detection of lost messages
• Scalability - costs are constant or grow slowly

6
Execution Environment

• Built on Ensemble platform
• Time is divided into execution periods in which
  group membership is static
• Period switching done when multicast is stable

7
Fault Model

• Percentage (75 e.g.) of messages are reliably
  sent. Errors are iid.
• Known, bounded delays
• Hard failures. Crashes and network partitions
• Soft failures. Failure to receive because of
  transient network conditions
• No handling of Byzantine failures

8
Pbcast Protocol, 2 Protocols in 1

• An unreliable, hierarchical broadcast (optimistic
  dissemination protocol) multicast or spanning
  tree broadcast. The choice of multicast protocol
  depends on the network (LAN or WAN) and the
  scalability required.
• 2-phase anti-entropy. Problems are detected and
  corrected by rounds of gossiping.

9
Example

• Q fails to receive M0, S fails to receive M1
• Recovered in anti-entropy phase
• Multicast and recovery rounds are concurrent, not
  as shown in figure

10
Anti-Entropy Phase

• Members randomly choose a recovery partner and
  say I just received these messages. (Send a
  digest of messages received.)b
• The recovery partner then will solicit any
  messages that it missed.
• To support real-time services, old messages are
  eventually lost if they cant be delivered.
• What about out-of-order delivery? How do you know
  that you can deliver a message?

11
Issues

• Are slow processes always going to fall behind
  and slow down other processes?
• What if a processes is bombarded with
  solicitations?
• How do you limit the costs of buffering?
  (Scalability)
• Random gossip through a router can be a
  bottleneck

12
Optimizations

• Soft failure detection retransmission requests
  handled only if received in the same round it was
  delivered
• Round retransmission limit
• Cyclic retransmissions avoids redundant
  retransmissions from previous rounds
• Most-recent-first retransmission
• Independent numbering of rounds no need to
  synchronize rounds among processes
• Random graphs for scalability full connectivity
  not needed
• Multicast for some retransmissions when
  multiple requests come in for the same packet,
  just multicast it

13
Hierarchical Gossiping

• Weight gossip so that probability of gossip to a
  remote cluster is smaller
• Can adjust weight to have constant load on router
• Now propagation delays rise but just increase
  rate of gossip to compensate

From Birman Lecture Slides
14
Hierarchical Gossiping
Latency to delivery (ms)
Load on WAN link (msgs/sec)
From Birman Lecture Slides
15
Computational Results

• Initial transfer unsuccessful. 5 messages lost.
  0.1 of processes crash during a run
• Predicate I More than 10 but less than 90 of
  the processes get the multicast
• Predicate II Roughly half get the multicast but
  crash failures might conceal outcome

16
Experimental Results
17
Experimental Results
18
Experimental Results
19
Conclusions

• Reliable multicast schemes typically fall into
  two camps
• Strong guarantees on delivery but throughput is
  not stable and is not scalable.
• Best-effort gives better throughput but at the
  cost of delivery guarantees
• pbcast good for applications that can tolerate
  some degree of lost information and a weakening
  of the atomic guarantee
• Streaming multimedia
• Stock market updates
• Air-traffic control
• Medical telemetry

20
Discussion

• What are the overhead costs associated with this
  protocol?
• How to handle multi-group multicasts?
• Could this be made to handle Byzantine failures?