Informed Content Delivery Across Adaptive Overlay Networks

1
Informed Content Delivery Across Adaptive Overlay
Networks

• J. Byers, J. Considine, M. Mitzenmacher and S.
  Rost
• Presented by Ananth Rajagopala-Rao

2
Motivation

• CDNs typically use overlay multicast and there is
  tremendous potential to leverage parallel
  downloads
• Encoded packets make it easy to coordinate
  transfers
• Need efficient algorithms to compute set
  differences between content stored at peers

3
Motivation (cont.)

1. Tree Topology
2. DAG
3. Fully collaborative scenario

4
The Digital Fountain Approach

• Gives a large unordered universe of encoded
  symbols
• Stateless Encoding Each packet is generated
  independent of the previously generated packets
• Additivity Parallel downloads from multiple
  sources with full content requires no
  orchestration

5
Sparse Parity Check Codes

• A file is a set of symbols of 1k blocks xi.
• Each encoded symbol is an XOR of a random set
  of these blocks.
• If we represent each symbol by a bit-vector of
  the blocks used to generate it, the rank of the
  matrix of bit-vectors of the symbols a host has
  is the of blocks it can recover.

6
Sparse Parity Check Codes (cont.)

• Re-encoding is very easy, and XOR of any set of
  encoded symbols is also a valid symbol.
• For carefully engineered distributions (taking
  into account the overhead of encoding), the
  expected overhead is approximately 3 to 5 percent.

7
Reconciliation of Content Between Peers

• A set of symbols that A has
• B set of symbols that B has
• We have to compute A-B (approximately?)
• If n is the number of symbols needed to
  reconstruct the file, the interesting numbers are
• o A-B/A
• r (n - B)/A-B

8
Estimating o

• Use random sampling, send a sample of k elements
  of A to peer B.
• Use min-wise summaries (from search engine
  literature for document similarity estimation).
• Give a good tradeoff between accuracy and amount
  of data to send.
• Trivial to combine min-wise summaries of A and B
  to get summary of (A U B).

9
Min-wise summaries
10
Exact Approaches (high r and high o)

• A sends entire set to B - O(A log u)
• Use a hash function, we make the miss-probability
  arbitrarily low for a packet of size O(A log
  A)
• Very high overhead

11
Bloom Filter Approach (low r and low o)

• Choose i hash functions h1, h2.. hi with range
  0,m).
• Construct a bitmap of m bits where for each
  element x of the set, hj(x) is set for all j in
  1,i.
• Probability of false positive is
• f (1-e-kA/m)k
• Low bandwidth overhead, high computation overhead
  of O(B).

12
Approximate Reconciliation Trees

• Low overhead bandwidth.
• Allows for more efficient computation than bloom
  filters
• O((A-BB-A)logB)
• Use bloom filters on top of trees similar to
  Merkle trees.
• What about the computation overhead at node A??

13
ART (cont.)
14
ART (performance)
15
One full sender and one partial sender
16
Four Partial Senders
17
Conclusions

• A whole bag of interesting algorithmic tools is
  presented, but it is not clear what is the best
  way to combine and use all these tools.
• Suggests some new and interesting directions of
  research in P2P, overlay multicast and CDNs.