Digital%20Fountains:%20Applications%20and%20Related%20Issues

1
Digital FountainsApplications and Related Issues

• Michael Mitzenmacher

2
Goals

• Explain the digital fountain paradigm for network
  communication.
• Examine related advances in coding.
• Summarize work on applications.
• Speculate on what comes next.
• For more, see
• Digital Fountains A Survey and Look Forward
• www.eecs.harvard.edu/michaelm/ListByYear.html

3
What is a Digital Fountain?

• For this talk, a digital fountain is an
  ideal/paradigm for data transmission.
• Vs. the standard (TCP) paradigm data is an
  ordered finite sequence of bytes.
• Instead, with a digital fountain, a k symbol file
  yields an infinite data stream once you have
  received any k symbols from this stream, you can
  quickly reconstruct the original file.

4
How Do We Build a Digital Fountain?

• We can construct (approximate) digital fountains
  using erasure codes.
• Including Reed-Solomon, Tornado, LT, fountain
  codes.
• Generally, we only come close to the ideal of the
  paradigm.
• Streams not truly infinite encoding or decoding
  times coding overhead.

5
History

• Reed-Solomon codes
• Tornado Codes
• Luby Transform
• Rateless/Raptor codes

6
Raptor/Rateless Codes

• Properties
• Infinite supply of packets possible.
• Need k(1e) symbols to decode, for some e gt 0.
• Decoding time proportional to k ln (1/e).
• On average, ln (1/e) (constant) time to produce
  an encoding symbol.
• Conclusion these codes can be made very
  efficient and deliver on the promise of the
  digital fountain paradigm.

7
Applications

• Reliable multicast
• Parallel downloads
• Long-distance transmission (avoiding TCP)
• One-to-many TCP
• Content distribution on overlay networks
• Streaming video

8
Reliable Multicast

• Many potential problems when multicasting to
  large audience.
• Feedback explosion of lost packets.
• Start time heterogeneity.
• Loss/bandwidth heterogeneity.
• A digital fountain solves these problems.
• Each user gets what they can, and stops when they
  have enough.

9
Downloading in Parallel

• Can collect data from multiple digital fountains
  for the same source seamlessly.
• Since each fountain has an infinite collection
  of packets, no duplicates.
• Relative fountain speeds unimportant just need
  to get enough.
• Combined multicast/multigather possible.
• Can be used for BitTorrent-like applications.

10
Point-to-Point Data Transmission

• TCP has problems over long-distance connections.
• Packets must be acknowledged to increase sending
  window (packets in flight).
• Long round-trip time leads to slow acks, bounding
  transmission window.
• Any loss increases the problem.
• Using digital fountain TCP-friendly congestion
  control can greatly speed up connections.
• Separates the what you send from how much you
  send.
• Do not need to buffer for retransmission.

11
One-to-Many TCP

• Setting Web server with popular files, may have
  many open connections serving same file.
• Problem has to have a separate buffer, state
  for each connection to handle retransmissions.
• Limits number of connections per server.
• Instead, use a digital fountain to generate
  packets useful for all connections for that file.
• Separates the what you send from how much you
  send.
• Do not need to buffer for retransmission.
• Keeps TCP semantics, congestion control.

12
Distribution on Overlay Networks

• Encoded data make sense for overlay networks.
• Changing, heterogeneous network conditions.
• Allows multicast.
• Allows downloading from multiple sources, as well
  as peers.
• Problem peers may be getting same encoded
  packets as you, via the multicast.
• Not standard digital fountain paradigm.
• Requires reconciliation techniques to find peers
  with useful packets.

13
Video Streaming

• For near-real-time video
• Latency issue.
• Solution break into smaller blocks, and encode
  over these blocks.
• Equal-size blocks.
• Blocks increases in size geometrically, for only
  logarithmically many blocks.
• Engineering to get right latency, ensure blocks
  arrive on time for display.

14
Other Applications

• Other possible applications outside of networking
• Storage systems
• Digital fountain codes for errors
• Others???

15
Putting Digital Fountains To Use

• Digital fountains are out there.
• Digital Fountain, Inc. sells them.
• Limitations to their use
• Patent issues.
• Perceived complexity.
• Lack of reference implementation.
• What is the killer app?

16
Patent Issues

• Several patents / patents pending on irregular
  LDPC codes, LT codes, Raptor codes by Digital
  Fountain, Inc.
• Supposition the theory/practice of digital
  fountains was greatly developed by the company
  and its employees.
• Supposition but this stifles external
  innovation.
• Potential threat of being sued.
• Potential lack of commercial outlet for research.
• Suggestion need unpatented alternative that
  approximate a digital fountain.
• There is work going on in this area, but more is
  needed to keep up with recent developments in
  rateless codes.

17
Perceived Complexity

• Digital fountains are now not that hard
• but networking people do not want to deal with
  developing codes.
• A research need
• A publicly available, easy to use, reasonably
  good black box digital fountain implementation
  that can be plugged in to research prototypes.
• Issue patents.
• Legal risk suggests such a black box would need
  to be based on unpatented codes.

18
Whats the Killer App?

• Multicast was supposed to be the killer app.
• But IP multicast was/is a disaster.
• Distribution now handled by content distributions
  companies, e.g. Akamai.
• Possibilities
• Overlay multicast.
• General wireless e.g. video phones.
• Specialized wireless e.g. automobiles.
• Others???

19
Conclusions

• Digital fountain paradigm and enabling codes have
  significant potential.
• Many proposed applications.
• More to come.
• Applications helped push forward the technology.
• Codes with better and better properties.
• Challenge in moving from a technology to use in
  the real-world.
• A simple, easy-to-use implementation based on
  non-proprietary might spur research community.
• Need more potential killer apps to spur business
  community.