End-to-end Publishing Using Bittorrent

1
End-to-end Publishing Using Bittorrent
2
Bittorrent

• Bittorrent is a widely used peer-to-peer network
  used to distribute files, especially large ones
• It has a number of legal uses which separate it
  from other P2P

3
Practical Applications

• Distributing large files
• Podcasting
• Vlogging
• Disk images
• Legal distribution of movies (see bittorrent.com)

4
Traditional vs. Bittorrent

• One server provides many clients

• Many clients provide many clients

5
Terminology

• Swarm clients downloading or uploading a given
  file through BIttorrent
• Tracker centralized server that clients connect
  to to ask for lists of other clients connected to
  the swarm
• Seed A client that has a complete copy of the
  file
• Peer (Leecher) A client that does not have a
  complete copy of the file

6
Problem

• Torrents that are less popular may eventually
  die when there are no longer any complete
  copies of the file in the swarm

7
Everseed

• Permanent seed running on the same server as the
  tracker
• Guarantees that there will always be a complete
  copy of the file

8
Related Research

• The creator of Bittorrent wrote a paper on the
  process of downloading a file using Bittorrent at
  http//www.bittorrent.org/protocol.html
• Maintainers of various Bittorrent clients wrote
  http//wiki.theory.org/BitTorrentSpecification,
  which is like the official specification except
  far more in depth
• Osprey (http//osprey.ibiblio.org/) seems to have
  thought of something similar, but haven't made
  much progress

9
Explanation

• The .torrent metadata file provides info about
  where to find the tracker and about the file
  being distributed
• Client connects to tracker
• Tracker gives client a list of other clients
• Client then downloads file from other clients
  (not a centralized server)
• Periodic update with tracker for new client list

10
Goals

• Complete internet publishing solution using
  Bittorrent
• Metadata file generator (.torrent)
• Tracker
• Everseed
• Web interface

11
.torrent File

• Official documention on bittorrent.org
• Metadata on the file to be downloaded (tracker
  URL, filename, size, checksum hashes)
• Stored as bencoded strings, integers, lists,
  dictionaries

12
Bencoding

• Integer 6 gt i6e
• String hello gt 5hello
• List hello,world gt l5hello5worlde
• Dictionary helloworld gt
  d5hello5worlde

13
Bencoding implementation

• Python, good string manipulation
• Structure of a .torrent file is a dictionary
  containing string keys and integer, string, list,
  and dictionary values
• Recursion to encode/decode

14
Tracker

• Makes use of the bencoding algorithm
• Handles two types of requests announce and
  scrape
• Stores data on peers and torrents in a SQLite
  database
• No performance issues

15
Network performance
Peer List Size
16
Database performance
17
Announce request

• Client sends an HTTP GET request to the announce
  URL in the .torrent file
• Tracker parses request, urldecodes data about the
  peer
• Tracker stores data in the database, sends
  appropriate response as bencoded string in a
  text/plain document
• Client bdecodes string, connects to other clients

18
Summary

• Python
• Benefits of P2P technology
• Everseed concept
• .torrent files and bencoding
• Tracker