Endtoend Publishing Using Bittorrent

1
End-to-end Publishing Using Bittorrent
2
Bittorrent

• Bittorrent is a widely used peer-to-peer network
  used to distribute files, especially large ones
• It has a number of legal uses which separate it
  from other P2P

3
Practical Applications

• Distributing large files
• Podcasting
• Vlogging
• Disk images
• Legal distribution of movies (see bittorrent.com)?

4
Traditional vs. Bittorrent

• One server provides many clients

• Many clients provide many clients

5
Terminology

• Swarm clients downloading or uploading a given
  file through Bittorrent
• Tracker centralized server that clients connect
  to to ask for lists of other clients connected to
  the swarm
• Seed A client that has a complete copy of the
  file
• Peer (Leecher) A client that does not have a
  complete copy of the file

6
Problem

• Torrents that are less popular may eventually
  die when there are no longer any complete
  copies of the file in the swarm

7
Everseed

• Permanent seed running on the same server as the
  tracker
• Guarantees that there will always be a complete
  copy of the file

8
Related Research

• The creator of Bittorrent wrote a paper on the
  process of downloading a file using Bittorrent at
  http//www.bittorrent.org/protocol.html
• Maintainers of various Bittorrent clients wrote
  http//wiki.theory.org/BitTorrentSpecification,
  which is like the official specification except
  far more in depth
• Osprey (http//osprey.ibiblio.org/) seems to have
  thought of something similar, but haven't made
  much progress

9
Explanation

• The .torrent metadata file tells client tracker
  URL other data
• Client connects to tracker
• Tracker gives client a list of other clients
• Client downloads file from other clients (not a
  centralized server)?
• Periodic update with tracker

10
Goals

• Complete internet publishing solution using
  Bittorrent
• Metadata file generator (.torrent)?
• Tracker
• Everseed
• Web interface

11
.torrent File

• Official docs on bittorrent.org
• Metadata on the file to be downloaded (tracker
  URL, filename, size, checksum hashes)?
• Stored as bencoded strings, integers, lists,
  dictionaries

12
Bencoding

• Integer 6 i6e
• String hello 5hello
• List hello,world l5hello5worlde
• Dictionary helloworld
  d5hello5worlde

13
Bencoding implementation

• Python has good string manipulation
• Structure of a .torrent file is a dictionary
  containing string keys and integer, string, list,
  and dictionary values
• Recursion to encode/decode

14
Tracker

• Makes use of the bencoding algorithm
• Handles two types of requests announce and
  scrape
• Stores data on peers and torrents in a SQLite
  database
• No performance issues

15
Network performance
Peer List Size
16
Database performance
17
Announce request

• Announces client's presence to tracker
• Used to get lists of IP addresses and BT ports of
  other clients in the swarm

18
(No Transcript)
19
Announce request

• Compact peer list response
• Recognition of seeding status

20
Scrape request

• Scrapes data from the tracker
• Used to get info on the different torrents that
  the tracker is tracking
• peers, seeds, total downloaded, total
  uploaded, completed etc...

21
Scrape request

• Client sends an HTTP GET request to tracker's
  scrape URL
• Tracker urldecodes request, selects the data the
  client is interested in
• Tracker responds with a bencoded text/plain
  document

22
Summary

• Python
• Benefits of P2P technology
• Everseed concept
• .torrent files and bencoding
• Tracker