NoTorrent A PeertoPeer Web Caching System Modeled Partially After BitTorrent but Without the Need fo

1
NoTorrentA Peer-to-Peer Web Caching System
Modeled Partially After BitTorrent but Without
the Need for .torrent Files

• Howie Vegter, hrv2101
• Columbia University
• Fall 2005
• CS 6901, Projects in Computer Science
• Advisors Dr. Markus Hofmann, Dr. Henning
  Schulzrinne, Salman Abdul Baset

2
NoTorrent Architecture Overview

• When the client requests a resource through the
  client proxy, we try to obtain the resource in
  this order
• First try to get resource from Origin Server
• Then try to get it from clients local cache
• Then try to get it from a peer

NoTorrent Tracker
Origin Server
1
3.1
Peer
Client
NoTorrent Client
NoTorrent Client
Browser
Browser
Proxy
Proxy
2
Cache
Cache
3.2
Server
Server
Note A client and a peer both refer to a
computer that is running the NoTorrent Client. I
will say client when I mean my computer and
peer when I am referring to the other NoTorrent
Clients I am interacting with.
3
Requesting a Resource First, Try the Origin
Server
NoTorrent Tracker
Origin Server
XML-encoded IHaveResourceMessage

• If origin server serves resource,
• Cache it locally
• Inform tracker client has the resource
• Send the resource to the browser
• If origin server fails to serve resource
• Try checking local cache

1
2.1
Client
NoTorrent Client
Browser
Proxy
3
2.2
Cache
Server
4
Requesting a Resource Next, Try Local Cache
Client
NoTorrent Client

• Client could not get resource from origin server.
• Client may have obtained resource earlier (from
  peer or origin server before it went down), so
  check local cache.
• If resource in cache,
• Get resource from cache
• Send resource to browser
• If resource not in cache,
• Ask peers for the resource

Browser
Proxy
Cache
Server
5
Requesting a Resource Next, Ask Peers (slide 1
of 3)
Client could not get resource from origin server
or cache, so we ask peers. 1. Ask tracker for
list of peers with resource 2. Tracker responds
with list 3. Ask a peer for the resource 4. Peer
checks its cache 5. (see next slide)
NoTorrent Tracker
XML-encoded PeerListRequestMessage
XML-encoded PeerListRequestMessage
Peer
Client
NoTorrent Client
NoTorrent Client
Browser
Browser
Proxy
Proxy
Cache
Cache
XML-encoded ResourceRequestMessage
Server
Server
6
Requesting a Resource Ask Peers (slide 2 of 3)

• If peer has resource
• Peer sends resource to client
• Client proxy caches resource and informs tracker
  is has the resource
• Client proxy sends resource to browser

NoTorrent Tracker
XML-encoded IHaveResourceMessage
Peer
Client
NoTorrent Client
NoTorrent Client
Browser
byte resource
Browser
Proxy
Proxy
Cache
Cache
Server
Server
7
Requesting a Resource Ask Peers (slide 3 of 3)

• If peer does not have resource
• Peer terminates connection
• Client informs tracker that peer failed to serve
• Client requests resource from another peer
• If no peers had resource, send error message to
  browser

NoTorrent Tracker
XML-encoded Peer 1 Failed To Serve
Client
X
NoTorrent Client
error
Browser
Proxy
Cache
Server
8
Messages are Encoded in XML

• MessagePeerListRequest
• lt?xml version1.0 encodingUTF-8?gt
• ltmessagegt
• ltmessageTypegtPeerListRequestlt/messageTypegt
• ltmessageContentsgt ltrequestedResourcegthttp
  //lt/requestedResourcegt
• lt/messageContentsgt
• lt/messagegt
• NoTorrent uses JDOM to relatively easily parse
  XML.

9
Setting Up NoTorrent

• Unpack notorrent.tgz to get NoTorrent.jar,
  README, nttrack, and ntclient (the latter two are
  helper scripts)
• On tracker machine
• Start tracker to listen for and handle messages
  from clients
• java cp NoTorrent.jar notorrent.tracker.Tracker
• or
• ./nttrack
• On client machine
• Start client proxy to handle requests from
  browser
• java jar NoTorrent.jar ta TRACKER_ADDRESS tp
  TRACKER_PORT
• or
• ./ntclient
• Configure browser to use NoTorrent client proxy
  as its HTTP proxy

10
Lessons Learned and Encountered Roadblocks

• When basing a project on another project, make
  sure the base project
• is well-documented
• is really as close to the problem you are trying
  to solve as you thought.
• Standards (e.g. XML) and good libraries (e.g.
  JDOM) can simplify certain parts of the project.
• Specific Things I Learned About
• Message encoding (XML, BEncoding). I had not
  anticipated that encoding would be a major part
  of system.
• TCP/IP sockets in Java, thread pools
• Python (although I ended up not using it)
• Roadblocks
• HTTP 403 (Forbidden) Errors on certain pages on
  Google, NYTimes, Yahoo, etc.
• Yahoo search results include tracking info in the
  URLs and then redirect the user to the
  destination. This currently works without the
  NoTorrent proxy, but it only works partially with
  the NoTorrent proxy. Sample Yahoo search result
  URL (for columbia.edu)http//rds.yahoo.com/_ylt
  Ai_xP1B0Ov3OWjt6ut6lMn1XNyoA_yluX3oDMTE2c3E4b3F2
  BGNvbG8DdwRsA1dTMQRwb3MDMgRzZWMDc3IEdnRpZANGNjY2Xz
  g1/SIG11cn6o65u/EXP1135100947/http3a//www.col
  umbia.edu/