In to

1

• In to
• Peer To Peer

2
Overlay Network

• A virtual network
• Has neighbor concepts
• Has routing between nodes
• Send messages over a physical (TCP/IP?) network
• application level routing
• Usually not identified as routers in the network

3
Tapestry

• An Infrastructure for Fault-tolerant Wide-area
  Location and Routing
• Ben Y. Zhao, John Kubiatowicz and Anthony D.
  Joseph
• UCB Tech. Report UCB/CSD-01-1141

4
Requirements

• Stability through statistics
• Redundancy
• Multiple messages via different routes
• Redundant links
• Spread information (caching and replication)
• Combined Location and Routing
• Local independent names
• Location and Routing (distributed) primitives

5
Fault Tolerance, Repair and Self Organization

• Repairable soft-state
• Re-transmitions
• Verify routes
• Malicious servers (cryptographic methods)
• Operate in a continuous change

6
Background The Plaxton Mesh

• Accessing Nearby Copies of Replicated Objects in
  a Distributed Environment
• C. Greg Plaxton, Rajmohan Rajaraman and Andrea W.
  Richa
• ACM Symposium on Parallel Algorithms and
  Architectures

7
Model

• A distributed data structure optimized to support
  a network overlay for locating named objects and
  routing of messages to those objects.
• Forwarding overlay messages is routing
• Overlay nodes are routers
• The plaxton data structure is The plaxton mesh

8
Model (2)

• Each node can take the role of
• Servers where objects are stored
• Routers which forward messages
• Clients Which is the origin of a request
• Objects and nodes have independent names of their
  location and semantics
• 40 Hex digit names (160 bits)
• Names are computed using hashing algorithms

9
Model (3)

• Combined location and routing
• Servers publish / advertise objects they
  maintain
• Messages route to nearest server given object ID
• Assume global network knowledge

10
Routing

• Example Octal digits, 218 namespace, 005712 ?
  627510

11
Routing (2)

• Neighbor map
• Multiple levels
• level i matching suffix entries for i digits
• Number of entries per level the ID base
• At each entry, the closest suffix matching node
• Size b log(n)

12
Routing (3)

• Route to node (xyz)
• Shared suffix n
• Look at level n1
• Match the next digit in destination
• Send message
• Assume that the destination node is a tree root

13
Location

• Allows a client to locate and send messages to a
  named object residing on a server in a Plaxton
  mesh.
• A server S publishes that it has an object O by
  routing a message to the root node of O
• The publishing process consists of sending a
  message toward the root node. At each hop along
  the way, the publish message stores location
  information in the form of a mapping
  ltObject-ID(O), Server-ID(S)gt

14
Publish / Location

• Each object has associated root node, e.g.
  identity f( )
• Root keeps a pointer to objects location
• Object O stored at server S
• S routes to Root(O)
• Each hop keeps ltO,Sgtin index database
• Client routes to Root(O), route to S when ltO,Sgt
  found

15
Benefits and Limitations

• Benefits
• Simple fault-handling
• Scalable state bLogb(N), hops Logb(N)bdigit
  base, N namespace
• Exploits locality
• Proportional route distance
• Limitations
• Global knowledge algorithms
• Root node vulnerability
• Lack of adaptability

16
What is Tapestry?

• A prototype of a decentralized, scalable,
  fault-tolerant, adaptive location and routing
  infrastructure(Zhao, Kubiatowicz, Joseph et al.
  U.C. Berkeley)
• Network layer of OceanStore
• Routing Suffix-based hypercube
• Similar to Plaxton, Rajamaran, Richa (SPAA97)
• Decentralized location
• Virtual hierarchy per object with cached location
  references
• Core API
• publishObject(ObjectID, serverID)
• routeMsgToObject(ObjectID)
• routeMsgToNode(NodeID)

17
Routing and Location

• Namespace (nodes and objects)
• 160 bits ? 280 names before name collision
• Each object has its own hierarchy rooted at Root
• f (ObjectID) RootID, via a dynamic mapping
  function
• Suffix routing from A to B
• At hth hop, arrive at nearest node hop(h) s.t.
• hop(h) shares suffix with B of length h digits
• Example 5324 routes to 0629 via5324 ? 2349 ?
  1429 ? 7629 ? 0629
• Object location
• Root responsible for storing objects location
• Publish / search both route incrementally to root

18
Routing to Object Plaxton

• Given desired ID Ni,
• Find set S of nodes in existing network nodes n
  matching most of suffix digits with Ni
• Choose Si node in S with highest valued ID
• Issues
• Mapping must be generated statically using global
  knowledge
• Must be kept as hard state in order to operate in
  changing environment
• Mapping is not well distributed, many nodes in n
  get no mappings

19
Routing to Object Tapestry

• Globally consistent distributed algorithm
• Attempt to route to desired ID Ni
• Whenever null entry encountered, choose next
  higher non-null pointer entry
• If current node S is only non-null pointer in
  rest of route map, terminate route, f(Ni) S
• Assumes
• Routing maps across network are up to date
• Null/non-null properties identical at all nodes
  sharing same suffix

20
Tapestry MeshIncremental suffix-based routing
21
More on Tapestry

• Return location of all replicas
• Allow application semantic on how to choose a
  replica
• Hot Spot detector
• Backpointers

22
Fault-tolerant Routing

• Strategy
• Detect failures via soft-state probe packets
• Route around problematic hop via backup pointers
• Handling
• 3 forward pointers per outgoing route (2
  backups)
• 2nd chance algorithm for intermittent failures
• Upgrade backup pointers and replace

23
Fault-tolerant Location

• Minimized soft-state vs. explicit fault-recovery
• Redundant roots
• Object names hashed w/ small salts ? multiple
  names/roots
• Queries and publishing utilize all roots in
  parallel
• P(finding reference w/ partition) 1
  (1/2)nwhere n of roots
• Soft-state periodic republish
• 50 million files/node, daily republish, b 16,
  N 2160 , 40B/msg, worst case update traffic
  156 kb/s,
• expected traffic w/ 240 real nodes 39 kb/s

24
Dynamic Insertion

• Operations necessary for N to become fully
  integrated
• Step 1 Build up Ns routing maps
• Send messages to each hop along path from gateway
  to current node N that best approximates N
• The ith hop along the path sends its ith level
  route table to N
• N optimizes those tables where necessary
• Step 2 Move appropriate data from N to N
• Step 3 Use back pointers from N to find nodes
  which have null entries for Ns ID, tell them to
  add new entry to N
• Step 4 Notify local neighbors to modify paths to
  route through N where appropriate

25
Dynamic Insertion Example
4