Compact Routing and Locality in Peer-to-Peer Systems

1
Compact Routing and Locality in Peer-to-Peer
Systems

• Ittai Abraham
• School of Computer Science and Engineering
• Hebrew University of Jerusalem

2
Internet Activity

• After 1992 Dominated by the web browser
• Client Server paradigm
• Clients are Lightweight and Transient
• Low bandwidth, computational power and storage
  requirements
• Most servers are relatively simple and static
  (HTTP)
• Distributed Computing community mostly focused on
  robustifying servers for load balancing and high
  availability, targeting mostly at clusters of
  dozens of servers at the most
• How will Internet Activity look like in the
  future ?

3
Tomorrows Internet ?

• Internet peers will be stateful and will have a
  persistent connection
• Have high bandwidth, computational power and
  storage capabilities
• Peers are capable of acting both as a server and
  as a client
• Will have an active network presence
• Symmetric, decentralized, self organizing paradigm

4
Peer-to-peer Systems

• Group of peers wish to maintain a shared
  information data structure
• Complications
• Large group
• Enormous amounts of information
• Group is spatially distributed
• Dynamically changing
• Heterogeneous
• Selfish/faulty/malicious participants
• Challenge provide efficient access to the shared
  information data structure

5
Distributed Hash Tables

• A universe U of object ids. A hash function h
  maps U into a smaller set S spreading ids without
  many collisions
• A set V of nodes that forms a distributed system
• The set S is partitioned in to V parts, and
  each node in V maintains its relevant part of the
  hash table
• Basic operations are lookup and store Given an
  object id A, find the hash value h(A), route to
  the node that maintains the key h(A), and
  read/write the object A

6
DHT Lookup Example

• Hash function h(x) x mod 1009
• For this example each node with id i maintains
  all the objects whose has hash value is in
  1009i/6,1009(i1)/6)

0
5
6
Hash value is 1, so need to route to node with id
0
3
2
1
4
Node 2 wants to find the value of the object with
id 1009001
7
Traditional Complexity Measures of Distributed
Hash Tables

• Degree (local memory) of the overlay network
• O(1), O(logk n), O(n1/k)
• Number of hops from source node to target node
• O(log n), O(log n/loglog n), O(1)
• But counting hops does not take into account a
  weighted communication network

8
Locality Awareness
x
t
s
9
Low Stretch Routing

• Two models
• Peers communicate through a weighted network
  GltV,E,?gt
• The cost of communication d(s,t) between peers in
  V induces a metric space MltV,dgtI
• General metric
• Growth bounded metric, Euclidean metric, Doubling
  metric
• In either case the stretch of a routing scheme RS
  is the maximal ratio over all pairs of
  dRS(s,t)/d(s,t)

10
Routing on a weighted graph

• Devise a distributed routing scheme such that a
  node that knows the label of a target node can
  send a message that will be routed to the target
  node
• Main complexity measures
• Stretch the ratio between the cost of the path
  taken by the routing protocol and the cost of a
  minimum cost path from source to destination.
• Memory the number of bits stored in each node.
• A solution is compact if memory is o(n)

11
Routing on a weighted graph

• Lower bounds
• Stretch lt 3 requires O(n) bits per node Gavoille
  Gengler 01
• Stretch lt 5 requires O (vn) bits per node Thorup
  Zwick 01
• Stretch lt 2k-1 requires O (n1/k) bits per node
  Thorup Zwick 01 Under the Erdosh conjecture
• Two main variants
• Labeled routing designer can choose the labels
  of nodes
• Name Independent routing node labels are given
  by an adversary
• Labeled routing
• Stretch 3 with Õ(n2/3) bits Cowen 99
• Stretch 3 with Õ(vn) bits Thorup Zwick 01
• Stretch 4k-1 with Õ(n1/k) bits Thorup Zwick 01

12
Name Independent Routing

• Awerbuch, Bar-Noy, Linial Peleg 89
• With Õ(n1/k) bits stretch O(k29k)
• With Õ(n2/3) bits stretch 468
• With Õ(vn) bits stretch 2593
• Awerbuch Peleg 90 (Sparse Partitions)
• For diameters that are polynomial in n
• With Õ(n1/k) bits stretch O(k2)
• With Õ(n2/3) bits stretch 624
• With Õ(vn) bits stretch 1088
• Arias, Cowen, Laing, Rajaraman Taka 03
• With Õ(vn) bits stretch 5

A, Gavoille, Malkhi, DISC 04, Stretch O(k)
A, Gavoille, Malkhi, Nisam, Thorup, SPAA 04,
Stretch 3
13
Compact Name-Independent Routing with Minimum
StretchA, Gavoille, Malkhi, Nisan, and Thorup
SPAA 2004

• Optimal stretch 3 with Õ(vn) bits
• Construction in polynomial time
• Routing decisions performed in constant time
• Surprisingly, with Õ(vn) bits allowing the
  designer to label the nodes does not improve the
  stretch factor compared to the task when node
  labels are predetermined by an adversary.

14
The Recipe

• Ingredients
• Vicinity routing
• Random coloring to vn colors
• Hash labels to colors
• Labeled routing on trees
• Landmarks
• Partial shortest path trees

15
Vicinity Routing

• Let B(u) denote the (vn log n)-closest nodes to u
  (ties broken consistently)
• For all v?B(u), node u stores the next hop of a
  minimum cost path from u to v
• Simple property ABLP 89 If v?B(u) and w is on
  a minimum cost path from u to v, then v?B(w)

u
B(u)
16
Random Coloring to vn Colors

• Every node u chooses a random color c(u)
• With high probability
• Every color set has O(vn) nodes
• Every node has in its vicinity at least one node
  from every color set
• Polynomial number of tests
• Each test can be done in logspace
• Derandomization using the pseudo random
  generator of Nisan

17
Hash Labels to Colors

• Label u is hashed to a color h(u)?1 vn
• At most O(vn log n) hashed to same color
• Trivial if node labels are a permutation of 1n
• Otherwise can collision free hash to n2.5 and
  then use the techniques of Tarjan and Yao to hash
  to vn in constant time. Can be deradomized
  similarly to deterministic dictionaries of
  Hagerup, Milerstein, Pagh

18
Labeled Routing on Trees

• Based on DFS Interval Routing, improved by Thorup
  Zwick 01 and Frainiaud Gavoille 01
• A node is heavy if its sub-tree contains more
  than half of the nodes of its parents sub-tree
• Each node stores its DFS interval and the DFS
  interval of its heavy child (if it has one)
• Storage is O(log n) bits
• A nodes label consists of the names of the
  non-heavy nodes on the path from the root
• Labels require O(log2 n) bits
• Routing to v on node u
• If v is not in us interval then send to parent
• If v is in us heavy child interval then send to
  heavy
• Otherwise us label contains the appropriate
  child

19
Landmarks

• Let R(T,v) be the routing information stored at
  node v for routing on tree T
• Let T(u) be the minimum cost tree rooted at u
• One color is designated as special
• Let L be the set of all nodes l such that
  c(l)special color
• Every node u maintains R(T(l),u) for all l?L
• This requires Õ(vn) bits
• For node u, let l(u) be a landmark in B(u)

20
Partial Shortest Path Trees

• Every node v stores R(T(u),v) for all u?B(v)
• Requires Õ(vn) bits
• Let L(T,u) be the label of u on tree T
• Simple property If x?B(y) then given L(T(x),y),
  node x can route to node y along a minimum cost
  path

x
w
y
B(y)
21
Case 1 Inside B(u)

• Use vicinity routing

u
v
22
Case 2 B(u) and B(v) are close

• Any node on any minimal path from u to v is
  either in B(u) or in B(v)

• Vicinity route to w?B(u) s.t. c(w)h(v)
• Node w stores L(T(w),u),x,(x?y),L(T(y),v)
• Partial tree route to u on T(w)
• Vicinity routing to y
• Partial tree route to v on T(y)

u
y
x
v
23
Case 3 B(u) and B(v) are far

• Any minimal path from u to v contains a node that
  is not in B(u) or in B(v)

• Vicinity route to w?B(u) s.t. c(w)h(v)
• Node w stores L(T(l(v)),l(v)) and L(T(l(v)),v)
• Tree route on T(l(v)) to l(v) and then to v

u
w
v
24
Storage on node u

• Routing information and colors in B(u) Vicinity
• R(T(l),u) for all l?L Landmarks
• R(T(v),u) for all v?B(u) Partial Trees
• For all v such that c(u)h(v) minimum of
• Path to l(v) and then to v. Store ltL(T(l(v)),
  l(v)) , L(T(l(v)), v)gt
• Let P(u,w,v) be a path from u to v composed of an
  MCP from u to w, and of an MCP from w to v, such
  that
• u \in B(w),
• there exists an edge (x ? y) along the minimum
  path from w to v such that x ? B(w) and y ? B(v)
• Among all the these paths choose the lowest cost
  path P(u,w,v) and store ltL(T(u), w), x, (x ? y),
  L(T(y),v)gt

25
Routing from u to v

• If v?B(u) use vicinity routing
• If v?L use tree routing on T(v)
• Otherwise vicinity route to w?B(u) such that
  c(w)h(v)
• Node w stores either
• ltR(T(l(v)), l(v)) , R(T(l(v)), v)gt, and routing
  proceeds to l(v) and then to v
• ltR(T(u), w), x, (x ? y), R(T(y),v)gt, and routing
  proceeds to w then x to y and finally to v

26
Better than stretch 3 ?

• There are worst case metrics in which stretch 3
  is the best possible (many edges and high girth)
• But do these metrics depict real world distances
  ?
• Studies show that many networks have a bounded
  expansion ratio. Density changes are somewhat
  gradual
• Plaxton, Rajaraman Rica 1997
• Required
• Expected (large) constant stretch
• Deployments Tapestry (Berkeley), Pastry (MS UK)