OpenDHT: A Public DHT Service

1
OpenDHT A Public DHT Service

• Sean C. Rhea
• UC Berkeley
• June 2, 2005

Joint work with Brighten Godfrey, Brad Karp,
John Kubiatowicz, Sylvia Ratnasamy, Scott
Shenker, Ion Stoica, and Harlan Yu
2
Peer-to-Peer File Sharing

• Very simple insight
• Most computers unused most of the time
• Idea harness this spare capacity to
• Quickly download music files Napster, Gnutella
• Search for aliens SETI_at_Home
• Make free long-distance phone calls Skype
• Question how to find desired resource(s)?
• Early approaches scoped flooding
• Downsides scalability, accuracy

3
A Better Search FacilityThe Distributed Hash
Table (DHT)

• Same interface as a programmatic hash table,
• put(key, value) stores value under key
• get(key) returns the value(s) stored under key
• But shared across many machines
• Implemented via an overlay network

4
A Better Search FacilityThe Distributed Hash
Table (DHT)
stores k1,v1
put(k1,v1)
get(k1)
5
DHTs and File SharingDHT Stores Pointers to
Files
6
DHTs and File SharingDHT Stores Pointers to
Files
pointer to file
7
DHTs and Spam DetectionDetecting Similar
Messages
8
DHTs and Spam DetectionDetecting Similar
Messages
I love you!
I love you!
9
DHTs and Spam DetectionDetecting Similar
Messages
I love you!
I love you!
10
DHTs and Spam DetectionDetecting Similar
Messages
I love you!
I love you!
I love you!
11
DHTs and Spam DetectionDetecting Similar
Messages
I love you!
I love you!
I love you!
12
More DHT Applications

• Distributed Storage Systems
• CFS, HiveCache, PAST, Pastiche
• OceanStore / Pond
• Content Distribution Networks / Web Caches
• Bslash, Coral, Squirrel
• Indexing / Naming Systems
• Chord-DNS, CoDoNS, DOA, SFR
• Internet Query Processors
• Catalogs, PIER
• Communication Systems
• Bayeux, i3, MCAN, SplitStream

13
Some Areas of DHT Research

• Better routing protocols
• One-hop, degree-optimal
• Load balancing
• Non-uniform key distributions
• Security
• Byzantine fault-tolerant routing
• Data redundancy and fault tolerance
• Replication, erasure-coding
• Stronger semantics
• Supporting read-modify-write

14
How Many DHTs Will There Be?
File Sharing
Company Machine Cant Share Files
Owns Stock in Spam Company
15
How Many DHTs Will There Be?
File Sharing
Redundant Link
16
How Many DHTs Will There Be?
File Sharing
Unshared Links
17
Benefits of Sharing a DHT

• Amortizes costs across applications
• Maintenance bandwidth, connection state, etc.
• Facilitates bootstrapping of new applications
• Working infrastructure already in place
• Allows for statistical multiplexing of resources
• Takes advantage of spare storage and bandwidth
• Facilitates upgrading existing applications
• Share DHT between application versions

18
Challenges in Sharing a DHT

• Robustness
• Must be available 24/7
• Shared Interface Design
• Should be general, yet easy to use
• Resource Allocation
• Must protect against malicious/over-eager users
• Economics
• What incentives are there to provide resources?

19
Challenges in Sharing a DHT

• Robustness
• Must be available 24/7
• Shared Interface Design
• Should be general, yet easy to use
• Resource Allocation
• Must protect against malicious/over-eager users
• Economics
• What incentives are there to provide resources?

20
The DHT as a Service
21
The DHT as a Service
OpenDHT
22
The DHT as a Service
OpenDHT Clients
23
The DHT as a Service
OpenDHT
24
The DHT as a Service
What is this interface?
OpenDHT
25
The Traditional Interface lookup
26
The Traditional Interface lookup
lookup(k)
On reaching the successor of k, message passed to
an upcall
27
DHTs and Spam DetectionDetecting Similar
Messages
Upcall Ive seen this message before!
I love you!
I love you!
28
DHTs and Spam DetectionDetecting Similar
Messages
I love you!
I love you!
29
Upcall Challenges

• Distribution
• How do we get new upcall code to all nodes?

30
Upcall Challenges
31
Upcall Challenges

• Distribution
• How do we get new upcall code to all nodes?
• Active networking experience is a warning

32
Upcall Challenges

• Distribution
• How do we get new upcall code to all nodes?
• Active networking experience is a warning
• Security
• How do we safely run untrusted clients upcalls?

33
What about Put/Get?

• Works great for some applications
• File sharing, for example

34
DHTs and File SharingDHT Stores Pointers to
Files
35
What about Put/Get?

• Works great for some applications
• File sharing, for example
• What about applications with upcalls?
• Our spam detection application, for example

36
What about Put/Get?

• Works great for some applications
• File sharing, for example
• What about applications with upcalls?
• Our spam detection application, for example
• Idea let application nodes run the upcalls
• Each node only runs upcalls for the applications
  that its participating in

37
Upcall Example
File Sharing
put/get
OpenDHT
put/get
38
Upcall Example
File Sharing
Spam Detection
put/get
OpenDHT
Whos handling hash(message)?
put/get
I love you!
39
Upcall Example
File Sharing
Spam Detection
put/get
OpenDHT
Whos handling hash(message)?
put/get
I love you!
I love you!
40
Upcall Example
File Sharing
Spam Detection
put/get
DHT keeps track of which nodes support which
upcalls via Recursive Distributed Rendezvous
(ReDiR)
OpenDHT
put/get
I love you!
I love you!
41
ReDiR

• Goal Implement two functions using put/get
• join(namespace, node)
• node lookup(namespace, identifier)

L0
L1
L2
42
ReDiR

• Goal Implement two functions using put/get
• join(namespace, node)
• node lookup(namespace, identifier)

A
L0
A
L1
A, B
C
L2
43
ReDiR

• Goal Implement two functions using put/get
• join(namespace, node)
• node lookup(namespace, identifier)

A
L0
A, C
D
L1
A, B
C
D
L2
44
ReDiR

• Goal Implement two functions using put/get
• join(namespace, node)
• node lookup(namespace, identifier)

A, D
L0
A, C
D
L1
A, B
C
D
E
L2
45
ReDiR

• Goal Implement two functions using put/get
• join(namespace, node)
• node lookup(namespace, identifier)

A, D
L0
A, C
D, E
L1
A, B
C
D
E
L2
46
ReDiR

• Join cost
• Worst case O(log n) puts and gets
• Average case O(1) puts and gets

A, D
L0
A, C
D, E
L1
A, B
C
D
E
L2
47
ReDiR

• Goal Implement two functions using put/get
• join(namespace, node)
• node lookup(namespace, identifier)

A, D
L0
A, C
D, E
L1
successor
A, B
C
D
E
L2
H(A)
H(B)
H(C)
H(D)
H(E)
48
ReDiR

• Goal Implement two functions using put/get
• join(namespace, node)
• node lookup(namespace, identifier)

A, D
L0
successor
A, C
D, E
L1
no successor
A, B
C
D
E
L2
H(A)
H(B)
H(C)
H(D)
H(E)
49
ReDiR

• Goal Implement two functions using put/get
• join(namespace, node)
• node lookup(namespace, identifier)

successor
A, D
L0
no successor
A, C
D, E
L1
no successor
A, B
C
D
E
L2
H(A)
H(B)
H(C)
H(D)
H(E)
50
ReDiR

• Lookup cost
• Worst case O(log n) gets
• Average case O(1) gets

A, D
L0
A, C
D, E
L1
A, B
C
D
E
L2
H(A)
H(B)
H(C)
H(D)
H(E)
51
ReDiR Performance(On PlanetLab)
52
OpenDHT Design Summary

• OpenDHT is a common infrastructure for
• Storage of values, pointers, etc.
• Organizing clients that handle application
  upcalls
• Benefits
• Amortizes maintenance costs across applications
• Facilitates bootstrapping of new applications
• Allows for statistical multiplexing of resources

53
Impact
54
Future Work

• OpenDHT makes a great common substrate for
• Soft-state storage
• Naming and rendezvous
• Many P2P applications also need to
• Traverse NATs
• Redirect packets within the infrastructure (as in
  i3)
• Refresh puts while intermittently connected
• All of these can be implemented with upcalls
• Who provides the machines that run the upcalls?

55
Future Work

• We dont want to add upcalls to the core DHT
• Keep the main service simple, fast, and robust
• Can we build a separate upcall service?
• Some other set of machines organized with ReDiR
• Security can only accept incoming connections,
  cant write to local storage, etc.
• This should be enough to implement
• NAT traversal, reput service
• Some (most?) packet redirection
• What about more expressive security policies?

56
For more information, seehttp//opendht.org/