Samsara: Honor Among Thieves in PeertoPeer Storage

1
Samsara Honor Among Thieves in Peer-to-Peer
Storage
2
Introduction

• Peer-to-Peer Paradigm
• A node stores some data in remote nodes
• Agree to do the same in return
• Replication ? fault-tolerance
• Decentralized
• Self-administering
• Scalable

3
Problem with the P2P Model

• The tragedy of the commons
• Consume without contributing
• Some existing solutions
• Third parties ? centralized administration
• Currency ? trusted infrastructure
• Certified evidence of storage consumption ?
  centralized authority

4
Observation

• Problem is simplified if we have
• Symmetric exchange of resources

5
Observation

• Problem is simplified if we have
• Symmetric exchange of resources
• Guarantees consumption lt contribution
• However, symmetric relationships are rare
• Replica A needs 1 GB replica B needs 1 MB

6
Samsara

• An infrastructure to enforce fairness in
  peer-to-peer systems
• No third trusted parties
• No monetary models
• No certified identities

7
Another Observation

• Symmetric storage relationships can be
  manufactured
• A claim-based system

8
Another Observation

• Symmetric storage relationships can be
  manufactured
• A claim-based system
• Based on incompressible storage claims

9
Storage Claims

• A node periodically checks its peer
• Make sure that the peer is adhering to the
  contract
• If the peer breaches the contract
• The node is free to drop the peers data
• Each node now can perform selfishly
• Collectively, all nodes need to play fair

10
Some Questions

• How to reduce the storage overhead
• How to punish cheaters
• How to tell failures from cheating

11
Background

• Pastiche
• Peer-to-peer, cooperative backup system
• Unsolved problem unchecked storage consumption

12
Design

• Goal Ensure that nodes consume no more
  resources than they contribute
• Manufacture symmetric storage- exchange
  relationships
• Through storage claims
• A claim can be passed along to form a dependency
  chain
• A claim can be removed if it forms a circular
  dependency

13
Design

• Punish cheaters by deleting their data
  probabilistically
• Short outage can recover from surviving copies
• Cheaters will eventually lose data

14
Claim Construction

• Requires three values
• A secret pass phrase
• A private, symmetric key
• A location in the storage space

15
Claim Construction

• Storage space initially filled with hash values

16
Querying Nodes

• Queries
• Monitor remote storage
• Once every few hours
• Need not be answered immediately

17
Querying Nodes

• Query sends h0 to verify data 1..n
• Remote site computes
• h1 SHA(data1, h0)
• h2 SHA(data2, h1)
• hn SHA(datan, hn-1)
• Remote site returns hn

18
Transient Failure

• Difficult to discern cheating from transient
  failures
• One solution
• Grace periods before deletion
• Problem revolving credits

19
Samsara Solution

• Replication independent probabilistic deletion
• Deletion rate is an exponential growing function
  of the number of failed queries
• A cheater (gt 32GB) cannot replicate fast enough
  to get a free ride
• Need to replicate 10 times in 3 days

20
Samsara Solution

• A node should only lose all of its data if it
  fails queries for an entire grace period
• Most outages are within 3 days

21
Probabilistic Discard Example
Failed queries
0
22
Overhead Reduction

• Storage claims can be forwarded

23
Overhead Reduction

• Storage claims can be forwarded

24
Overhead Reduction

• Storage claims can be forwarded
• However, if something goes wrong
• The forwarding replica is responsible
• Increase the incentive for not forwarding

25
Diffie-Hellman Key Exchange

• Need a prime number p
• Need a base integer g between 1 and p 1
• Site A picks x between 1 and p 2
• Site B picks y between 1 and p 2

• p 13
• g 7
• A 3
• B 5

26
Diffie-Hellman Key Exchange

• Site A computes
• gx mod p
• Site B computes
• gy mod p
• Site A and B exchange public values

• A 73 mod 13 5
• B 75 mod 13 11
• A 3, 11(from B)
• B 5, 5 (from A)

27
Diffie-Hellman Key Exchange

• Site A computes
• (gy mod p)x mod p
• Site B computes
• (gx mod p)y mod p
• Now A and B have a shared secret
• Problem Prone to man-in-the-middle attacks

• A 3, 11(from B)
• B 5, 5 (from A)
• A 113 mod 13 5
• B 55 mod 13 5

28
Forwarding and Reliability

• Longer forwarding chain ? lower reliability
• Cyclic chains are okay, because the
  accountability is wrapped around
• Unfortunately, cycles are rarely found

29
Limitations

• Cannot handle malicious nodes
• Cannot force nodes to store data for others
• Cannot create place holders for bandwidth and
  processing power

30
Implementation

• Written in C
• Three layers
• Messaging layer
• Replica manager
• Storage layer
• A single flat file
• Linked list of free space

31
File copy benchmark

• 13MB file copied between two nodes

32
Query benchmark

• 2 hours to verify 32GB claims _at_550MHz

33
Reliability simulations

• Examine chain length and reliability
• What percentage of files lost?
• Simulate the absolute worst case
• Limit chain length
• Transfer as much as possible w/i limit
• All failures occur
• Permanently
• Simultaneously
• Before new replicas can be created

34
Reliability results