Epidemic Algorithms for Replicated Database Management

1
Epidemic Algorithms for Replicated Database
Management

• A. Demers, D. Greene, C. Hauser, W. Irish, J.
  Larson, S. Shenker, H. Sturgis, D. Swinehart, and
  D. Terry
• Presented by Cole Trapnell

2
Problem

• Replicate a database at many sites, spread all
  over the globe
• Be able to distribute frequent updates
• Maintain consistency - make sure everyone gets
  all updates
• Dont require gaurantees from the underlying
  layers

3
Their solution

• Spread updates like we spread disease!

Ebola virus (false color) Scott Camazine/Photo
Researchers, Inc
4
Step back - Direct Mail protocol

• When a machine generates an update, it mails it
  out to all other machines
• Expensive
• Works

5
Step back - Direct Mail protocol

• When a machine generates an update, it mails it
  out to all other machines
• Expensive
• Works
• Most of the time

6
Anti-entropy

• Every node chooses another at random, and
  exchanges database contents with it
• During the exchange, each node resolves the
  differences

7
Anti-entropy

• PUSH (s, s)
• if s.time gt s.time
• s.value ? s.value

• PULL (s, s)
• if s.time lt s.time
• s.value ? s.value

8
Anti-entropy

• PUSH (s, s)
• if s.time gt s.time
• s.value ? s.value

• PULL (s, s)
• if s.time lt s.time
• s.value ? s.value

PUSH-PULL (s, s) if s.time gt
s.time s.value ? s.value else if s.time lt
s.time s.value ? s.value
9
Anti-entropy

• Is a simple epidemic protocol
• Eventually, all nodes will get all updates
• How Eventually?

What if most nodes already have the update?
10
Anti-entropy

• Let pi be the probability node p has not received
  the update after the ith anti-entropy cycle

For pull
For push
11
Rumor mongering

• Complex epidemic protocol
• 3 kinds of nodes
• Susceptible
• Infected
• Removed

12
Rumor mongering

• Receive a rumor at a node (update injected)
• That node sends the rumor to another node picked
  at random.
• If that other node has not heard rumor, goto 2.
• Else, with probability 1/k, that node stops
  calling.

13
Rumor mongering - results
Push, counters, response, 1000 sites
14
Variations

• Push vs. Pull
• Blind vs. Response
• Counters vs. Coin
• Connection limits

15
How to delete keys?
16
How to delete keys

• Death certificates
• When to delete the death certificates?
• Dormant death certificates
• Allows the network to reduce the stable storage
  needed for the certs.

17
Spatial preference

• Might want to select nodes for rumor exchange
  with a local bias

18
Conclusions

• We can replace complex deterministic algs with
  simpler, randomized ones.
• But we have to live with the possibility of an
  incomplete update.
• But we can make that update arbitrarily small.

19
Questions?