Characterization of Byzantine Quorum Systems

1
(Characterization of)Byzantine Quorum Systems

• Dahlia Malkhi
• Michael Reiter

Presented by Dave Lillethun
Published June 1998
davel_at_cc.gatech.edu
2
Terminology

• U the universe of servers
• n U the number of servers
• Q a quorum of servers ? U
• Q a quorum system the set of all Q
• B a set of (potentially) faulty servers
• B a fail-prone system the set of all B

3
Assumptions

• Every two processes (clients or servers)
  communicate over a point-to-point channel.
• If both endpoints of a channel are correct (i.e.
  non-faulty) then the channel is both
  authenticated and reliable.
• Communication is asynchronous.

4
What is a quorum system?
A universe of servers U
S1
S6
S0
S5
S4
S8
S2
S7
S3
Another quorum Q
A quorum Q
5
What is a quorum system?
S0
Q
Q
S1
S5
S4
S2
S3
Q
All quorums must have at least one server in
common with all other quorums.
6
Writes

• Client queries a quorum Q for a timestamp from
  each server in Q
• Client chooses a timestamp t greater than any of
  those returned, and greater than its own
  timestamp
• Client writes v by sending ltv, tgt to servers
• Each server updates v iff t gt vs timestamp on
  the server
• The server returns an ack regardless of whether
  it updated v
• Once the client has received an ack from every
  server in some quorum Q, the write is complete
• Q does not have to be the same as Q

7
Reads

• Client queries every server in some quorum Q for
  value/timestamp pairs ltv,tgt
• Client applies some deterministic function
  Result(A) to the returned pairs
• Result(A) should return the correct value v or ?
  to indicate a read failure
• However, the details of Result(A) are
  intentionally left unspecified to allow different
  implementations of quorum systems

8
Correctness of Reads and Writes
S1
S2
S6
S5
S3
S4
A read from any of the quorums includes at least
one copy of the latest (green) write the
timestamp will tell you which it is!
9
Load on Quorum Systems

• Let w be an access strategy gt the probability
  distribution of accessing each Q in Q
• w(Q) gt the probability of accessing Q
• Let lw(u) be the load on server u ? U
• The load on a system from strategy w is
• Lw(Q) maxu?U (lw(u))
• i.e. the load on the most loaded server
• The system load (or just load) is
• L(Q) minw (Lw(Q))
• i.e. the load of the access strategy that gives
  the smallest load

10
Load on Quorum Systems
System Load L(Q) L2(Q)
11
Masking Quorum Systems
out of date
correct
arbitrary values
12
Masking Quorum System Definition

• A masking quorum system is a system that has the
  following two properties
• M-Consistency ?Q1,Q2 ? Q ?B1,B2 ? B
• (Q1 ? Q2) \ B1 ? B2
• The set of correct servers (w.r.t. any B) must
  not be a subset of any other B, or
• There must be at least one server in the set of
  correct servers that is not a member of any B ? B
• M-Availability ?B ? B ?Q ? Q
• B ? Q ?
• For every B ? B, there must be some quorum that
  does not have any servers in B

13
Masking Quorum Systems
out of date
correct
arbitrary values
14
Example Masking Quorum System

• Description
• Allow up to f Byzantine failures - this is the
  definition of B
• i.e. each set of f servers is a B ? B
• Every pair of quorums intersects with (2f 1)
  servers
• Performing Reads
• Client receives ltv, tgt pairs from all servers in
  Q2
• Eliminate any pairs that do not have (f 1)
  matching copies
• this ensures what remains are not Byzantine
  values
• Select the remaining pair with the latest
  timestamp
• this ensures we have the latest copy

15
Properties of Masking Quorum Systems

• Lemma 4.2 A read that is not concurrent with any
  writes returns the value written by the last
  write
• A read that is concurrent with one or more writes
  may return ?
• Theorem 4.3 There exists a masking quorum system
  for B iff Q U \ B B ? B is a masking quorum
  system for B
• Corollary 4.4 There exists a masking quorum
  system for B iff for all B1,B2,B3,B4 ? B, U ?
  B1?B2?B3?B4
• Supposing B allows up to f Byzantine failures,
  then there exists a masking quorum system for B
  iff n gt 4f

(proofs omitted)
16
Load on Masking Quorum Systems

• Theorem 4.5
• Let c(Q) denote the size of the smallest quorum Q
  ? Q

(proof in reference 31)
17
Example 1 f-masking

• Model
• Up to f servers my fail B
• n gt 4f
• Q
• M-Consistency
• Every pair of quorums intersects in at least 2f1
  servers
• M-Availability
• Load
• Assign equal probability to each quorum

18
Example 2 Grid Quorums
5 x 5 25 servers
Q 1 column i rows, where i 2f1
So if f 1, then choose 1 column and 3 rows
19
Example 2 Grid Quorums

• Model
• Up to f servers my fail B
• 3f1 ? ?n
• Q is defined as per the previous slide
• M-Consistency
• Every pair of quorums intersects in at least 2f1
  servers
• The column of one quorum intersects with the 2f1
  columns of the other
• M-Availability
• For any f faulty servers, 2f1 full rows and 1
  full column remain available in the system
• Load
• Assign equal probability to each quorum

20
Example 3 Partition Quorums

• Model
• B B1, , Bm, where m gt 4
• e.g. WAN clusters, where at most 1 cluster is
  faulty
• Let f 1
• Q Q?U Q is ? of subsets of i different B,
  where
• M-Consistency
• Every pair quorums intersects with elements from
  at least three different B
• M-Availability
• Since m gt 4, no Q contains elements from all B ?
  B
• Load
• Assign equal probability to each quorum

21
Dissemination Quorum Systems
out of date
correct
unverifiable and/or out of date values
22
Dissemination Quorum System Definition

• A dissemination quorum system is a system that
  has the following two properties
• D-Consistency ?Q1,Q2 ? Q ?B1 ? B
• (Q1 ? Q2) ? B1
• The set of correct servers (w.r.t. any B) must
  contain at least one server
• Contrast M-Consistency required more than B2
  in the correct set
• D-Availability ?B ? B ?Q ? Q
• B ? Q ?
• For every B ? B, there must be some quorum that
  does not have any servers in B

23
Reads in Dissemination Quorum Systems

• Recall that Byzantine failures come in two
  flavors
• Unverifiable values
• Verifiable but out of date values (same as
  non-faulty out of date values)
• Client receives ltv, tgt pairs from all servers in
  Q2
• Eliminate any pairs that are not verifiable
• This eliminates Byzantine failure flavor 1
• Select the remaining pair with the latest
  timestamp
• This eliminates Byzantine failure flavor 2, as
  well as any non-faulty out of date values

24
Properties of Dissemination Quorum Systems

• Lemma 5.2 A read that is not concurrent with any
  writes returns the value written by the last
  write
• Lemma 5.3 A concurrent read returns either the
  value written by the last write or any of the
  values being written concurrently
• This is a regular variable as opposed to masking
  quorum systems which provide multi-writer
  multi-reader safe variables
• Theorem 5.4 There exists a dissemination quorum
  system for B iff Q U \ B B ? B is a
  dissemination quorum system for B
• Analagous to Theorem 4.3 for Masking Quorums
• Corollary 4.4 There exists a dissemination
  quorum system for B iff for all B1,B2,B3 ? B, U ?
  B1?B2?B3
• Supposing B allows up to f Byzantine failures,
  then there exists a dissemination quorum system
  for B iff n gt 3f
• Contrast n gt 4f for masking quorum systems

(proofs omitted)
25
Examples of Dissemination Quroum Systems

• f-dissemination
• Allow up to f Byzantine failures n gt 3f
• contrast with f-masking
• Load
• Grid
• Allow up to f Byzantine failures 1f 1 ? ?n
• Q any 1 column and any (f 1) rows
• Load
• Partition
• m gt 3 (as opposed to m gt 4 for masking
  partitions)
• Load is the same equation as masking partitions,
  but note that in the best case m is allowed to be
  lower

26
Opaque Quorum Systems
out of date
correct
arbitrary values
27
Opaque Quorum System Definition

• An opaque quorum system is a system that has the
  following three properties
• O-Consistency1 ?Q1,Q2 ? Q ?B ? B
• (Q1 ? Q2) \ B ? (Q2 ? B) ? (Q2 \ Q1)
• The set of correct servers (w.r.t. any B) must be
  at least as large as the total of all faulty and
  out of date servers
• O-Consistency2 ?Q1,Q2 ? Q ?B ? B
• (Q1 ? Q2) \ B gt (Q2 ? B)
• The set of correct servers (w.r.t. any B) must be
  larger than the set of faulty servers
• O-Availability ?B ? B ?Q ? Q
• B ? Q ?
• For every B ? B, there must be some quorum that
  does not have any servers in B

28
Reads in Opaque Quorum Systems

• Client receives ltv, tgt pairs from all servers in
  Q2
• Choose the pair that appears most often
• i.e. voting system
• Break ties by choosing the pair with the latest
  timestamp

29
Opaque Quorum Systems
Q2
out of date
correct
arbitrary values
30
Properties of Opaque Quorum Systems

• Lemma 5.10 A read that is not concurrent with
  any writes returns the value written by the last
  write
• Theorem 5.12 Supposing B allows up to f
  Byzantine failures, then there exists an opaque
  quorum system for B iff n ? 5f
• Contrast n gt 4f for masking quorum systems
• Testing whether an opaque quorum system exists
  for a given B is an open problem!
• (aside from an exhaustive 2U serach, that is!)
• Theorem 5.14 The load of any opaque quorum
  system is at least ½ (this is a tight bound)
• Contrast There exist masking quorum systems
  whose load decreases as a function of n i.e.
  grid quorum systems

(proofs omitted)
31
Faulty Clients

• Might send different updates to different servers
• Might fail to contact a full quorum
• Must maintain availability
• Correct clients should be able to complete a
  write with as little as one quorum available

32
New Write Protocol

• Client chooses a timestamp t greater than any t
  it has chosen before
• Client sends an update message to each server in
  a quorum
• ltupdate, Q, v, tgt
• After a timeout period, if the client has not
  received an ack from every server in the quorum,
  it repeats step 2

33
New Server Update Protocol

• If a server receives an ltupdate, Q, v, tgt then it
  sends an ltecho, Q, v, tgt to every other server in
  Q, unless
• t lt t where t is some timestamp previously
  received from the same client, or
• It has previously received ltupdate, Q, v , tgt
  from the same client, where v ? v
• If a server receives ltecho, Q, v, tgt from every
  other server in Q, then it sends ltread, Q, v, tgt
  to every other server in Q
• If a server receives ltready, Q, v, tgt from a set
  of servers that is not a subset of any failure
  set B, then it sends ltready, Q, v, tgt to every
  other server in Q (if it has not done so already)
• If a server receives ltready, Q, v, tgt from a set
  of servers in Q that does not include any servers
  in some failure set B, then it delivers the
  message, which consists of
• If t gt the servers current timestamp, it updates
  its v and t values
• It sends an ack to the client

34
Server Update Protocol Step 1
S0
Check against previous updates ltupdate, Q, v,
tgt Abort if t gt t or t t and v ?
v. Otherwise, send echos
S1
S5
S4
S2
S3
35
Server Update Protocol Step 2
S0
S1
S5
S4
S2
S3
36
Server Update Protocol Step 3
S0
S1
S5
S4
S2
S3
37
Server Update Protocol Step 4
S0
Deliver ltv, tgt If t gt current timestamp, update
variable with ltv, tgt
S1
S5
S4
S2
S3
38
The End