Distributed systems Principles and Paradigms 6. Consistency and Replication

1
Distributed systems Principles and Paradigms6.
Consistency and Replication

• B4. ??? ??? ???

2
?????

• Replication ? Consistency
• Consistency models
• Distribution protocols
• Consistency protocols

3
1. Replication ? Consistency

• Replication???
• Replication????????
• Consistency???
• Replication as Scaling technique

4
Replication???

• Replication(??)
• ??????????????
• Replication???
• Reliability(???)????
• Performance?????
• ?
• Web server?replicate??
• ?????(memory, web pages)

5
Replication????????

• Replication??????????
• ??????????????
• ??????
• ??????
• Replicas(???)??????(inconsistency)
• ??replica??update???replica???????????????replica
  ?????
• ??replicas??????(consistency)???

6
Consistency???

• Replicas???????
• ????????
• ???????
• ????????!!
• ??????????????
• ????????????????consistency??????

7
Replication as Scaling technique

• Scaling technique
• System?Scale??????
• System???????????????performance??????????????
• Replication?scaling technique???????
• ?????????????????????????

8
2. Consistency models

• 2.1. Consistency model ???
• 2.2. Data-centric consistency
• 2.3. Client-centric consistency

9
2.1. Consistency model ???

• Consistency????????????read/write??????
• ???????data store???
• Distributed shared memory, distributed shared
  database, distributed file system??data
  store????
• Consistency model???processes?data store?????????
• ?????????????????data store????????

10
Consistency model

• ????????read????????????write?????????Distributed
  system(???replica???????????)????????????????
• ????distributed system??????????????????????write?
  ?????write?????????
• ????????????????
• ??????read?????????????
• Read/write????????????????????

11
2.2. Data-centric consistency models

• Data store????????????consistency????????
• Concurrent processes????data store?update?????????
  ?????????
• ??concurrency???consistency???
• Data-centric consistency models???????????????????
  
• ?????(strictness)???????

12
2.2.Data-centric consistency models

• Strict consistency
• Linearizability Sequential consistency
• Causual consistency
• FIFO consistency
• Weak consistency
• Release consistency
• Entry consistency

13
Strict consistency(1)

• Strict consistency?????
• ?????x????Read??????????????Write???????????(Any
  read on a data item x returns a value
  corresponding to the result of the most recent
  write on x)
• ??????????
• ????????most recent write????????????????????????

14
Strict consistency(2)

• Distributed system???????

Machine 2
Machine 1
T1 Read(x)
T2(gtT1) Write(x)
x
Network
???????????????????T1 lt T2?????Machine
2???Write?Read??????
15
Sequential Linearizability Consistensy

• Sequential consistency
• ???????(???????)??????????????????????????????????
  ???????
• ?
• Wi(x)a i????????x??a?????
• ?
• P1 W(x)a W(x)a
• P2 W(x)b W(x)b
• P3 R(x)b R(x)a
  R(x)b R(x)a
• P4 R(x)b R(x)a R(x)a R(x)b

??
16
Linearizability consistency

• Sequential consistency????????
• ????timestamp?????
• tsOp1(x) lt tsOp2(x) ? Op1(x)?Op2(x)?????????????
  (tsOp1(x)???Op1?timestamp)
• Sequential consistency????(???)

17
Causal consistency(1)

• ?????????(causally related)
• Read(x) Write(x) (potentially causal related)
• Write(x) Read(x) (causal related)
• Causal consistency
• ??????????Writes?????????????????????????????Write
  s(concurrent writes)????????

18
Causal consistency(2)

• ?
• P1 W(x)a W(x)c
• P2 R(x)a W(x)b
• P3 R(x)a R(x)c
  R(x)b
• P4 R(x)a R(x)b
  R(x)c
• W1(x)a ? W2(x)b??????????????. W2(x)b ?
  W1(x)c????????????
• ?????sequential consistency???

19
FIFO consistency

• ???????Writes???????????????????????????Writes????
  ????????????
• ?
• P1 W(x)a
• P2 R(x)a W(x)b W(x)c
• P3 R(x)b
  R(x)a R(x)c
• P4 R(x)a
  R(x)b R(x)c
• ?????causal consistency ???(W1(x)a ?
  W2(x)b???????????)

20
Weak consistency(1)

• FIFO consistency?????????????Writes???????????????
  ???
• ??Writes????? (??????????)???????????
• Weak consistency???????????????????????
• Synchronization variable ???
• Synchronization variable S??synchonize????????????
  ???????local ??????update???(????????)

21
Weak consistency(2)

• Weak consistency???
• Synchronization variable ?access?sequentially
  consistent (P1?P2????synchronize(S1),
  synchronize(S2)???????2?????????????)
• Synchronization variable???????????????copies???Wr
  ites????????????
• Synchronize????????read/write??????

22
Weak consistency(3)

• Weak consistency??
• P1 W(x)a W(x)b S
• P2 R(x)a R(x)b S
• P3 R(x)b R(x)a S
• Weak consistency????
• P1 W(x)a W(x)b S
• P2 S R(x)a

23
Release consistency(1)

• Weak consistency??critical region?????????????????
  ???????local writes?????????????writes???gather???
  ?
• Release consistency???critical region???????Acquir
  e??????????????Release??????
• Acquire????local????update???
• Local update??????????
• Release????local update??????
• Local????update????

24
Release consistency(2)

• ?
• P1 Acq(L) W(x)a W(x)b Rel(L)
• P2 Acq(L) R(x)b Rel(L)
• P3 R(x)a
• X?L?protected????????
• Release??????update????????????(??????????????????
  ???????)
• ????Release?????update??????????????Acquire???????
  ??????update????
• Lazy Release Consistency???

25
Entry consistency

• Release consistency?????????????????synchronizatio
  n variable???????????Acquire?Release??????????
• ???????????????????????Lock?Acqquire???????????
• ?
• P1 Acq(Lx) W(x)a Acq(Ly) W(y)b Rel(Lx) Rel(Ly)
• P2 Acq(Lx) R(x)a R(y)NIL
• P3 Acq(Ly) R(b)b

26
2.3. Client-centric consistency models

• ????update???????????(?????????)data
  store????????consistency model???????
• ??????eventual consistency model???
• Eventual consistency????????????client-centric
  consistency model???

27
Eventual consistency

• WWW?web pages???????proxy?cache??????(??????????)?
  
• ??????????????????update???
• Eventual consistency
• ???????(high degree of inconsistency)
• ???????update?????????consistent???(gradually
  become consistent)

28
Client-centric consistency(1)

• Eventual consistency????????client????replica?????
  ?????????????????
• ???????????client-centric consistency???
• Client-centric consistency????client? data
  store?access??????????????

29
Client-centric consistency(2)

• ??
• Xit local copy Li??? t??x??
• ??????t???Write??????WS????
• WS(x1t1) t1?L1?x???write???
• WS(x1t1x2t2) L1, t1, x?write, L2, t2,
  x?write?????

30
Client-centric consitency models

• Monotonic Reads
• Monotonic Writes
• Read Your Writes
• Writes Follow Reads

31
Monotonic Reads

• ???????x?read????????(???????)x?read??????read????
  ?????
• Monotonic Reads??
• L1 WS(x1) R(x1)
• L2 WS(x1x2) R(x2)
• Monotonic Reads????
• L1 WS(x1) R(x1)
• L2 WS(x2) R(x2) WS(x1x2)

32
Monotonic Writes

• ??????????x?Write????????????Write(x)?????????
• Monotonic Writes??
• L1 W(x1)
• L2 W(x1) W(x2)
• Monotonic Writes????
• L1 W(x1)
• L2 W(x2)

33
Read Your Writes

• ???????write??????????(???????)read??????
• Read Your Writes??
• L1 W(x1)
• L2 WS(x1x2) R(x2)
• Read Your Writes????
• L1 W(x1)
• L2 WS(x2) R(x2)

34
Writes Follow Reads

• ???????read(x)???write(x)?????????Write??x???read?
  ????????????write???
• Writes Follow Reads??
• L1 WS(x1) R(x1)
• L2 WS(x1x2) W(x2)
• Writes Follow Reads????
• L1 WS(x1) R(x1)
• L2 WS(x2) W(x2)

35
3. Distribution protocols

• 3.1. Replica???
• 3.2. Update ???
• 3.3. Epidemic protocols

36
3.1. Replica???

• Permanent Replicas(???replicas)
• Replicas?????
• Distributed data store??????
• Server-Initiated Replicas
• Server?????replicas
• ???request?????????replica???
• Client-Initiated Replicas
• Client?????replica(Cache)
• Client??????LAN??machine???????

37
Server-Initiated replicas ???replicate???

• Access counting ????????????Threshold?????replica
  te
• P????clients?Q????????F?????????????cnt(P,F)????Cn
  t(P,F) gt threshold???F?P??????

38
3.2. Update propagation

• State vs. Operation
• Pull vs. Push Protocols
• Unicasting vs. Multicasting

39
State vs. Operations

• ????????
• Update?notification??????
• Invalidation protocols
• Update???????invalidate??
• ??invalidated??????(write, read)????????update????
  
• ????????????
• Update????????????
• Write/read????????

40
Pull vs. Push protocols

• Push-based(server-based) protocols
• Client????????update?????
• Pull-based(client-based) protocols
• Server????client???server?update???????

41
Unicasting vs. Multicasting

• Unicasting
• N??servers?update?????????N??messages???(server???
  1?)
• Multicasting
• N??servers?update?????????1??message?multicast??(m
  ulticasting service?????)?

42
3.3. Epidemic Protocols

• Eventual-consistency system ?update????epidemic
  protocols???????????????????????
• Update Propagation Models(Update??????)
• Removing data

43
Update Propagation Models

• ??
• Infective server(??server) update????????????????
  ??
• Susceptible server ??update???????
• Removed server ??update????update?????????????
• ?????
• Anti-entropy
• Rumor speading (gossiping)

44
Anti-entropy

• ?????P??????????Q?????update???. 3????
• P ????update?Q?push????
• P?Q???update?pull????
• P?Q?update?????
• ????????????push?????infective??susceptible???????
  ??????update??????Pull?????susceptible?infective??
  ???????????update?????
• ???????infective??????????????????update???

45
Rumor Spreading(Gossiping)

• Update????????P?????????Q?????update????????Q?????
  update???????P?update?interest????(P?removed??????
  1/k?????)
• ???????P?update???????????Removed???(Removed???1??
  ????)?
• Gossiping?update???????????????update?????????
• ????
• K 3 s lt 0.02

46
Removing data

• Epidemic?side-effect ???????????????
• ????????update????????????
• ???
• ????????????(death certificates???)
• ??????death certificates?????

47
4. Consistency Protocols

• Consistency protocols???consistency???????????
• Primay-based Protocols
• Remote-Write Protocols
• Local-Write Protocols
• Replicated-write Protocols
• Active Replication
• Quorum-based Protocols
• Cache-coherence Protocols

48
4.1.Primary-based Protocols

• Data store??????x???primary????????primary?write(x
  )?????
• 2????
• Remote-write Protocols
• Local-write Protocols

49
Remote-write Protocols(1)

• ???read/write????????(remote)server?????
• ????replicate???
• ???????? ? primary-backup protocols
• Write????????????????read????????????Write??update
  ???????
• Write?????process????????
• Sequential consistency?????????

50
Remote write protocol(2)
Single Server for data item x
Client
Client
x
W1
W4
R1
R4
W2
R2
Data store
W3
R3
W1 Write request W2 Forward request to
server W3 Acknowledge request completed W4
Acknowledge request completed
R1 Read request R2 Forward request to
server R3, R4 response
51
Primary-backup protocol
Primary server for item x
Client
Client
x
Backup server
W1
W5
R1
R2
W4
W4
W3
W3
W3
W2
W4
Data store
W1 Write request W2 Forward request to
server W3 Tell backup to update W4 Acknowledge
update W5 Acknowledge write complete
R1 Read request R2 Response
52
Local-Write Protocols

• Write?local?????
• 2??????
• ??????????(no replica)
• ?????????local??????write??
• Full migration???
• ???????????primary???write?????
• Primary????local??????write??
• Non-block?????process?read??

53
Local-write protocol(no replica)
Client
Current Server for item x
New server for item x
x
1
4
2
3
Data store

1. Read or Write request
2. Forward request to current server for x
3. Move item x to clients server
4. Return result of operation on clients server

54
Local write protocol(replicas)
55
4.2.Replicated-Write Protocols

• Write??????replicas?????
• 2????
• Active Replication
• Quorum-based Protocols

56
Active Replication

• Replica??????????????write???
• Update?operation??????????????????operation???????
  
• ???????????????????
• Totally-ordered multicast???
• Coordinator????????????????????????????(sequencer)
  
• Relicated invocation???
• Ojbect A invoke B.method, B.method?C.method?invoke
  , B?replicated???C.method????invoke???

57
Replicated invocation????

• B?C?coordinator???
• B.method()???????B?coordinator??forwad???Coordinat
  or??????????invoke???
• C?????C?coordinator???

58
Quorum-based Protocols

• Voting???
• ?Replicated item?Read/Write???????????permission??
  ???????????????
• ?Distributed file system??????N??replicated
• ???????write??????N/21??????write?????????read?
  ???version number?N/21?????????????????OK.
• Read?????????write????(N/21??????????)?????????
  ???
• ????read quorum(Nr), write quorum(Nw)?????????????
  ??OK.
• Nr Nw gt N (read/write ?????)
• Nw gt N/2 (write/write??)

59
4.3.Cache-Coherence Protocols

• Cache-coherence protocols?cache?server-initiated
  replica??????????????
• Coherence dection strategy
• ??inconsistency????detect??
• Static solution ?????compiler???
• Dynamic solution runtime????????
• Coherence enforcement strategy
• ?????????????
• Update??????server?invalidate???
• Update?????

60
???

• Replication?consistency????
• Replication????????????
• ????consistency????????????
• Consistency????????????????????????
• Consistency?????????consistency protocols????