Consistency

1
Consistency ReplicationChapter No. 6

• Muhammad Rafi
• Meng Min
• Guan Donghai

2
Motivation

• Large Scale distributed systems required
  scalability in every respect of their function.
• Data are generally replicated to enhance
  reliability, improve performance and increase
  availability
• One major problem of replication is to maintain
  consistency
• To make consistency we need to have
• Distribution of updates (amount, frequency,
  means)
• Keeping the replica consistent (immediate updates
  is required)
• We will discuss a variety of protocols for data
  and client centric consistency
• We also discuss the distribution protocols and
  consistency protocols
• Example
• Simulation

3
Outline

• Motivation Rafi
• Introduction Meng Min
• Data-Centric Consistency Models Meng Min
• Client-Centric Consistency Models Donghai
• Distribution Protocols Donghai
• Consistency Protocols Rafi
• Examples Rafi
• Simulation of Consistency technique Rafi

4
Content

• Definition of Consistency and Replication
• 6.1 Introduction
• Reasons for Replication Problems of Replication
  
• Object Replication
• 6.2 Data-centric consistency models

5
Consistency and Replication

• Definition
• Replication
• Replication of data
• Consistency
• Consistency of replicated data
• Reason for Consistency
• Keep replicas to be the same

6
Consistency and Replication

• 6.1 Introduction
• Reasons for Replication
• Reliability
• Performance
• Problem of Replication
• Consistency Problem
• Whenever a replica is updated, that replica
  becomes
• different from the others. ? Synchronization
  Problem

Performance Replication
Consistency Synchronization
7
Object Replication (1)

• Purpose
• managing data in distributed systems
• Object Replication
• Consider objects instead of data alone
• Benefit of encapsulating and operating data

8
Object Replication (2)

• Organization of a distributed remote object
  shared by two different clients.

9
Object Replication (3)

• A remote object capable of handling concurrent
  invocations on its own.
• A remote object for which an object adapter is
  required to handle concurrent invocations

10
Data-centric Consistency Models

• The general organization of a logical data store,
  physically distributed and replicated across
  multiple processes.

Consistency Model A contract between processes
and the data store.
11
Data-centric Consistency Models

• Data-centric Consistency Models
• Strict Consistency
  Strict
• Linearizability and Sequential Consistency
• Causal Consistency
• FIFO Consistency
• Weak Consistency
• Release Consistency
• Entry Consistency
  Weak

12
Strict Consistency

• Condition
• Any read on a data item x returns a value
  corresponding to the result of the most recent
  write on x.
• Disadvantage Assume the existence of absolute
  global time.
• Behavior of two processes, operating on the same
  data item.
• A strictly consistent store.
• A store that is not strictly consistent.

13
Linearizability and Sequential Consistency

• Sequential Consistency Condition The result of
  execution is the same as if the operations by all
  processes were executed in some sequential order
  and the operations appear in the order specified
  by its program. Time does not play a role.
• A sequentially consistent data store.

14
Causal Consistency (1)

• Necessary conditionWrites that are potentially
  causally related must be seen by all processes in
  the same order. Concurrent writes may be seen in
  a different order on different machines.

This sequence is allowed with a
causally-consistent store, but not with
sequentially or strictly consistent store.
15
Causal Consistency (2)

• A violation of a casually-consistent store.
• A correct sequence of events in a
  casually-consistent store.

16
FIFO Consistency

• Necessary ConditionWrites done by a single
  process are seen by all other processes in the
  order in which they were issued, but writes from
  different processes may be seen in a different
  order by different processes.

A valid sequence of events for FIFO consistency
17
Weak Consistency (1)

• Properties
• Accesses to synchronization variables associated
  with a data store are sequentially consistent
• No operation on a synchronization variable is
  allowed to be performed until all previous writes
  have been completed everywhere
• No read or write operation on data items are
  allowed to be performed until all previous
  operations to synchronization variables have been
  performed.

18
Weak Consistency (2)
19
Release Consistency (1)

• Rules
• Before a read or write operation on shared data
  is performed, all previous acquires done by the
  process must have completed successfully.
• Before a release is allowed to be performed, all
  previous reads and writes by the process must
  have completed
• Accesses to synchronization variables are FIFO
  consistent (sequential consistency is not
  required).

20
Release Consistency (2)

• A valid event sequence for release consistency.

21
Entry Consistency (1)

• Conditions
• An acquire access of a synchronization variable
  is not allowed to perform with respect to a
  process until all updates to the guarded shared
  data have been performed with respect to that
  process.
• Before an exclusive mode access to a
  synchronization variable by a process is allowed
  to perform with respect to that process, no other
  process may hold the synchronization variable,
  not even in nonexclusive mode.
• After an exclusive mode access to a
  synchronization variable has been performed, any
  other process's next nonexclusive mode access to
  that synchronization variable may not be
  performed until it has performed with respect to
  that variable's owner.

22
Entry Consistency (2)

• A valid event sequence for entry consistency

23
Summary of Consistency Models
24
6. Consistency and Replication 6.3
Client-Centric Consistency models6.4
Distribution Protocols

• Guan Donghai
• 2005-11-15

25
Client-Centric Consistency Models

• The previously studied consistency models
  concern themselves with maintaining a consistent
  (globally accessible) data-store in the presence
  of concurrent read/write operations.
• Another class of distributed datastore is that
  which is characterized by the lack of
  simultaneous updates. Here, the emphasis is more
  on maintaining a consistent view of things for
  the individual client process that is currently
  operating on the data-store.

26
More Client-Centric Consistency

• How fast should updates (writes) be made
  available to read-only
• processes?
• Think of most database systems mainly read.
• Think of the DNS write-write conflicts do no
  occur.
• Think of WWW as with DNS, except that heavy use
  of client-side caching is present even the
  return of stale pages is acceptable to most
  users.
• These systems all exhibit a high degree of
  acceptable inconsistency with the replicas
  gradually become consistent over time.

27
Toward Eventual Consistency

• The only requirement is that all replicas will
  eventually be the same.
• All updates must be guaranteed to propogate to
  all replicas eventually!
• This works well if every client always updates
  the same replica.
• Things are a little difficult if the clients
  are mobile.

28
Eventual Consistency
A mobile user accessing different replicas of a
distributed database has problems with eventual
consistency
29
Client-Centric Consistency

• Four models in client-centric consistency
• Monotonic Read Consistency
• Monotonic Write Consistency
• Read-your-writes Consistency
• Writes-follows-reads Consistency

30
Client-Centric Consistency

• xit -version of data x at copy Li at time t
• xit is result of a set of write operations
  applied to x since initialization
• this set is notated as WS(xit)
• When these operations later (t2) are performed
  to x at copy Lj ,it is written as WS(xit1
  xjt2)

31
Monotonic Reads
If a process reads the value of a data item x,
any successive read operations on x by that
process will always return that same value or a
more recent value.

• A monotonic-read consistent data store
• A data store that does not provide monotonic
  reads.
• Example distributed email database

32
Monotonic Writes
A write operation by a process on a data item x
is completed before any successive write
operation on x by the same process
a) A monotonic-write consistent data store. b) A
data store that does not provide monotonic-write
consistency. Example The MW-guarantee could be
used by a text editor when editing replicated
files
33
Read Your Writes
The effect of a write operation by a process on
data item x will always be seen a successive read
operation on x by the same process.

• A data store that provides read-your-writes
  consistency.
• A data store that does not.
• Example updating passwords

34
Writes Follow Reads
A write operation by a process on a data item x
following a previous read operation on x by the
same process, it is guaranteed to take place on
the same or a more recent value of x that was
read.

• A writes-follow-reads consistent data store
• A data store that does not provide
  writes-follow-reads consistency
• Example replicated bulletin board database

35
Client centric model-Summary

• Monotonic read
• If a process reads the value of a data item
  x, any successive read operation on x by that
  process will always return that same value or a
  more recent value
• Monotonic write
• A write operation by a process on a
  data item x is completed before any successive
  write operation on x by the same process
• Read your writes
• The effect of a write operation by a
  process on a data item x will always be seen by
  a successive read operation on x by the same
  process
• Writes follow reads
• A write operation by a process on a data
  item x following a previous read operation on x
  by the same process, is garanteed to take place
  on the same or more recent values of x that was
  read

36
Distribution Protocols

• Purpose Solve the following problem
• What is exactly propagated?
• Where updates are propagated?
• By whom propagation is initiated?
• Three Distribution Protocols
• Replica Placement
• Update Propagation
• Epidemic Protocols

37
Replica Placement
The logical organization of different kinds of
copies of a data store into three concentric
rings.
38
Permanent Replicas

• Permanent Replicas
• Initial set of replicas that constitutes a
  distributed data store
• Typically, the number of it is small
• Example Web site

39
Server-Initiated Replicas

• Server-Initiated Replicas
• Created at the initiative of the owner of the
  data store
• Exist to enhance performance
• Work Scheme
• How to decide where and when replicas should
  be created or deleted?
• Each server keeps track of access counts per
  file, and where access requests come from.
• When a server Q decides to reevaluate the
  placement of the files it stores, it checks the
  access count for each file.
• If the total number of access requests for F
  at Q drops below the deletion threshold del
  (Q,F), it will delete F unless it is the last
  copy.

40
Server-Initiated Replicas
Counting access requests from different clients.
41
Client-Initiated Replicas

• Work Scheme
• When a client wants access to some data, it
  connects to the nearest copy of the data store
  from where it fetches the data it wants to read,
• When most operations involve only reading
  data, performance can be improved by letting
  the client store requested data in a nearby
  cache.
• The next time that same data needs to be
  read, the client can simply fetch it from this
  local cache.
• Client-Initiated Replicas
• Created at the initiative of clients
• Commonly known as client caches

42
Update Propagation

• Introduction
• Generally initiated at a client
• Subsequently forwarded to one of the copies
• Three design issues
• State Vs. Operations
• What is actually to be propagated
• Pull Vs. Push Protocols
• Whether updates are pulled or pushed
• Unicasting Vs. Multicasting
• Whether unicasting or multicasting should be used

43
State Vs. Operations

• What is actually to be propagated?
• Propagate only a notification of an update
• Use little network bandwidth
• Work best when read-to-write ratio is
  relatively small
• Transfer data from one copy to another
• Useful when the read-to-write ratio is
  relatively high
• Propagate the update operation to other copies
• Tell each replica swhich update operation it
  should perform
• Updates can often be propagated at minimal
  bandwidth costs

44
Pull versus Push Protocols

• Whether updates are pulled or pushed?
• Push-based approach (server-based)
• Updates are propagated to other replicas without
  those replicas even asking for the updates
• Pull-based approach (client-based)
• a server or client requests another server to
  send it any updates it has at that moment.

A comparison between push-based and pull-based
Protocols in the case of multiple client, single
server systems.
45
Unicasting Vs. Multicasting

• Whether unicasting or multicasting should be
  used?
• Introduction
• With multicasting, the underlying network
  takes care of sending a message efficiently to
  multiple receivers.
• In unicast communication, when a server that
  is part of the data store sends its update to N
  other servers, it does so by sending N separate
  messages, one to each server.
• Compare
• In many cases, it is cheaper to use available
  multicasting facilities.
• Multicasting can often be efficiently
  combined with a push-based approach to
  propagating updates.

46
Epidemic Protocols

• Ensures eventual consistency
• Epidemic spreading of updates, propagates
  updates to all replicas efficiently
• Does not solve any update conflicts directly
• Concern update to all replicas in as few
  messages as possible
• Terminology
• Infective a server that holds an update and
  it is willing to spread
• Susceptible a server that has not been
  updated
• Removed a server that is not willing or able
  to update

47
Epidemic Protocols

• Anti-entropy Propagation Approach
• A server P picks another server Q at random,
  three approaches in exchanging
• updates
• 1. P only pushes its updates to Q
• 2. P only pulls in new updates from Q
• 3. P and Q exchange updates (push-pull)
• Gossip Protocol
• If server P has just been updated for data
  item x,
• Contact an arbitrary server Q and push
  update to Q
• If Q was already updated, P may lose
  interest with some probability
• Good way of rapidly spreading updates
• Note cannot guarantee that all servers
  will actually be updated

48
Consistency ReplicationChapter No. 6Muhammad
Rafi

• 6.5 Consistency Protocol
• 6.6 Examples Orca

49
Consistency Protocol

• A consistency protocol describes an
  implementation of a specific consistency model
• The consistency model in which operations are
  globally serialized are the most important and
  widely applied models
• Sequential Consistency
• Weak consistency with synchronization variables
• Atomic transaction
• There are two ways to classify the consistency
  protocols, one with the primary or one with write
  on any replica
• We discussed the following protocols
• Primary-Based Protocols
• Remote-Write Protocols Local-Write Protocols
• Replicated-Write Protocols
• Active Replication Quorum-Based Protocols
• Cache-coherence Protocols

50
Primary-Based Protocols

• Primary based protocols use a primary server to
  responsible for every write operation in
  distributed data sources.
• There are several strategies for Primary based
  approach
• Remote-write protocol with no backup replica
• Remote-write protocol primary-backup approach
  (passive replication)
• Local-write protocol with single migrating
  primary copy (no backup)
• Local-write protocol with migrating primary copy
  and non-migrating backups (Primary-backup
  approach)

51
Primary-Based Protocols
Remote-Write Protocols (1) (No Backup Replica)

• Primary-based remote-write protocol with a fixed
  server to which all read and write operations are
  forwarded.

52
Primary-Based Protocols
Remote-Write Protocols (2) Backup Replica

• The principle of primary-backup protocol.

53
Primary-Based Protocols

• Problems
• Update process implemented as blocking
• Update process may wait longer to continue

Only p can write to A. All writes to A must be
forwarded to p
Others read A locally
54
Primary-Based Protocols
Local-Write Protocols (1) (No Backup Replica)

• Primary-based local-write protocol in which a
  single copy is migrated between processes.
• Non-Replicated, but distributed move resource
  to client.
• Problem- keeping track about where the single
  copy is located. (LAN Broadcast)

55
Primary-Based Protocols
Local-Write Protocols (2) Backup Replica

• Primary-backup protocol in which the primary
  migrates to the process wanting to perform an
  update.

56
Replicated-Write Protocols

• In replicated-write protocols, write operations
  can be carried out at multiple replicas
• A distinction can be made between Active
  Replication ( updated goes to all replicas) and
  consistency protocols based on majority voting.
• Active Replication
• In active replica operation is sent to each
  replica and an associated process carries out
  that operation there.
• Operation order is very important
• Totally-ordered multicast Lamport Timestamps
• Central coordinator (Sequencer) can be a
  bottleneck
• Another problem is Replicated Invocation (A?B? C)
• No general purpose solution available

57
Replicated-Write Protocols
Active Replication (1)

58
Replicated-Write Protocols
Active Replication (2)

• Forwarding an invocation request from a
  replicated object.
• Returning a reply to a replicated object.

59
Replicated-Write Protocols
Quorum-Based Protocols

• Basic Idea require clients to request and
  acquire the permission(Quorum) of multiple
  servers before either reading or writing a
  replicated data item
• Scheme
• - Suppose
• N replicas exist, NRread quorum, NWwrite
  quorum
• - Constraints
• 1. NRNWgtN 2. NWgtN/2
• The first constraint is used to prevent
  read-write conflicts, whereas the second prevents
  write-write conflicts. Only after the appropriate
  number of servers has agreed to participate can a
  file be read or written.

60
Replicated-Write Protocols
Quorum-Based Protocols

• Three examples of the voting algorithm
• A correct choice of read and write set
• A choice that may lead to write-write conflicts
• A correct choice, known as ROWA (read one, write
  all)

61
Cache-Coherence Protocols

• Caches are special case of replication
• Cache solutions differ in their coherence
  detection strategy
• Distributed Database Example
• In a Transaction a cache data is verified at the
  server for consistency and if OK transaction is
  executed locally based on cache.
• Let the transaction execute while checking for
  cache consistency, if not Ok abort the
  transaction
• Verify cache data only when a transaction is
  committed.
• Coherence enforcement strategy
• Server send an invalidation whenever server data
  is changes.
• Propagate the update
• Hardware Cache
• Write through
• Write back

62
Orca Parallel Programming Language

• Orca is a language for parallel programming on
  distributed systems, based on the shared
  data-object model.
• This model is a simple and portable form of
  object-based distributed shared memory.

63
Orca
OBJECT IMPLEMENTATION stack top
integer variable indicating the top
stack ARRAYinteger 0..N-1 OF integer
storage for the stack OPERATION push (item
integer) function returning nothing BEGIN
GUARD top lt N DO stack top
item push item onto the stack
top top 1 increment the stack pointer
OD END OPERATION pop()integer
function returning an integer BEGIN
GUARD top gt 0 DO suspend if the stack is
empty top top 1 decrement
the stack pointer RETURN stack
top return the top item OD
ENDBEGIN top 0 initializationEND

• A simplified stack object in Orca, with internal
  data and two operations.

64
Management of Shared Objects in Orca

• Four cases of a process P performing an operation
  on an object O in Orca.

65
Casually-Consistent Lazy Replication

• The general organization of a distributed data
  store.
• Clients are assumed to also handle
  consistency-related communication.

66
Processing Read Operations

• Performing a read operation at a local copy.

67
Processing Write Operations

• Performing a write operation at a local copy.

68
Summary (1/2)

• Two reasons to replicate data 1. Reliability 2.
  Performance
• Replication introduced Consistency problems
• Data centric consistency models are
• Strict consistency hard to implement in
  distributed systems
• Sequential (all operation seen by everyone in
  same order)and Linearizability (order operation
  according to a global clock) are good choice in
  concurrent programming
• Client centric consistency model do not perceive
  that the data is shared. These model focuses on
  individual client aspect of consistency.
• In mobile/ distributed database client may
  connect to different replicas in the course of
  time but all the differences must be transparent
  from the client.

69
Summary (2/2)

• To propagate updates among different replicas
  different techniques can be applied.
• What is exactly propagated ?
• Where updates are propagated?
• By whom propagation is initiated ?
• Frequency of updates?
• Consistency Protocols describe a specific
  implementation of a consistency model.
• Primary based protocols
• Replicated-write protocols
• Orca example

70
Question Answer Thank you!!!