Synchronization

1
Synchronization

• Chapter 5

2
Synchronization

• Has any one given any through to how time is
  kept?
• How do we keep the time reasonably correct on our
  own computers?

3
Synchronization

• Why is time important?

4
Clock Synchronization

• When each machine has its own clock, an event
  that occurred after another event may
  nevertheless be assigned an earlier time.

5
Time

• What is time?

6
Physical Clocks (1)

• Computation of the mean solar day.

7
Physical Clocks (2)

• TAI seconds are of constant length, unlike solar
  seconds. Leap seconds are introduced when
  necessary to keep in phase with the sun.

8
Clocks

• Two clocks hardly ever agree. This difference is
  called clock skew.

9
Problems with Maintaining Accurate Time

• Timer accuracy, modern timer chips have relative
  tick rate errors of 10-5 which means that a timer
  that generates 60 ticks a second, generates
  216,000 ticks an hour /- 2 ticks.
• This relative error is called the clock drift
  rate and manufacturers of clocks will specify a
  maximum drift rate for their clocks.

10
Clock Synchronization Algorithms

• The relation between clock time and UTC when
  clocks tick at different rates.

11
Clock Synchronization

• A computer can set its clock a number ways. It
  can use aGPS system (/- 1msec) or a WWV radio
  receiver (/- 3 to 10msec) or a Geostationary
  Operational Environmental Satellite (GOES /- .1
  msec) but typically it will use another computer
  on the network to obtain correct time.

12
Problems with Maintaining Accurate Time

• A problem with clocks getting time over the
  network has to do with the latency that occurs
  when one clock is used to set another clock.
• This problem can be addressed by calculating the
  propagation time for the update.
• The client knows when it sent the request and
  when it received the update relative to its own
  clock. Half this time is the approximate time for
  the time server to sent the update. The client
  can then correct the update by this amount.

13
Cristian's Algorithm

• Getting the current time from a time server.

14
Christian Algorithm

• The network delay can be calculated by (T1-T0)/2
• The new system time then becomes.
• Tnew Tserver (T1-T0)/2

15
Problems with Maintaining Accurate Time

• A more serious problem is the actual correction
  and how it should be applied.
• For example in an extreme case assume the current
  client time is 120000pm and it gets an update
  from the time server that says the correct time
  is 115948pm. What kinds of problems might occur
  if this time correction was applied immediately?

16
Problems with Maintaining Accurate Time

• Time changes should be applied gradually by
  whatever algorithm is in use. Otherwise problems
  such as makefile difficulties might occur when a
  clock is set back.

17
Correcting Clocks

• If a clock is too fast it has to be slowed down
  until it has the correct time. If its too slow
  it has to be sped up until it has the correct
  time. The adjustment is called a linear
  compensation function. Clocks should never go
  backwards.

18
Berkley Algorithm

• In Christians algorithm the time server is
  passive, in the Berkley algorithm the time server
  is active.
• This algorithm uses a polling technique to get
  times from all participating servers. From these
  times an average is computed which the time
  server uses to slow down or speed up the time on
  participating systems.

19
The Berkeley Algorithm

• The time daemon asks all the other machines for
  their clock values
• The machines answer
• The time daemon tells everyone how to adjust
  their clock

20
Synchronization

• Has any one heard of NTP or Network Time Protocol?

21
NTP

• How does it work and how accurate is it?

22
NTP

• Enable clients across the Internet to be
  accurately synchronized to UTC (universal
  coordinated time) despite message delays.
  Statistical techniques are used for filtering
  data and gauging the quality of the results.
• Provide a reliable service that can survive
  lengthy losses of connectivity. This means having
  redundant paths and redundant servers.
• Enable clients to synchronize frequently and
  offset the effects of clock drift.
• Provide protection against interference
  authenticate that the data is from a trusted
  source.

23
NTP

• The ntpd program is an operating system daemon
  which sets and maintains the system time of day
  in synchronism with Internet standard time
  servers.
• Check any UNIX box that is an important server
  and it either has ntpd running or uses rdate set
  up in a crontab.

24
NTP

• Christians Algorithm and the Berkley algorithm
  are intended for intranet configurations. NTP is
  a time protocol designed for the Internet.
• NTP has an accuracy of 1-50msec worldwide.

25
NTP

• NTP servers are divided up into different
  categories.
• Primary servers are connected to external,
  accurate UTC clocks.
• Secondary servers are synchronized with the
  primary servers.
• The NTP configuration of servers is hierarchical.

26
NTP
UTC source -Atomic Clock
Accurate
1
Primary Server
Stratum 1
2
2
Secondary Server
Stratum 2
3
3
3
Stratum 3
Less accurate
W/S
Workstations and servers
27
NTP

• NTP servers synchronize using one of 3 modes.
• Multicast mode
• Used in high-speed LAN, achieves low accuracy
  levels
• Procedure-call
• Similar to Christians algorithm, suitable for
  higher accuracies than multicasting.
• Symmetric mode
• Achieves the highest level of accuracy, usually
  implemented in low strata numbers (used between
  redundant servers). In this mode pairs of time
  servers with a know association exchange times.

28
NTP

• In all modes messages are delivered unreliably
  using standard UDP protocol.
• In the procedure-call and symmetric mode servers
  exchange messages, each message is time stamped
  with the local time of the last send and receive
  and the time of the current send and receive.
• There are therefore 4 times.

29
NTP
Ti-1
Ti-2
Server B
Time
m
m
Time
Server A
Ti-3
Ti
Four times for messages m and m
30
NTP

• For each pair of messages sent between the two
  servers NTP The client now has four time stamps.
  From this it can determine the round-trip delay
  and clock offset.
• Let a Ti-2 Ti-3and b Ti Ti-1
  Round-trip delay, D a - b andClock offset, O
  (ab)/2
• O is therefore an estimate of the offset between
  the clocks a D is a measure of the accuracy of
  this estimate.

31
NTP

• To successive pairs of offsets and delay times
  NTP calculates the quality of this estimate
  statistical using a filter dispersion method.
• Dispersion A measure, in seconds, of how
  scattered the time offsets have been from a given
  time server.

32
Logical Clocks

• When we write and execute message passing
  applications is the actual time important?

33
Logical Clocks

• Sometimes actual time is important but most time
  at least for scientific computational application
  we are concerned with the sequence of events.
• What really matters is the order in which event
  occurred.
• A logical clock is a clock that can generate
  sequence numbers that make sense system wide.

34
Lamport

• Lamport developed a happens before notation
  expressed as a-b meaning a happens before b.
• If a is a message sent and b is the message
  received then a must be before b.
• Likewise if a-b and b-c then a-c (transitive)

35
Lamport

• The importance of measuring time is to assign a
  time value to each event on which everyone will
  agree on the final order of events.
• Thus if, a-b then clock(a) and b occur on different processes that dont
  exchange messages then a-b is not true. We then
  say that these events are concurrent.

36
Lamport

• Consider the sequence of events depicted below

In 3 of the 6 messages (c,d and f) time is moving
back. If we sort the message by depart timestamp
the order is a,b,e,d,c,f instead of a,b,c,d,e,f
37
Lamport

• Lamport?s algorithm remedies the situation as
  follows
• Each message carries a timestamp of the sending
  time (according to the senders clock).
• When a message arrives and the receiver?s clock
  is less than the timestamp on the received
  message, the systems clock is forwarded to the
  messages timestamp 1. Otherwise nothing is
  done.

38
Lamport

• If we apply this algorithm to the same sequence
  of messages, we can see that message ordering is
  now preserved. Note that between every two
  events, the clock must tick at least once.

39
Lamport

• Lamports algorithm needs a monotonically
  increasing software counter for a clock that
  has to be incremented at least when events that
  need to be time stamped take place.
• For any two events a-b, L(a) represents the Lamport timestamp for x.

40
Lamport

• For the sequence of events in the figure we can
  conclude that a-b and c-d because events within
  a process are sequenced.
• b-c and d-f because Lamport imposes a
• send(m)-receive(m) relationship
• a-e and e-a because these events are concurrent.

41
Problems with Lamport

• It is possible that pairs of distinct events can
  have the same Lamport time stamp. To overcome
  this problem the process ID is utilized.
• Therefore we define a global logical timestamp as
  (Ti,i) where Ti is the Lamport timestamp and i
  the process ID.
• We then conclude that
• (Ti,i)
• Ti
• or Ti Tp and i
• Of course this is arbitrary since there is no
  physical evidence of the process ordering, but it
  does insure no two Lamport timestamps are the
  same.

42
Problems with Lamport

• Given that L(e) relationship between e and e such that e-e
• The answer is no we cant, unless we introduce
  the concept of vector clocks.

43
Vector Clock

• A vector clock in a system of N processes is a
  vector of N integers.
• Each process maintains its own vector clock (Vi
  for process Pi) to timestamp local events.
• Like timestamps, vector timestamps (the vector of
  N integers) are sent with each message.

44
Vector Clock Rules

• The vector is initialized to 0 for all processes
• Vij0 for i,j1,,N
• Before a Process Pi timestamps an event, it
  increments its element of the vector in its local
  vector.
• Vii Vii1
• A message is sent from process Pi with Vi
  attached to the message.
• When process Pj receives vector timestamp t, it
  compares the two vectors element by element
  setting its local vector clock to the higher of
  the two values.
• Vji max(Vji,ti) for i1,,N

45
Vector Clock

• We compare two vector timestamps by defining
• V V iff Vj Vj for j 1,N
• V
• For any two events e,e if e-e then V(e_ V(e) which is the same for Lamports algorithm
• With vector clocks we now have the condition that
  if V(e) e
• Two events are concurrent if neither V(e)nor V(e)

46
Vector Clock

• We can see that a-f because (1,0,0)
• But we can also see that c is concurrent with e
  because neither
• (0,0,1)

47
Global State

• What is a global state?

48
Global State

• It is the local state of each process, together
  with the messages that are currently in transit.
• What is the local state?

49
Global State

• Chandy and Lamport introduced the notion of a
  distributed snapshot as a means for recording the
  global state.
• The notion of a global state is illustrated in
  the following slide.

50
Global State (1)

• A consistent cut
• An inconsistent cut

51
Distributed Snapshot

• Assume a distributed system is represented by a
  collection of processes connected through
  unidirectional point-to-point communication
  channels.
• An initiating process P starts by recording its
  local state. Then it sends a marker in a message
  along each of its outgoing channels to initiate a
  global state snapshot.
• When process Q receives the marker it records its
  local state and send the marker out its outgoing
  channels. When it has received a marker from all
  its incoming channels it can send its resultant
  local state to the initiating process.

52
Global State (2)

• Organization of a process and channels for a
  distributed snapshot

53
Global State (3)

• Process Q receives a marker for the first time
  and records its local state
• Q records all incoming message
• Q receives a marker for its incoming channel and
  finishes recording the state of the incoming
  channel

54
Global State

• Why is the Global state important to know?

55
Global State

• It depends on what it is.
• For parallel scientific computations on a
  multicomputer one of the more difficult states to
  determine is that of the termination state.

56
Global State

• What problems might we have to deal with in
  detecting termination?
• How might we detect termination in a distributed
  parallel computation.

57
Election Algorithms

• Many distributed algorithms require one process
  to act as a coordinator
• Sometime we select the coordinator and setup
  master-slave configuration.
• At other times the coordinator is selected
  through a election algorithm.

58
Bully Algorithm

• The highest-numbered process always wins this
  election.
• P send an ELECTION message to all processes with
  higher numbers
• If no one responds, P wins the election and
  becomes coordinator
• If one of the higher-ups answers, it takes over.
  Ps job is done.

59
The Bully Algorithm (1)

• The bully election algorithm
• Process 4 holds an election
• Process 5 and 6 respond, telling 4 to stop
• Now 5 and 6 each hold an election

60
Bully Algorithm

• Process 6 tells 5 to stop
• Process 6 wins and tells everyone

61
Ring Algorithm

• This works by having processes send ELECTION
  messages to their successor. If the successor is
  down it is skipped.
• Each successor adds itself to the list and send
  the message on to its successor.
• When the message gets back to the originator of
  the ELECTION, a second COORDINATOR message is
  sent out. The process with the highest number on
  the list in this message is the coordinator.

62
A Ring Algorithm

• Election algorithm using a ring.

63
Mutual Exclusion

• Controlling access to critical regions in a
  distributed system is harder to do than in a
  uniprocessor or shared memory system.
• We are familiar with the use of semaphores and
  mutex on shared memory systems, what about a
  distributed system?

64
Centralized Algorithm

• This simulates a uniprocessor system.
• One process is elected coordinator.
• Whenever a process wants access to a critical
  region is requests access from the coordinator.
• The coordinator can choose not to reply for a
  request as a method of indicating the critical
  region has already been accessed by another
  process. Alternatively the coordinator can issue
  a permission denied message.

65
Mutual Exclusion A Centralized Algorithm

• Process 1 asks the coordinator for permission to
  enter a critical region. Permission is granted
• Process 2 then asks permission to enter the same
  critical region. The coordinator does not reply.
• When process 1 exits the critical region, it
  tells the coordinator, when then replies to 2

66
Centralize Algorithm

• What issue might be a problem with this method?

67
Centralized Algorithm

• Single point of failure maybe?
• If a process blocks after making a request, is
  this a permission denied situation or is a
  system down?
• How many messages does it take to gain access to
  a critical region?

68
Centralized Algorithm

• 3 messages
• Request to gain access
• Acknowledgement
• Message back to coordinator releasing critical
  region

69
Distributed Algorithm

• For this algorithm when a process wants access to
  a critical region is builds a message and sends
  it to all other processes.

70
Distributed Algorithm

• When a process receives this message the action
  it takes depended on its state.There are 3 cases.
• If the receiver is not in the critical region and
  does not want to enter this region its sends back
  an OK message
• If the receiver is in the critical region it does
  not reply. It queues the request.
• If it wants the critical region it does a
  priority check by examining the timestamps. The
  lowest timestamped message wins.

71
A Distributed Algorithm

• Two processes want to enter the same critical
  region at the same moment.
• Process 0 has the lowest timestamp, so it wins.
• When process 0 is done, it sends an OK also, so 2
  can now enter the critical region.

72
Distributed Algorithm

• How many messages are required to gain access?

73
Distributed Algorithm

• One message to each process requesting access and
  an OK message back from each process or 2(n-1)
  messages.
• Is this algorithm an improvement over the
  centralized algorithm with respect to points of
  failure?

74
Token Ring Algorithm

• A logical ring is constructed.
• A token is passed around this ring.
• A token is required to enter a critical region.
• If a process gets the token and does not need to
  enter the critical region it passes it on.
• If a token is lost it is hard to regenerate since
  detecting its lose is difficult. Maybe its taking
  a long time to get a round the ring.

75
A Toke Ring Algorithm

• An unordered group of processes on a network.
• A logical ring constructed in software.

76
Comparison

• A comparison of three mutual exclusion algorithms.

77
Distributed Transactions

• Like mutual exclusion algorithms transactions
  also protect shared resources from simultaneous
  access from concurrent processes.
• Additional transactions allows a process to
  access and modify multiple data items as a single
  atomic operation.
• Any operation that is suspended before completion
  results in everything restored to the point
  before the transaction started.

78
The Transaction Model (1)

• Updating a master tape is fault tolerant. If
  anything goes wrong during the update the tapes
  are all rewound and started over. This behavior
  is similar to the all-or-nothing property of
  transactions

79
Programming Transactions

• Special primitives are required at a minimum
  level.
• Either a programming language or the underlying
  system supplies them.
• The exact list of primitives available depend on
  the kinds of objects that are being used.

80
The Transaction Model (2)

• Examples of primitives for transactions.

81
Example

• Processing of reserving an airline seat from
  while plains ny, to malindi, kenya.
• There are 3 flights in this trip therefore 3
  possible operations to obtain a seat on each leg
  of the trip.

82
The Transaction Model (3)

• Transaction to reserve three flights commits
• Transaction aborts when third flight is
  unavailable

83
Properties of Transactions

• We have already seen the all or nothing property,
  in addition
• Atomic
• Consistent
• Isolated
• Durable

84
Properties

• Atomic, transactions either happen completely or
  not at all.
• Consistent, if certain conditions must hold
  before a transaction, they also hold after a
  transaction.
• I.e. conservation of money in banks, transferring
  money from your checking to your savings account
  does not change the amount of money a bank has
  either before or after the transaction.

85
Properties

• Isolated, Transactions are serializable, that
  means that 2 or more transactions running at the
  same time yields the same result as if they all
  ran in sequence.
• Durable, once a transaction commits the
  transaction goes ahead and results are permanent.

86
Classification of Transactions

• The type of transaction that holds to these
  properties is called a flat transaction. This
  type of transaction has some limitations.
• Partial results are not allowed.
• I.e. 2 legs of a trip are booked while attempts
  to book the 3rd leg continue.

87
Nested Transaction

• Constructed from a number of sub-transactions.
• This type of transaction must be able to undo a
  commit made by one of the sub-transactions if the
  parent aborts.

88
Distributed Transaction

• A distributed transaction allows for operations
  on distributed data.
• This is different than what a nested transaction
  does. If a nested transaction has a
  sub-transaction that carries out one operation
  (booking a trip leg) that requires access to
  distributed data (2 operations) it cannot do so.

89
Distributed Transactions

• A nested transaction
• A distributed transaction

90
Implementation

• How are transaction implemented?

91
Implementation

• The text uses a file system and transaction on it
  to describe implementation.

92
Private Workspace

• Transactions can operate on private workspace
  thus protecting the original files from change
  until a commit takes place.
• Rather than copying all the files in a private
  workspace an assumption is made that read-only
  access is initially required, therefore the
  private workspace consists of file pointers back
  to the actual files.
• Only when a write occurs is the file copied into
  the private workspace. To optimize this operation
  only blocks that undergo change are actual copied.

93
Private Workspace

• The file index and disk blocks for a three-block
  file
• The situation after a transaction has modified
  block 0 and appended block 3
• After committing

94
Writeahead Log

• This is another implementation method.
• In this method a log is kept of old setting and
  new settings as a result of changes to the file.
• Once a commit is made the changes become
  permanent.
• If the transaction is aborted the old settings
  are restored form the data stored in the log.
  This action is called rollback.

95
Writeahead Log

• a) A transaction
• b) d) The log before each statement is executed

96
Concurrency Control

• Transactions should be allowed to execute
  concurrently
• Data should be left in a consistent state after
  the concurrent transactions have committed.
• Consistency is maintained if the concurrent
  transaction execute in a specific order and the
  final result is the same a as if they executed in
  a sequential order.

97
Concurrency Control

• Assume three different layered managers in a
  system that enforces concurrency control.
• Data Manager - does all read and writes on data
• Scheduler - controls the passing of read and
  writes from transactions to the data manager.
  Uses Locks or timestamps.
• Transaction Manager - Guarantees atomicity of
  transactions.

98
Concurrency Control (1)

• General organization of managers for handling
  transactions.

99
Concurrency Control (2)

• General organization of managers for handling
  distributed transactions.

100
Serializability

• It is important that the scheduler schedule the
  transactions in such a way that shared data is
  kept consistent.
• In the next slide schedules 1 and 2 are legal
  because the value of x could be obtained by
  executing the transaction sequentially, schedule
  3 is illegal because the interleaving results in
  a value for x that cannot e obtained by
  sequential processing of transaction.

101
Serializability
(d)

• a) c) Three transactions T1, T2, and T3
• d) Possible schedules

102
Locks

• A simple serializing mechanism is the use of
  exclusive locks.
• In the locking scheme the server attempts to lock
  any object that is about to be used by any
  operation of the clients transaction.
• If a lock is already on the object the requesting
  transaction is suspended until the lock is
  released.

103
Locks
104
Two-Phase Locking

• One way to handle access to shared data is to
  implement a scheduler that can grant locks if
  shared data is accessed.
• The scheduler grants and releases locks as
  needed. To do so an algorithm is needed so as to
  preserve data consistency by allowing only
  serializable schedules.
• Two-Phase locking is one such algorithm

105
Two-Phase Locking Algorithm

• Locks are granted to transactions if they do not
  conflict with locks already granted. Otherwise a
  delay takes place.
• Locks are not released until the operation it was
  granted for takes place.
• A given transaction can only have one lock, a
  request for another lock is a programming error.
• Eswaran et al 1976 proved that if all
  transactions use two-phase locking all schedules
  formed by interleaving them are serializable.

106
Two-Phase Locking (1)

• Two-phase locking.

107
Deadlock

• The use of locks can lead to deadlock.
• If transaction T locks account A and transaction
  U locks account B and T needs access to B before
  it releases A and U needs access to A before it
  releases B then deadlock occurs.
• Deadlock detection is required to find cycles
  like this in the wait-for graph that is built of
  transactions waiting for access.

108
Wait-For Graph
Held by
Waits for
A
T
U
B
Waits for
Held by
Deadlock cycle from previous example Timeouts can
also be used on locks as a way of handling
deadlocks
109
Timestamps

• Instead of locks a timestamp system can be
  implemented.
• This system uses Lamports algorithm
• There is a timestamp associated with a
  transaction
• Operations timestamp
• And timestamps associated with the data item
• Read timestamp
• Write timestamp

110
Timestamp Ordering

• A transactions request to write an object is
  valid only if that object was last read and
  written by earlier transactions. A transactions
  request to read an object is valid only if that
  object was last written by an earlier transaction.

111
Timestamps

• Operation conflicts for timestamp ordering