Linear Time Byzantine SelfStabilizing Clock Synchronization

1
Linear Time Byzantine Self-Stabilizing Clock
Synchronization

• Ariel Daliot1, Danny Dolev1, Hanna Parnas2
• 1School of Engineering and Computer Science,
• 2Department of Neurobiology and the Otto Loewi
  Center for Cellular and Molecular Neurobiology,
  The Hebrew University of Jerusalem, Israel
• This research is supported in part by Intel
• COMM Grant - Internet Network/Transport Layer
  QoS Environment
• (IXA)

2
Lecture Outline

• What is Pulse Synchronization
• Examples of pulse synchronization in nature
• A biologically inspired pulse synchronization
  algorithm for distributed computer networks
• Efficient Byzantine Self-Stabilizing clock
  synchronization above pulse synchronization

3
The target is to synchronize pulses from any
state and any faults
cycle

• ...............................................
  .................................
• .............................................
  ..................................
• ...............................................
  ................................... t
• ..........................................
  .......................................
• ..............................................
  ....................................
• ..........................................
  ...................................
• .............................................
  ......................................
• ...................................
  ............................
• ....................................
  ..............

Arbitrary state
Synchronized state
4
Self-Stabilizing Pulse Synchronization

• Convergence Starting from an arbitrary state s,
  the system reaches a synchronized state in finite
  time
• Closure If s is a synchronized state of the
  system at real-time t0 then ? real-time t t0
• The system state at time t is a synchronized
  state
• Linear Envelope, for every correct node
  pat-t0 b ? ?p(t, t0) ? g t-t0 h
• ?p(t1,t2) is is the number of pulses a correct
  node pi invoked during a real time interval
  t1,t2 within which pi was continuously correct

5
Pulse Synchronization, in distributed computer
systems

• The computers are required to
• Invoke regular pulses in tight synchrony
• Synchronize from ANY state (self-stabilization)
• Limit the pulse frequency
• Tolerate a number of node and network faults
• Examples of other synchronization problems
• Firing Squad, Clock Synchronization, etc.

6
Fault Models

• Many problems trivial with no faults, some
  unsolvable with a single fault (E.g. Byzantine
  Generals)
• Common fault models Crash/Link/Message faults
• Byzantine failures (malicious faults)
• Usually proven to require n3f to tolerate f
  faults
• Not solvable for some problems
• Transient faults (system in arbitrary state or
  total chaos)
• Requires Self-Stabilizing algorithms in order to
  overcome
• Not solvable for some problems (Clock
  Synchronization)

7
Self-Stabilization

• Addresses the situation when ALL nodes can
  concurrently be faulty for a limited period of
  time
• A SS algorithm realizes its task once the system
  is back within the assumption boundaries
• Is orthogonal to Byzantine failures, i.e. these
  are uncorrelated fault models
• Byzantine algorithms typically focus on limiting
  the influence of faulty nodes once the task has
  been realized
• Self-stabilizing algorithms focus on realizing
  the task following a catastrophic state

8
Synchrony phenomena in biology

• The phenomenon of synchronization is displayed by
  many biological systems
• Synchronized flashing of the male malaccae
  fireflies
• Oscillations of the neurons in the circadian
  pacemaker, determining the day-night rhythm
• Crickets that chirp in unison
• Coordinated mass spawning in corals
• Audience clapping together after a good
  performance
• We were inspired by the pacemaker network in the
  cardiac ganglion of lobsters

9
Synchrony phenomena in biology

• The phenomenon of synchronization is displayed by
  many biological systems
• Synchronized flashing of the male malaccae
  fireflies
• Oscillations of the neurons in the circadian
  pacemaker, determining the day-night rhythm
• Crickets that chirp in unison
• Coordinated mass spawning in corals
• Audience clapping together after a good
  performance
• We were inspired by the pacemaker network in the
  cardiac ganglion of lobsters

10
Synchrony phenomena in biology

• The phenomenon of synchronization is displayed by
  many biological systems
• Synchronized flashing of the male malaccae
  fireflies
• Oscillations of the neurons in the circadian
  pacemaker, determining the day-night rhythm
• Crickets that chirp in unison
• Coordinated mass spawning in corals
• Audience clapping together after a good
  performance
• We were inspired by the pacemaker network in the
  cardiac ganglion of lobsters

11
Synchrony phenomena in biology

• The phenomenon of synchronization is displayed by
  many biological systems
• Synchronized flashing of the male malaccae
  fireflies
• Oscillations of the neurons in the circadian
  pacemaker, determining the day-night rhythm
• Crickets that chirp in unison
• Coordinated mass spawning in corals
• Audience clapping together after a good
  performance
• We were inspired by the pacemaker network in the
  cardiac ganglion of lobsters

12
Synchrony phenomena in biology

• The phenomenon of synchronization is displayed by
  many biological systems
• Synchronized flashing of the male malaccae
  fireflies
• Oscillations of the neurons in the circadian
  pacemaker, determining the day-night rhythm
• Crickets that chirp in unison
• Coordinated mass spawning in corals
• Audience clapping together after a good
  performance
• We were inspired by the pacemaker network in the
  cardiac ganglion of lobsters

13
Synchrony phenomena in biology

• The phenomenon of synchronization is displayed by
  many biological systems
• Synchronized flashing of the male malaccae
  fireflies
• Oscillations of the neurons in the circadian
  pacemaker, determining the day-night rhythm
• Crickets that chirp in unison
• Coordinated mass spawning in corals
• Audience clapping together after a good
  performance
• We were inspired by the pacemaker network in the
  cardiac ganglion of lobsters

14
Synchrony phenomena in biology

• The phenomenon of synchronization is displayed by
  many biological systems
• Synchronized flashing of the male malaccae
  fireflies
• Oscillations of the neurons in the circadian
  pacemaker, determining the day-night rhythm
• Crickets that chirp in unison
• Coordinated mass spawning in corals
• Audience clapping together after a good
  performance
• We were inspired by the pacemaker network in the
  cardiac ganglion of lobsters

15
Synchrony phenomena in biology

• Synchronization typically attained in spite of
• Inherent difference among the biological elements
  
• High levels of noise from external sources or
  from the biological elements
• Displays high degree of Self-Stabilization

16
Cardiac ganglion of the lobster (Sivan, Dolev
Parnas, 2000)

• Four interneurons tightly synchronize their
  pulses in order to give the heart its optimal
  pulse rate (though one is enough for activation)
• Able to adjust the synchronized firing pace, up
  to a certain bound (e.g. while escaping a
  predator)

17
Cardiac ganglion of the lobster (Sivan, Dolev
Parnas, 2000)

• Must not fire out of synchrony for prolonged
  times in spite of
• Noise
• Single neuron death
• Inherent variations in the firing rate
• Firing frequency regulating Neurohormones
• Temperature change
• The vitality of the cardiac ganglion suggests it
  has evolved to be optimized for
• Fault tolerance
• Re-synchronization from any state
  (self-stabilization)
• Tight synchronization
• Fast re-synchronization

18

• We present a Byzantine self-stabilizing pulse
  synchronization algorithm which works similarly
  to the cardiac pacemaker
• Input to the algorithm n, f (f
• Output of the algorithm tells the node when to
  invoke a pulse
• The algorithm is proven to solve the pulse
  synchronization problem

19
The neurobiological principles borrowed

• Endogenous Periodic Spiking
• Time dependent Refractory Function, R
• Summation
• Nodes communicate with Messages which are then
  Summated to generate the Counter that is sent in
  the message and compared to the Refractory
  Function

20
Pulse Synchronization Algorithm
21
Algorithm Analysis and Complexity
log(n) cycles
½-1 cycle

• ...............................................
  ............................................
• .............................................
  ......................................
• ...............................................
  ............................................
  t
• ..........................................
  ..............................
• ..............................................
  .........................................
• ..........................................
  .............................
• .............................................
  ......................................
• ...................................
  ............................
• ....................................
  ...........

f
tightness d (near optimal)
arbitrary initial state
22
Algorithm Complexity

• Tolerates f concurrent Byzantine faults, f
• Self-stabilizing, synchronizes from an arbitrary
  state
• Faults can cause the synchronized pulse frequency
  to be up to twice the original frequency
• Synchronization is VERY tight, d time units,
  theoretical bound is
• The best non-stabilizing Byzantine CS reaches d
  time units
• Self-Stabilizing Byzantine CS reaches (n-f).d
• Time complexity is O(logn)
• Best Self-stabilizing Byzantine Digital Clock
  Synchronization converges in expected
  (n-f).n6(n-f) pulses
• Non-stabilizing Byzantine Clock Synchronization
  use O(f)-O(n2) time

23
A related problem real-time-Clock
Synchronization (rCS)

• There exists ?, t0 , ?, a and b such that?tt0
• Agreement. For any correct nodes p, q Cp(t) -
  Cq(t) ?, (precision)
• Validity. For every correct node p (1?)-1t a
  Cp(t) (1?)t b, (accuracy)Optimal precision
  is d.(1-1/n)Optimal accuracy is ? ?

24
real-time-Clock Synchronization

• rCS has two additional constraints over pulse
  synchronization
• The pulses have labels (the time)
• The time needs to approximate real time
• Most Byzantine rCS use the following principles
• At every time the computers exchange clock values
• They operate some function on the received values
  (which seeks to neutralize the effect of the
  Byzantine values and set the clocks close to each
  other)

25
Clock Synchronizationsimple example

• All computers exchange clocks at time X
• They must reach time X at roughly the same time
• Assume some computer receives the values
• 435, 2, 2, 3, 2, 935, 3, 3, 2, 2, 6984
• Throw away the values that look bad
• Do some statistical function on what is left and
  set the clock accordingly
• It can be proven this leaves all the computers
  with very close clock values that are good
  estimates of real time

26
real-time-Clock Synchronization impossibility
result with no external time source

• This works only if the clocks initially have
  close values
• Which implies rCS cannot be solved when all
  clocks hold arbitrary times
• Which means there is no self-stabilizing
  algorithm for rCS
• I.e. if the clocks are initially far apart they
  cannot both synchronize AND estimate real time
• Internal rCS assume clocks are initially
  synched

27
FAB8 AIT WSR - ww16-17/2003

• Main outages and issues .
• synchronize problem in VAX - On WW16.5 -
  Impact 8 CW SC's unable to introduce lots for a
  period of 4 hr and 15 min.( from 2300 until
  0315). Root cause - the job which synchronize
  the time between the VAXes failed on 2200
  (Thursday Night) and created gaps between the
  machines clocks. This gap caused the remotes
  which worked with CW to get to loop status, with
  error message of FCM message. Solution Time
  synchronized. Helpdesk will get alerts when this
  job will fail again.

28
A Distributed System according to Lamport

• A distributed system is one in which the failure
  of a computer you didn't even know existed can
  render your own computer unusable. 

29
Applicability of logical clocks

• Many algorithms depending on clock
  synchronization actually only require
  synchronized logical clocks
• E.g. TDMA, Kerberos tickets, DHCP leases, global
  snapshots, data base time stamps and many others
• Why then not use a self-stabilizing Byzantine
  clock synchronization algorithm that synchronizes
  logical time?

30
Self-Stabilizing Byzantine Clock Synchronization

• Because the known previously best
  self-stabilizing Byzantine clock synchronization
  algorithm converges in expected (n-f).n6(n-f)
  time! (Dolev-Welch, 95)
• The difficulty lies in the fact that
• the initial clock values can differ arbitrarily
• there is no agreed time for exchanging the values
  and setting the clock according to the values
  received
• the clocks can wrap around

31
Clock Synchronization using synchronized pulses

• We assume no outside source of real-time
• At every pulse exchange clock values and operate
  some clock adjustment function on the received
  multiset
• If clocks were initially close to real time then
  they will stay close to real time
• If not then the clocks will proceed synchronously
  close to logical time
• This scheme yields a Byzantine self-stabilizing
  clock synchronization algorithm with convergence
  time, accuracy and precision on the order of
  existing rCS

32
The Byzantine Self-Stabilizing Clock
Synchronization Algorithm

• At pulse event
• Begin
• Clock ET
• Wait for every correct node to invoke a pulse
• ET SS-Strong-Byz-Agreement(ET cycle mod M)
• End
• ET - Expected Time of next pulse
• Cycle - Expected elapsed logical time until next
  pulse
• M - Bound on clock value

33
The Byzantine Self-Stabilizing Pulse
Synchronization Algorithm

• if (cycle_countdown 0) then
• send Propose-Pulse message to all
• if (received f1 distinct Propose-Pulse
  messages) then send propose-Pulse message to
  all
• if (received n-f distinct Propose-Pulse
  messages) then
• invoke pulse event
• cycle_countdown cycle
• flush Propose-Pulse message counter
• ignore Propose-Pulse messages for 2d(1?)
  time

34
The Self-Stabilizing Byzantine Strong Agreement
Algorithm

• Any Strong Byzantine Agreement algorithm can be
  used
• Agreement and validity is not ensured until the
  pulses synchronize
• Self-stabilization is supported by counting
  recovering nodes as correct only following
  cycletime-for-BA of correct behavior
• We use a slightly modified version of the Toueg,
  Perry and Srikanth (1987) Strong Byzantine
  Agreement algorithm
• It has the advantage of early stopping if all
  correct nodes start with identical values then
  termination is within 2 rounds
• Hence, during continuous correct system behavior
  clock synchronization is maintained with very
  little overhead

35
global pulse Comparison chart
36
The teaching of Pythagoras

• Evolution is the law of life
• Unity is the law of God
• Number is the law of the universe

37

• Everything in the universe is subject to
  predictable progressive cycles.
• Pythagoras,

38
Future researchNoise tolerance of reciprocal
cyclic/oscillatory behavior

• Things are not static, events happen again and
  again
• Almost everything displays some degree of
  oscillatory behavior
• Examples earth rotation, the moon spins around
  its axis with exactly the same period as its
  period of rotation around the earth, yearly
  seasons, day-night, human metabolic cycle,
  electron rotation, moods, fashion, culture,
  hormone levels, animal population levels,
  menstruation period, virus epidemics, animal
  migration, trends in politics, stocks, etc, etc,
  
• The phenomena have a reciprocal effect on each
  other
• What mechanisms prevent from entering a chaotic
  mess?
• Refractoriness can be used to explain this point
  in simple terms
• It is a previously unused element in non-linear
  dynamics

39
Questions?
?
?
?
?
?
?
40
Synchronization is rewarding
41
(No Transcript)
42
Related Problems

• Digital Clock Synchronization
• Agreement on pulse counters, with or without a
  global pulse
• Clock Synchronization
• Common notion of real time, high precision and
  accuracy
• Phase Clocks
• Agreement on pulse counters in asynchronous
  settings
• Synchronized Rates
• Clocks progress at approximately the same rate,
  the times may differ
• Firing Squad
• All nodes enter the same state in step k after a
  process has initialized fire
• Pulse Synchronization
• Precise synchronization of regular pulses, slack
  linear envelope accuracy

43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
Pulse Synchronization algorithm (n, f, cycle)
1. Send a Message consisting of the Counter
(Pulse)
2. Summate messages from the other computers
receivedwithin a time window and update the
Counter accordingly
3. If (Counter current Threshold) Then Goto
1Else Goto 2
48
Linear Envelope
49
The hunting of the Rk

• Given cycle, f and n, the solution to the whole
  problem is by solving the set of linear
  constraints on R

50
The hunting of the Rk

• Given cycle, f and n, the solution to the whole
  problem is by solving the set of linear
  constraints on R
• How many equations (partitions) are there?

51
The hunting of the Rk

• Given cycle, f and n, the solution to the whole
  problem is by solving the set of linear
  constraints on R
• How many equations (partitions) are there?
• Turns out there is no analytic solution to this
  question, (integer partitioning), the number is
  denoted P(k)

52
The hunting of the Rk

• Given cycle, f and n, the solution to the whole
  problem is by solving the set of linear
  constraints on R
• How many equations (partitions) are there?
• Turns out there is no analytic solution to this
  question, (integer partitioning), the number is
  denoted P(k)
• Asymptotically equalse.g.P(50)
  204226P(1000) 24061467864032622473692149727991

53
Still hunting Rk

• No computer can solve this exponential set of
  constraints or do Gaussian elimination
• Some other method for reaching a solution (if
  exists) is sought after