CENG 532 Distributed Computing Systems

1
CENG 532- Distributed Computing Systems

• Measures of Performance

2
Groschs Law-1960s

• To sell a computer twice as much, it must be
  four times as fast
• It was Ok at the time, but soon it became
  meaningless
• After 1970, it was possible to make faster
  computers and sell even cheaper.
• Ultimately the switching speeds rach a limit, the
  speed of the light on an integrated circuit

3
Von Neumanns Bottleneck

• Serial single processor computer architectures
  followed John Von Neumanns architecture of
  1940-1950.
• One processor, single control unit, single memory
• This is no more valid Low cost parallel
  computers can easily deliver the performance of
  the fastest single processor computer

4
Amdahls Law 1967

• Let speedup (S) be ratio of serial time (one
  processor) to parallel time (N processors)
• ST1/TN lt 1/f
• Where f is the serial fraction of the problem,
  1-f is the parallel fraction of the problem, then
• Tn T1fT1(1-f)/N
• S1/(f(1-f)/N), thus slt1/f

5
Amdahls Law 1967

• At f0.10, Amdahl Law predicts, at best a
  tenfold speedup, which is very pessimistic
• This was soon broken, encouraged by Gordon Bell
  Prize!

6
Gustafson-Barsis Law 1988

• The team of researchers of Sandia Labs (John
  Gustafson and Ed Barsis) , using 1024 processor
  nCube/10, overthrew Amdahls Law, by achieving
  1000 fold speedup with f-0.004 to 0.008.
• According to Amdahls Law, the speedup would have
  been from 125 to 250.
• The key point was that 1-f was not independent of
  N.

7
Gustafson-Barsis Law 1988

• They interpreted the speedup formula, by scaling
  up the problem to fit the parallel machine
• T1f(1-f)N
• TNf(1-f)1, then the speedup can be computes as
• SN-(N-1)f

8
Extreme case analysis

• Assuming Amdahls Law, an upper and lower bound
  can be given for the speedup, under unrealistic
  assumptions
• N/log2N lt S lt N
• where N is based on single processor logN is
  based on divide and conquer

9
Inclusion of the communication time

• Some researchers (Gelenbe) suggests speedup to be
  approximated by
• S1/C(N) where C(N) is some function of N
• For example, C(N) can be estimated as
  C(N)ABlog2N
• where A and B are constants determined by the
  communication mechanisms

10
Benchmark Performance

• Benchmark is a program whose purpose is to
  measure a performance characteristic of a
  computer system, such as floating point speed,
  I/O speed, or for a restricted class of problems
• The benchmarks are arranged to be either
• Kernels of real applications, such as Linpacks,
  Livermore Loops, or
• Synthetic, approximating the behavior of the real
  problem, such as Whetstone and Wichmann