1. Introduction to Distributed Systems

1
1. Introduction to Distributed Systems
2
1. Introduction

• Two advances in computer technology
• A. The development of powerful microprocessors.
• B. The invention of high-speed computer networks.

3

• The advances make it possible to put together
  computing systems composed of large numbers of
  CPUs connected by a high-speed network.
• They are called distributed systems.

4
What is a distributed system?

• A distributed system is a collection of
  independent computers that appear to the users of
  the system as a single computer.
• Eg. 1 a network of workstations in a university
  or company department.
• Eg. 2 a factory full of robots, each containing a
  powerful computer for handling vision, planning,
  communication, and other tasks.
• Eg. 3 a large bank with hundreds of branch
  offices all over the world.

5
What is a centralized system?

• A centralized system (or a single-processor
  system) consists of a single CPU, its memory,
  peripherals, and some terminals.

6
Advantages of Distributed Systems over
Centralized Systems
7
Advantages of Distributed Systems over
Independent PCs
8
Disadvantages of Distributed Systems
9
Multiple CPU computer system categories

• SISD a computer with a single instruction stream
  and a single data stream.
• e.g. all traditional uniprocessor computers
  (those having only one CPU), from personal
  computers to large mainframes.
• SIMD single instruction, multiple data stream.
• e.g. array processors with one instruction
  unit that fetches an instruction, and then
  commands many data units to carry it out in
  parallel, each with its own data.

10

• MISD multiple instruction stream, single data
  stream.
• e.g. no known computers fit this model.
• MIMD multiple instruction stream, multiple data
  stream.
• e.g. a group of independent computers, each
  with its own program counter, program, and data.

11
Hardware
MIMD
Parallel and distributed computers

Tightly coupled
Loosely coupled
Multiprocessors (shared memory)
Multicomputers (private memory)
Switched
Switched
Bus
Bus
Sequent, Encore
Hypercube, Transputer
Ultracomputer, RP3
Workstations on a LAN
12

• Tightly-coupled system
• the delay experienced when a message is sent
  from one computer to another is short, the data
  rate is high.
• Loosely-coupled system
• the delay is large and the data rate is low.

13
Bus-Based Multiprocessors
CPU
CPU
CPU
Memory
Cache
Cache
Cache
Bus
14
Drawback

• As few as 4 or 5 CPUs, the bus will be
  overloaded.
• Solution cache memory.
• Problem with cache incoherent.
• Solution to incoherent snoopy write through
  cache.

15

• Write through whenever a word is written to the
  cache, it is written through to memory as well.
• Snoopy cache a cache is always snooping on the
  bus. Whenever it sees a write occurring to a
  memory address present in its cache, it either
  invalids the entry or updates it.

16
Switched Multiprocessors-crossbar switch
Memories
M
M
M
M
CPUs
C
Crosspoint switch
C
C
C

17
Cross switch

• Advantage many CPUs can be accessing memory at
  the same time.
• Disadvantage need N2 crosspoint switches with n
  CPUs and n memories.
• One solution use omega network.

18
Switched Multiprocessors-omega switch
2 X 2 switch
C
M
C
M
M
C
M
C
An omega switching network
19
Omega network

• With n CPUs and n memories, it requires log2n
  switching stages, each containing n/2 switches,
  for a total of (n log2n)/2 switches.
• Better than crossbar switch, but still big.
• Another drawback delay (due to the number of
  stages).

20
NUMA machine

• To reduce delay, use NUMA (NonUniform Memory
  Access)
• Some memory is associate with each CPU. Each CPU
  can access its own local memory quickly, but
  accessing anybody elses memory is slower.

21
NUMA machine

• Advantage have better average access times than
  omega machines
• Disadvantage the placement of the programs and
  data becomes complicated because most access
  should go to the local memory.

22
Bus-Based Multicomputers
Workstation
Workstation
Workstation
Local memory
Local memory
Local memory
CPU
CPU
CPU
Network
A multicomputer consisting of workstations on a
LAN
23
Switched Multicomputers- Grid
24
Switched Multicomputers-Hypercube
25
Loosely-coupled software on loosely-coupled
hardware

• Network Operating Systems A network of
  workstations connected by a LAN.

26
Tightly-coupled software on loosely-coupled
hardware

• True distributed systems It is tightly-coupled
  software on the same loosely-coupled
  (multicomputer) hardware.

27
Characteristics of a distributed systems

• There must be a single, global interprocess
  communication mechanism so that any process can
  talk to any other process.
• Process management must also be the same
  everywhere.
• The file system must look the same everywhere,
  too.
• It is normal that identical kernels run on all
  the CPUs in the system.
• Each kernel can have considerable control over
  its own local resources.

28
Tightly-coupled software on tightly-coupled
hardware

• Multiprocessor Timesharing Systems
• The key characteristic of this class of system is
  the existence of a single run queue a list of
  all the processes in the system that are
  logically unblocked and ready to run.

29
Comparison
30
Design Issues-Transparency
31
Design Issue - Flexibility

• one school maintains that each machine should run
  a traditional kernel that provides most services
  itself.
• The other maintains that the kernel should
  provide as little as possible, with the bulk of
  the operating system services available from
  user-level servers.

32
Design Issue - Reliability

• High availability
• Security
• Fault tolerance

33
Design Issue - performance

• Fine-grained parallelism jobs that involve a
  large number of small computations, especially
  ones that interact highly with one another, may
  cause trouble on a distributed system with
  relatively slow computation.
• Coarse-grained parallelism jobs that involve
  large computations, low interaction rates, and
  little data.

34
Design Issue - Scalability

• Potential bottlenecks that designers should try
  to avoid in very large distributed systems.