CS556: Distributed Systems

1
CS-556 Distributed Systems
Inter-process Communication (II)

• Manolis Marazakis
• maraz_at_csd.uoc.gr

2
IPC

• Techniques
• shared data
• message passing
• Objectives
• data exchange
• synchronization
• Processes at different hosts, executing at
  different rates, need to influence the overall
  execution pattern
• constraints on the order of events

3
Communication Infrastructure
gateway
4
Client/Server Protocol Design

• Synchronization bet. client server
• Admissible request/response sequences
• QoS
• performance
• response times, throughput, timeliness
• dependent on CPU network ( scheduling)
• reliability
• adaptability
• Dependability
• fault tolerance
• security

5
UDP Failure Model

• Omission failures
• timeouts
• duplicate messages
• lost messages
• Need to maintain history
• Last reply sent to each client
• provided that a client can make only one request
  at a time
• interprets each request as the ACK for the
  previous reply
• periodic purge of history
• No ACK for the last response received before
  client terminates
• Fixed max. buffer size (8 KB)
• No message order guarantee
• Process crash failures

6
TCP Failure Model

• Reliable message delivery
• checksums, sequence numbers, timeouts
• no need for applications to deal with
• retransmissions
• duplicates
• reordering
• no need for histories
• Flow control mechanism
• large transfers without overwhelming the receiver
• BUT not reliable sessions
• Connections may be severed or severely congested
• Processes cannot distinguish network from process
  failure
• Processes cannot tell if their recent messages
  were received

7
HTTP

• Methods
• GET, HEAD, POST
• PUT, DELETE, TRACE, OPTIONS

• Resource MIME-encoded data
• Content negotiation
• Authentication

8
Group Communication

• Multicasting 1-to-many comm. pattern
• Applications
• replicated services (better fault tolerance)
• discovery of services
• replicated data (better performance)
• propagation of event notifications
• Failure model
• depends on implementation
• IP multicast (UDP datagrams) omission failures
• class-D Inet addresses 1110 bit prefix
• TTL
• reliable multicast
• ordered multicast
• FIFO
• Causal
• Total

9
Conventional Procedure Call

• Parameter passing in a local procedure call the
  stack before the call to read
• The stack while the called procedure is active

10
Software layers
RPC is more than a (transport) protocol a
structuring mechanism for distributed systems
11
Steps of a Remote Procedure Call

• Client procedure calls client stub in normal way
• Client stub builds message, calls local OS
• Client's OS sends message to remote OS
• Remote OS gives message to server stub
• Server stub unpacks parameters, calls server
• Server does work, returns result to the stub
• Server stub packs it in message, calls local OS
• Server's OS sends message to client's OS
• Client's OS gives message to client stub
• Stub unpacks result, returns to client

12
Example (Sun RPC - ONC)

• long square(long) example
• client ren.eecis.udel.edu 11
• result 121
• Need RPC specification file (square.x) defines
  procedure name, arguments results
• Run rpcgen square.x generates square.h,
  square_clnt.c, square_xdr.c, square_svc.c
• square_clnt.c square_svc.c Stub routines for
  client server
• square_xdr.c XDR (External Data Representation)
  code - takes care of data type conversions

13
Client and Server Stubs

• Principle of RPC between a client server
  program.

14
RPC Specification File (square.x)
struct square_in long arg1 struct
square_out long res1 program SQUARE_PROG
version SQUARE_VERS square_out
SQUAREPROC(square_in) 1 // procedure
1 // version 0x321230000 // program
IDL Interface Definition Language
15
Parameter Specification Stub Generation
procedure
Corresponding message
16
Writing a Client a Server

• The steps in writing a client a server in DCE
  RPC.

17
Binding (SUN RPC)

• Port Mapper (rpcbind) listens at UDP port 111
• Server registers program ID version
• rpcinfo -p -gt display all registered RPC servers
• When client issues clnt_create, the port mapper
  is contacted
• program-to-port number mapping
• arguments (program ID, version, protocol)
• response servers port number

18
Binding (DCE)
19
Passing Value Parameters (I)
20
Passing Value Parameters (II)

• a. Original message on Pentium (little-endian)
• b. The message after receipt on SPARC
  (big-endian)
• c. The message after being inverted.

21
Passing Value Parameters (III)

• How to pass pointers ?
• Meaningful only within a specific address space !
• Arrays (of known length) structures
• Copy/restore semantics (bet. stubs)
• IN/OUT/INOUT markers
• Optimization may eliminate one copy operation
• Pointer to an arbitrary data structure ?
• No general solution
• Work-around
• Pass back the pointer to its source

22
External Data Representation (I)

• Data structures
• flattened on transmission
• rebuilt upon reception
• Primitive data types
• byte order (big-endian MSB comes first)
• ASCII vs UNICODE (2 bytes per character)
• marshalling/unmarshalling
• to/from agreed external format

23
External Data Representation (II)

• XDR (RFC 1832), CDR (CORBA), Java
• data -gt byte stream
• object references
• HTTP/MIME
• data -gt ASCII text

24
CORBA CDR example
25
Which Transport Protocol to Use With RPC?

• TCP provides reliable transfers
• ACKs, timeouts, retransmissions
• UDP does not provide reliability, but RPC
  implements a RETRANSMIT timeout
• Both TCP UDP repeat request after total timeout
• TCP requires connection establishment
• significant overhead for a few short RPCs

26
RPC Implementation (I)

• A.D. Birrell and B.J. Nelson, Implementing
  remote procedure calls, ACM Trans. Computer
  Systems, 2(1), pp. 39-59, 1984.
• Distinction bet. input and output parameters
• Call-by-value parameters
• call-by-reference ?
• Language-dependent vs Language-independent
• Interface Definition Language (IDL)
• Choice of network transport protocol ?
• Semantics in error/failure conditions ?
• Binding mechanism
• Data integrity authentication

27
RPC Implementation (II)
28
RPC Implementation (III)

• Messaging protocol
• must handle undesirable network
  properties/behaviors
• 3 distinct types of functionality
• fragment/reassemble messages of arbitrary size
• Sender does not block waiting for ACKs
• Receiver issues selective retransmission requests
• request/reply correlation
• At-Most-Once semantics logical channel bet.
  processes
• RETRANSMIT timer, at both client server
• PROBE timer at client
• dispatch of requests to procedures
  (de-multiplexing)
• procedure space (eg 16-bit selector field)
• concurrent invocations
• Programming language compiler support
• pack/unpack arguments results, binding

29
How Simple is RPC ?

• The main objective of RPC was to provide
  simplicity in distributed computing
• Sun RPC 164 runtime functions (!)
• Sockets 25 functions
• Semaphores 10 functions
• Posix threads 15 functions
• Cond-vars 10 functions

30
RPC Protocols (I)

• A.Z. Spector Performing Remote Operations
  Efficiently on a Local Computer Network, CACM,
  25(4), pp. 246-260, 1982
• Request
• Request-Reply
• Servers reply is regarded as ACK of previous
  request
• Clients request is regarded as ACK of previous
  response
• Request-Reply-Acknowledge
• request IDs ACKs allow server to discard from
  its history all responses with lower IDs
• An optimization Piggy-backing ACKs on requests
• eg Client uses local timer to schedule an ACK
  after a fixed period, new requests cancel any
  pending ACK transmissions

31
RPC Protocols (II)

• RPC semantics
• Zero-or-More (Maybe)
• At-Least-Once
• Keep requesting RPC until valid response arrives
  at client
• At-Most-Once
• No reply may mean that no execution took place
• Exactly-One
• Idempotent vs non-idempotent procedures
• Require transaction processing techniques

32
Doors

• RPC for co-located server client

include ltdoors.hgt
33
Asynchronous RPC (I)

• The interconnection between client and server in
  a traditional RPC
• The interaction using asynchronous RPC

34
Asynchronous RPC (II)

• A client server interacting through 2
  asynchronous RPCs

35
Distributed Objects (I)

• Common organization of a remote object with
  client-side proxy.

36
Distributed Objects (II)

• Compile-time (language-level) objects
• Dependency on a particular language
• but developer is more-or-less unaware of
  distribution
• Object adapters (wrappers)
• Run-time objects
• Require registration
• Accept invocation requests
• Allow for objects written in different
  languages
• Persistent vs Transient objects
• Static vs Dynamic Invocation
• App. Scenarios
• Object browser, Batch Processing
• System-wide object references
• Enhanced transparency of distribution

37
Distributed Objects (III)

• Implementation of object references
• Minimum (network address, end-point)
• BUT if server crashes all references become
  invalid
• Keep track of server-endpoint assignments
• Location daemon per machine
• BUT the server cannot be moved to another
  machine !
• Location server
• References contain location servers network
  address and a system-wide ID for the server
• Scalability issues ?

• Implicit assumptions
• Client server use the same protocol stack
• Transport, marshalling/unmarshalling
• Initial connection set-up, error handling, flow
  control,

Implementation Handle as part of an object
reference -gt refers to an implementation of a
proxy that can be loaded by the client
38
Binding a Client to an Object
Distr_object obj_ref //Declare a systemwide
object referenceobj_ref // Initialize the
reference to a distributed objectobj_ref-gt
do_something() // Implicitly bind and invoke a
method (a) Distr_object objPref //Declare a
systemwide object referenceLocal_object
obj_ptr //Declare a pointer to local
objectsobj_ref //Initialize the reference
to a distributed objectobj_ptr
bind(obj_ref) //Explicitly bind and obtain a
pointer to the local proxyobj_ptr -gt
do_something() //Invoke a method on the local
proxy (b)

• Example with implicit binding using only global
  references
• Example with explicit binding using global and
  local references

39
Parameter Passing

• The situation when passing an object by reference
  or by value.

Local vs remote object references
40
The DCE Distributed-Object Model

• Distributed dynamic objects in DCE.
• Distributed named objects