InterProcess Communication II

1
Inter-Process Communication II

• Reliable Vs. Unreliable Communication
• Marshalling and Unmarshalling
• Remote Procedure Call (RPC)
• Remote Method Invocation (RMI)

2

• Reliable Vs
• Unreliable Communication
• Any ack?
• Communication Overheads?

3
Unreliable Vs. Unreliable Messages

• Unreliable message is used to refer to a single
  message transmitted from sender to receiver
  without acknowledgement or retries
• The message may lose and sender does not notice
• e.g. UDP only makes its best effort to deliver
  a message
• Reliable message delivery may be constructed from
  an unreliable one by using ack
• Positive ack receivers send ack message whenever
  a message is received
• Negative ack receivers do not send ack message
  until something wrong (timeout or receiving any
  incorrect message)
• When should we positive or negative acks?
• How to reduce the number of ack? The receiver may
  delay the receipt of ack and merge multiple ack
  into one, how?
• Reliable message may be implemented in the
  network/middleware/application
• More rounds of messages gt a higher communication
  overhead (heavier workload on the network) and a
  longer delay
• Can ack be applied to asynchronous/non-blocking
  communication?

4

• Preparation of Message
• Data Conversion and Re-conversion
• Marshalling Vs. Unmarshalling
• External Data Representation

5
Marshalling and Unmarshalling

• In order for two computers to exchange data
  values using messages, we need to map data
  structures and data values into a message (a
  sequence of bits)
• Data structures must be flattened into a message
  before transmission by the sender/middleware
• On receiving a message, the data structures must
  be rebuilt by the receiver/middleware
• How to build a message and rebuild the data
  structures?
• Different computers may use different
  representation methods for different data
  structures
• Different byte ordering
• Big-endian the most significant byte come first
  in representing an integer
• Little-endian the least significant byte come
  first in representing an integer
• Different sizes of integers and other types
• Different floating point representations
• Different character sets
• ASCII and EBCDIC for characters

6
Marshalling and Unmarshalling

• The original message on the Pentium using little
  endian.
• (b) The message after receipt on the SPARC using
  big endian.
• (c) The message after being inverted. The little
  numbers in boxes indicate the address of each
  byte.
• How to resolve the problem?

7
Marshalling and Unmarshalling

• How to exchange binary data values between two
  computers using different representation schemes
• (1) The values to be sent are converted to an
  agreed external data form before transmission and
  then converted to the local form on receipt
  (external data representation)
• (2) The values are transmitted using the sender
  format and the method for building a message from
  data structures are included in the message
• Marshalling
• The process of taking a collection of data items
  and assembling them into a form suitable for
  transmission in a message, i.e., an agreed
  external data representation
• Unmarshalling
• The process of disassembling them on arrival to
  produce an equivalent collection of data items at
  the destination

8
External Data Representation

• COBRAs Common Data representation defined with
  CORBA 2.0
• It consists of primitives types (i.e., short
  (16-bit), long (32-bit), char, boolean, ) and
  constructed types (i.e., sequence, string, array,
  ...)
• The types of a data item is not given with the
  data representation in the message as the sender
  and recipient have common knowledge of the order
  and types of the data items in a message
  (assuming the receiver gets the specification
  from other sources)
• Specification of the types of data items to be
  transmitted in a message are described in CORBA
  interface definition language (IDL)
• The CORBA interface complier generates
  appropriate marshalling and unmarshalling
  operations for the arguments and results of
  remote methods from the definitions of the types
  of their parameters and results

9
External Data Representation
CORBA CDR for constructed types
10
Struct Person string name String
place Unsigned long year
11

• Basic Communication Protocol
• Request-Reply Protocols
• What is a protocol?
• Between Two Processes

12
Request-Reply Communication Protocol

• The basic form of communication is the
  request-reply communication (synchronous) model
• Sending a request and then wait for the reply
  from the receiver
• The request-reply communication protocol can be
  adopted by the network to provide reliable
  message transmission
• The request-reply communication protocol can also
  be performed by the application processes if the
  services provided by the underlying network is
  unreliable (i.e., UDP)
• Client-server model uses request-reply in
  communication
• Request-reply is normally synchronous. Why?
• A client waits for the reply and a server waits
  for the request
• Could it be asynchronous?
• Request-reply can be asynchronous if the client
  can afford to retrieve the reply later and the
  current work not depends on the reply

13
Request-Reply Communication Protocol

• Request-reply communication protocol (assuming
  the use of UDP)
• doOperation
• The client invokes a remote operation through the
  doOperation
• The argument specifies the remote object and
  which method to invoke together with the
  information required by the method
• Marshal the arguments into an array of bytes and
  unmarshal the results from the array of bytes
  that is returned
• After sending the request message, doOperation
  invokes a receive operation to get a reply
  message from which it extracts the result and
  return it to the caller
• Note the caller is blocked (synchronous) until
  it receives the reply
• Where to send the request? The system needs to
  maintain the locations of remote objects gt
  server port and address of the server
• A remote object reference is an identifier for a
  remote object that is valid throughout a
  distributed system

14
Request-Reply Communication Protocol

• getRequest
• The server acquires service requests and provides
  remote services
• It tries to get a client request via the server
  port serverPort
• It is blocking normally and the server is blocked
  until it gets the request
• sendReply
• clientPort PortId reply Message
• Send a reply message to reply to client at its
  port clientPort
• How about the case for TCP?
• Communication protocol becomes the jobs of the
  TCP middleware and the application processes do
  not need to handle the reply

15
Request-Reply Communication Protocol
Server
Client
Request
doOperation
getRequest
message
select object
execute
(wait)
method
Reply
sendReply
message
(continuation)
Fr. Dollimore
16
Operations of Request-Reply Protocol
public byte doOperation (RemoteObjectRef o, int
methodId, byte arguments) sends a request
message to the remote object and returns the
reply. The arguments specify the remote object,
the method to be invoked and the arguments of
that method. public byte getRequest
() acquires a client request via the server
port. public void sendReply (byte reply,
InetAddress clientHost, int clientPort) sends
the reply message reply to the client at its
Internet address and port.
Representation of remote object reference
Fr. Dollimore
17
Request-Reply Message Structure
Fr. Dollimore
RequestID generated by doOperation in the client
used to check against the reply message Message
identifier RquestID port Internet address
18
Request-Reply Communication Protocol

• Communication failures the sender does not
  receive the reply
• Byzantine (arbitrary) failure and omission
  failure
• Loss of request message communication link fails
  / network switch fails / receiver's node is down
• Loss of reply message communication link fails /
  network switch fails / sender's node is down
• Process failure
• Unsuccessful execution of the request server
  crashes while executing the request
• Server fails before it receives the request
• Client fails after sending the request

19
Request-Reply Communication Other Issues

• Failure handling
• Concentrate on message lost
• How to deal with process failure?
• Time-out (failure handling)
• Occur when a request message is lost or the
  network becomes partitioned, or the server is
  overloaded (and hence slow) or the reply message
  is lost
• doOperation repeats sending the request message N
  times (time-outs) before reporting failure
• It is impossible to distinguish between a process
  failure and a communication failure. When process
  does not reply after some agreed number, N of
  attempts to communicate with it, it is assumed to
  be unavailable
• The choice of N is difficult (?)
• Failure recovery what to do after the timeout,
  i.e., no reply?
• Resend or abort the operation

20
Request-Reply Communication Other Issues

• Duplicated request messages
• Occur when request message is retransmitted (on
  time-outs)
• Duplicates can be detected using Request-Id (like
  a sequence number) ( the sender ID) and be
  discarded
• Lost reply messages
• If the server has already sent the reply message,
  it may need to execute the request again to
  obtain the result. Re-executing is only possible
  for idempotent operation
• An idempotent operation is an operation that can
  be performed repeatedly with the same effect as
  if it had been performed exactly once
• If server operation is not idempotent a record of
  past results (called history) can be kept.
  History can be kept from growing too large by
  discarding results which have passed a certain
  time limit (how to define the time limit?)

21
Request-Reply Communication Other Issues

• Multipacket messages
• Datagram with limited length (often as 8 kbytes)
• Not enough if a request or reply is too large
• Solution ? multipacket a message made up of a
  sequence of datagrams
• Drawbacks complicated in design and control
  (receive in sequence), low efficiency in
  retransmission

22
Request-Reply Communication Protocol

• In the presence of communication failures, three
  protocols may be used for implementing various
  types of client-server communication
• The request (R) protocol
• The client issues a send (server-id, request) and
  continues
• It is suitable for cases in which there is no
  reply required from the server and that the
  client requires no confirmation that the request
  has been carried out (any example?)
• The client may proceed immediately after sending
  the request
• The request-reply (RR) protocol
• Most commonly used
• The reply message from the server also acts as
  acknowledgment to the original request message
• A subsequent request from the client may be
  regarded as an acknowledgment of the server's
  message
• I.e., the HTTP protocol

23
Request-Reply Communication Protocol
24
Request-Reply Communication Protocol
25
Request-Reply Communication Protocol

• The request-reply-acknowledge reply (RRA)
  protocol
• The ack includes the request-Id and acknowledges
  all request messages up to that request-Id
• Although the exchange involves an additional
  message, it needs not block the client as the
  acknowledgement may be transmitted after the
  reply has been given to the client, but it does
  use processing and network resources
• On receiving the ack, the server may remove all
  related the history entries
• Any further ack? I.e., the server acks the
  receipt of the ack from the client

Client Server DoOperation GetRequest
(execute the request)
SendReply SendAck
26
Request-Reply Communication Protocol
27
Request-Reply Communication Exchange Protocol

Name
Messages sent by
Client
Server
Client
R
Request
R
R
Reply
Request
R
R
A
Acknowledge reply
Request
Reply
Fr. Dollimore
28
HTTP an example of Request-Reply Communication

• A client request specifies the name of a method
  (procedure) to be applied to a resource at the
  server and the URL of that resource
• The reply from the server reports on the status
  of the request
• Requests and reply may also contain resource
  data, the contents of a form or the output of a
  program resource run on the web server

HTTP request message
HTTP reply message
Fr. Dollimore
29
Middleware Layer
Application of the Request-Reply Communication
Protocol
Fr. Dollimore
30
Remote Procedure Call (RPC)

• Remote Procedure Call (RPC) is a high-level model
  for client-sever communication
• RPC was originally proposed by Birrell Nelson
  in 1984
• RPC aims to provide a transparent service for
  procedure calls such that they should appear to
  the callers that normal calls are taking place
  locally
• Why we need Remote Procedure Call (RPC)?
• The client needs a easy way to call the
  procedures of servers to get services
• RPC enables clients to communicate with servers
  by calling procedures in a similar way to the
  conventional use of procedure calls in high-level
  languages (transparency and flexibility)
• RPC is modeled on the local procedure call, but
  the called procedure is executed in a different
  process and usually a different/remote computer

31
Remote Procedure Call (RPC)

• How to operate RPC?
• When a process on machine A calls a procedure on
  machine B, calling process on A is suspended
  (blocking), and the execution of the called
  procedure takes place on B
• Information can be transported from the caller to
  the callee in the parameters and can come back
  from callee in the procedure result
• No message passing or I/O at all is visible to
  the programmer
• How to support RPC? Procedure calling at a remote
  node
• Create stub functions to make it appear to the
  user that the call is local
• Stub function contains the functions interface
• RPC hides all network code into stub functions
• Application programmers do not have to worry
  about details, i.e., sockets, port numbers, byte
  ordering
• A server process defines in its service interface
  the procedures available for calling remotely
• RPC is generally implemented over a request-reply
  protocol using transient (often synchronous)
  communication

32
Remote Procedure Call (RPC)

• The RPC model

33
Asynchronous RPC
(a) The interconnection between client and server
in a traditional RPC (b) The interaction using
asynchronous RPC
34
Asynchronous RPC
2-13
A client and server interacting through two
asynchronous RPCs
35
RPC Characteristics

• The called procedure is in another process which
  may reside in a remote machine
• No shared variables and address spaces
• How to pass the procedure parameters and return
  the results?
• Communication (the parameters and results of the
  procedure) between the caller and callee can only
  be done through passing messages
• The processes do NOT share address spaces
• Passing of parameters by reference and passing
  pointer values are NOT allowed
• Parameters are passed by values
• Is it possible to support pass by references?
• The called remote procedure executes within the
  environment of the server process
• The called procedure does not have access to the
  calling procedure's (clients) environment
  (separated)

36
RPC Characteristics

• Speed remote procedure calling (and return) time
  (i.e., overheads) can be significantly (1 - 3
  orders of magnitude) slower than that for local
  procedure
• Not just slow, it is also unpredictable due to
  network jitter (overloaded network) and failures
  (communication and process failures)
• This may affect real-time design and the
  programmer should be aware of its impact
  (real-time systems require predictable and
  guaranteed performance)
• Failure issues RPC is more vulnerable to failure
  (why?)
• Process failures, communication failures, lost of
  messages, synchronization problems,
• The programmers should be aware of the call
  semantics, i.e., programs that make use of RPC
  must have the capability of handling errors that
  cannot occur in local procedure calls

37
RPC Design Issues

• Exception handling
• Necessary because of possibility of network and
  nodes failures
• RPC uses return value to indicate errors (message
  passing)
• Transparency
• Syntactic ? achievable, exactly the same syntax
  as a local procedure call
• Semantic ? impossible because of RPC limitations
  failure (similar but not exactly the same)
• Delivery guarantees (fault tolerance measures)
• Retry request message whether to retransmit the
  request message until either a reply or the
  server is assumed to be failed
• Duplicate filtering when retransmission are
  used, whether to filter out duplicates at the
  server
• Retransmission of replies whether to keep a
  history of reply messages to enable lost replies
  to be retransmitted without re-executing the
  server operations

38
RPC Design Issues (Call Semantics)

• Maybe call semantics
• The remote procedure may have executed once or
  not at all
• The request or reply may lose and the server may
  crash
• After an RPC time-out (or a client crashed and
  restarted), the client is not sure if the remote
  procedure may or may not have been called
• This is the case when no fault tolerance measure
  is built into RPC mechanism
• Clearly, maybe semantics is not desirable
• Compare with request protocol. The RPC may
  require no reply
• At-least-once call semantics
• The caller receives either a result, in which
  case the caller knows that the procedure was
  executed at least once or an exception informing
  it that no result was received
• Can be implemented by retransmission of the
  (call) request message on time-out for solving
  the communication failure problems
• Acceptable only if the servers operations are
  idempotent

39
RPC Design Issues (Call Semantics)

• At-most-once call semantics
• When an RPC returns, it can be assumed that the
  remote procedure has been called exactly once or
  not at all
• Retransmission of request upon time-out to solve
  the communication failure problem
• Implemented by the server's filtering of
  duplicate requests (which are caused by
  retransmissions due to IPC failure, slow or
  crashed server) and caching of replies (in reply
  history)
• This ensures the remote procedure is called
  exactly once if the server does not crash during
  execution of the remote procedure
• When the server crashes during the remote
  procedure 's execution, the partial execution may
  lead to erroneous results
• In this case, we want the effect that the remote
  procedure has not been executed at all
• How to resolve? Recovery procedure to clear up
  partial results

40
Invocation Semantics
41
RPC Mechanisms

• How does the client know the procedure (names)
  that it can call and which parameters it should
  provide from the server?
• Server interface definition
• RPC interface specifies those characteristics of
  the procedures provided by a server that are
  visible to the clients
• The characteristics includes names of the
  procedures and type of parameters
• Each parameter is defined as input or output
• This interface is made known to the clients
  through a server process binder

42
RPC Mechanisms

• How does the client transfer its call request
  (the procedure name) and the arguments to the
  server via network?
• Marshalling and communication with server
• For each remote procedure call, a (client) stub
  procedure is generated and attached to the
  (client) program
• Replace the remote procedure call to a (local)
  call to the stub procedure
• The (codes in the) stub procedure marshals (the
  input) arguments and places them into a message
  together with the procedure identifier (of the
  remote procedure)
• Use IPC primitives to send the (call request)
  message to the server and wait the reply (call
  return) message (doOperation)

43
RPC Mechanisms

• How does the server react the request of the
  client? From which port? How to select the
  procedure? How to interpret the arguments?
• Despatching, Unmarshalling, communication with
  client
• A dispatcher is provided. It receives the call
  request message from the client and uses the
  procedure identifier in the message to select one
  of the server stub procedures and passes on the
  arguments
• For each procedure at the server which is
  declared (at the sever interface) as callable
  remotely, a (server) stub procedure is generated
• The task of a server stub procedure is to
  unmarshal the arguments, call the corresponding
  (local) service procedure

44
RPC Mechanisms

• How does the server transmit the reply back?
• On return, the stub marshals the output
  arguments into a reply (call return) message and
  sends it back to the client
• How does the client receive the reply?
• The stub procedure of the client unmarshals the
  result arguments and returns (local call return).
  Note that the original remote procedure call was
  transformed into a (local) call to the stub
  procedure

45
RPC Mechanism
46
RPC Mechanism (Summary)

• 1. The client provides the arguments and calls
  the client stub in the normal way
• 2. The client stub builds (marshals) a message
  (call request) and traps to OS network kernel
• 3. The kernel sends the message to the remote
  kernel
• 4. The remote kernel receives the message and
  gives it to the server dispatcher
• 5. The dispatcher selects the appropriate server
  stub
• The server stub unpacks (unmarshals) the
  parameters and call the corresponding server
  procedure
• The server procedure does the work and returns
  the result to the server stub.
• 8. The server stub packs (marshals) it in a
  message (call return) and traps it to OS
  network kernel
• 9. The remote (receiver) kernel sends the message
  to the client kernel
• 10. The client kernel gives the message to the
  client stub
• 11. The client stub unpacks (unmarshals) the
  result and returns to client

47
RPC Implementation

• Communication handling
• TCP, UDP communication and socket programming
  (hidden and supported by the underlying network
  or middleware)
• Binding
• It specifies a mapping from a name to a
  particular object (procedure) usually identified
  by a communication ID
• Binding is important because
• An interface definition specifies a textual
  service name for use by clients and servers
• Clients request message must be addressed to a
  server port
• Binder a separate service that maintains a table
  containing mappings from service names to server
  ports
• All other services depend on the binder service
• Binder interface used by server
• Register (String serviceName, Port serverPort,
  int version)
• Withdraw (String serviceName, Port serverPort,
  int version)
• Binder interface used by client
• PortLookUp (String serviceName, int version)

48

• For Object-Based Systems
• Remote Procedure Calls
• Remote Object (Method) Invocations
• Another set of terminologies but
• the same principles and procedures

49
Remote Method Invocation (RMI)

• Object model An object-oriented programming,
  i.e., Java and C, consists of a collection of
  interacting objects, each of which consists of a
  set of data and a set of methods
• Distributed objects
• The state of a program is partitioned into
  separated parts, each of which is associated with
  an object (object states)
• Objects may be distributed in the system over
  distributed nodes (sites)
• In a client-server architecture, the server
  manages a set of objects while the client may
  manage another set of objects
• In remote method invocation (RMI)
• The clients request to invoke a method of an
  object is sent in a message to the server
  managing the object
• The invocation is carried out by executing a
  method of the object at the server
• The result is returned to the client in another
  message
• To allow for chains of invocation, objects in
  servers are allowed to become clients of objects
  in other server
• To make it more fault tolerant, objects can be
  replicated
• How does it compare with RPC?

50
Remote Method Invocation (RMI)
B and F are remote objects
51
Remote Method Invocation (RMI)

• Remote object reference It is an identifier that
  can be used throughout a distributed system to
  refer to a particular unique remote object
• Remote interface every remote object has a
  remote interface that specifies which of its
  methods can be invoked remotely

52
RMI Implementation

• Communication module
• Carry out the request-reply protocol
• Select the dispatcher for the class of the object
  to be invoked
• Remote reference module
• Translate between local and remote object
  references and for creating remote object
  references
• In each process, it has a remote object table
  that records the correspondence between local
  object references in that process and remote
  object references
• I.e., at server, an entry of remote object B will
  be recorded in the table
• I.e., at client, an entry of proxy for B will be
  recorded in the table
• Servant
• A servant is an instance of a class which
  provides the body of a remote object
• It handles the request passed on by the
  corresponding skeleton
• They are created when remote objects are
  instantiated and remain in use until they are no
  longer needed to be deleted in garbage collection

53
RMI Implementation
server

client
remote
skeleton
object B
object A
proxy for B
Request
dispatcher
for Bs class
Reply
servant
Communication
Remote reference
Communication
Remote
module
module
module
reference module
54
RMI Software

• Proxy
• One proxy for each remote object for which a
  process holds a remote object reference
• The role of proxy is to make remote method
  invocation transparent to clients by behaving
  like a local object to the invoker
• It hides the details of the remote object
  reference, the marshalling of arguments,
  unmarshalling of results and sending and
  receiving of messages from the client
• The class of a proxy implements the methods in
  the remote interface of the remote object it
  represents
• Dispatcher
• A server has one dispatcher and skeleton for each
  class representing a remote object
• It receives the request message from the
  communication module and uses the methodId to
  select the appropriate method in the skeleton,
  passing on the request message

55
RMI Software

• Skeleton
• The class of a remote object has a skeleton which
  implements the methods in the remote interface
• It unmarshals the argument in the request message
  and invoke the corresponding method in the
  servant
• It waits for invocation to complete and then
  marshals the result, together with any exception,
  in a reply message to the sending proxys method
• Compare proxy with client stub
• Compare skeleton with server stub
• Compare RMI with RPC

56
References

• Dollimore chapters 4.3.1, 4.4, 4.6, 5.1, 5.2 and
  5.3
• Tanenbaum chapters 2.2 and 2.4