InterProcess Communication I

1
Inter-Process Communication I

• Basic Problems in Inter-Process Communication
  (IPC)
• Synchronous and Asynchronous Communication
• Message-Queue Communication

2
The Basic Problems in IPC Variations in
Representation Methods for Message
Transmission Synchronous vs. Asynchronous
Communication
3
Reasons for Inter-Process Communication (IPC)

• Multiple processes may improve system performance
  and enhance resource sharing amongst systems and
  users
• Also, multi-threading may improve system
  performance and give a better system design
  (i.e., divide a large task into sub-tasks)
• Problems
• How to divide and organize the processes and
  threads for applications
• A (big) task (implemented with an algorithm) is
  divided into subtasks to be implemented by
  multiple processes/threads (i.e., distributed
  searching)
• Subtasks (distribute algorithms) are related and
  need coordination both in execution and accessing
  shared resources
• The system components of a distributed system are
  both logically and physically separated. They
  must communicate/interact for management purposes
  (i.e., processes and resources)
• How to do the coordination?
• The basis for coordination is communication
  through a network in a distributed system (a
  network is a set of connections in which the
  communication times cannot be ignored)
• Inter-process communication (within the same node
  or with remote nodes)

4
SupportingInter-Process Communication (IPC)

• Communications between two processes located at
  different nodes are always based on message
  passing (i.e., send and receive a message) using
  the underlying network services
• An important issue in the design of a middleware
  is to offer a high-level abstraction to easier
  and better express communication between
  processes (applications)
• Middleware provides various services including
  management of resources, communication
  facilities for accessing remote resources
• Middleware is an application/a set of system
  services logically living in the application
  layer, but it may have its own layers,
  independent of other, more specific applications
  (in between OS and applications)
• Many high-level middleware services are based on
  message-oriented communication, these services
  include remote procedure call (PRC), remote
  method invocation (RMI), message queuing services
  etc

5
System Layers
6
Problems in Inter-Process Communication

• Two basic communication patterns
• Client-server communication request and reply
  messages provide the basis for communication
  between two processes (one to one)
• Group communication some messages are sent to
  several processes in a group (one to many), i.e.,
  broadcast and multicast
• Two basic operations, send and receive, for
  message passing
• Process A wants to send a message with a specific
  length to process B
• Process B wants to receive a message with a
  specific length from process A
• Processes A and B are connected by a network
  which handle the transmission of the message
  (specific length)
• Synchronization problem
• When to send (A) and when to get (B)
• If there are multiple messages to send, when to
  send the next one
• Transmission problem how to define the message
  context and how to send (i.e., reliable
  communication, stream, packet, through a defined
  path, )
• Data representation problem how B can understand
  the message content
• Reliability how A can ensure that B has received
  the message and then leave
• Errors the message may be corrupted or lost in
  transmission, ack? retransmission?

7
Example Client-Server Communication
The basic idea of the model is to structure the
distributed system as a group of cooperating
processes, i.e. the servers, that offer services
to the users, namely, the clients
receive
send
Note sometimes, the reply may not be necessary,
i.e., informing the completion of a task
send
receive
8

• Synchronization Problems
• Send and Receive are done independently or
  coordinated???
• Blocking Vs. Non-blocking
• Asynchronous Vs. Synchronous

9
Synchronization Problems

• Synchronization
• The receiving/sending process are coordinated,
  i.e., ready to get/send the message
• If it is not ready, the sending/receiving process
  has to wait until the opposite one is ready
• The association will be removed only after the
  message has been sent
• Two basic types of communication operations
  (i.e., send and receive)
• Blocking the invocation blocks the execution of
  its invoker (caller)
• Non-blocking the invocation does not block the
  execution of its invoker
• Blocking
• Blocking send
• Issuing process blocks (i.e., control is not
  passed back) until the message has been sent and
  received
• Blocking receive
• Issuing process blocks until a message has
  arrived and passed to the process
• Lower the degree of concurrency
• How to resolve the problem? Fork a new thread for
  communication
• Simpler in design since a process knows when to
  send/receive a message (the most traditional way
  for IPC), i.e., it is easier to set the time-out
  period

10
Synchronization Problems

• Non-Blocking
• Non-blocking send
• Issuing process continues (i.e. control is passed
  back) execution after the message has been copied
  out of the process's environment
• Non-blocking receive
• Issuing process continues if there is no message
  waiting to be received
• Receiver process will have to be notified later
  on message arrival, either by polling or
  interrupt mechanism
• Higher concurrency but how to generate the
  interrupt? Through the middleware or within the
  application
• How to synchronize the send and receive
  operation?
• The sender may be no longer active when the
  receiver gets the message
• Error handling is then a problem as no
  retransmission
• What are the differences combinations?
• Blocking send and non-blocking receive
• Non-blocking send and blocking receive

11
Synchronization Blocking send and receive
12
Synchronization Non-Blocking send and receive
13
Synchronous vs. Asynchronous Communication
(Synchronous)

• Synchronous communication
• Blocking is for synchronization between the
  sender and receiver, i.e., blocking send and
  receive
• At the synchronization point, they know the
  opposite process is ready to send/receive the
  message
• Know when to send and when to receive after
  synchronization
• Advantages
• Can make definite assumptions on message send and
  receipt
• Easier design and control of the distributed
  processes

14
Synchronous vs. Asynchronous Communication
(Synchronous)

• Drawback
• Can lead to inefficiency due to waiting (lower
  concurrency)
• The blocking could be indefinite if the opposite
  process is failed
• Solution time-out method
• Receive (A, msg, time_out)
• If a message is not arrived in time_out seconds,
  the waiting process will be unblocked (and the
  receive operation aborted)
• Send (B, msg, time_out)
• Sender blocks and if the message is not received
  in time_out seconds, the process will be
  unblocked
• How to set the value for time_out?

15
Synchronous vs. Asynchronous Communication
(Asynchronous)

• Asynchronous communication
• Sender and receiver do not synchronize at message
  transfer
• Non-blocking send non-blocking receive
• Non-blocking send blocking receive
• I.e., in a client-server communication, the
  server is blocking and wait for incoming client
  request
• More flexible and potentially more parallelism
• Less assumptions can be said about sending and
  receiving - more difficult to verify program
  properties
• How to do resend if the message is
  corrupted/lost?
• Non-blocking send requires buffering of messages
• How to ensure no buffer overflow?
• When can the buffer be clear?
• What are the differences between blocking and
  synchronization?
• Blocking synchronous and non-blocking
  asynchronous?

16

• Where to send/receive?
• Ports and Socket
• Socket Primitives
• How to send/receive?

17
Sockets and Ports

• Where to send? Specification of the destination
• In the Internet protocols, messages are sent to
  (Internet address, local port)
• A port is a location identifier (integer) which
  can be mapped into low-level address of a
  computing unit in order to deliver/receive
  messages
• A port could have a single receiver (belonging to
  a process/thread) and be multiple senders
• Sockets
• Socket interface is a transport level abstraction
  for IPC
• A socket is an end-point of a two-way (in and
  out) communication connection between two
  programs running on the network
• For a process to receive messages, its socket
  must be bounded to a local port and one of the
  Internet addresses of the computer
• A physical computer has multiple processes each
  for a logical event
• Normally, a server runs on a specific computer
  and has a socket that is bound to a specific port
  number. The server just waits listening to the
  socket for a client to make a connection request
  (blocking receive and non-blocking send)

18
Sockets and Ports

• On the client-side, the client knows the hostname
  (address) of the machine on which the server is
  running and the port number to which the server
  is connected. To make a connection, the client
  sends a connection request to try to connect to
  the server on the server's machine and port
• On receiving a connection request, if everything
  goes well, the server accepts the connection.
  Upon acceptance, the server gets a new socket
  bound to a different port
• It needs a new socket (and consequently a
  different port number) so that it can continue to
  listen to the original socket for connection
  requests while tending to the needs of the
  connected client
• The client and the server each reads from and
  writes to the socket bound to the connection

19
Sockets and Ports
Create a socket at an agreed port to receive
messages
Create a socket binding to a free port at the
time of request
Fr. Dollimore
20
Socket Primitives
Fr. Tanenbaum

• Socket primitives for TCP/IP

21
Example Socket Primitives

• Server
• Socket the OS creates a new communication by
  reserving resources for sending and receiving
  messages for the specified transport protocol
• Bind associates a local address with the newly
  created socket, i.e., bind the IP address of its
  machine together with a specific port to a socket
• Listen for connection-oriented communication and
  is non-blocking
• Accept block the caller until a connection
  request arrives. Create a new socket with same
  properties as the original one to allow the
  server to fork a process to handle the
  communication and it can wait for other connect
  requests from the original socket
• Receive read data over the connection
• Close close a connection
• Client
• Socket a new socket is created
• Connect bind the socket dynamically to a local
  port and connect the socket to the specified
  transport-level address (blocking until
  connected)
• Write write data over the connection
• Close

22
Socket Primitives and Operations
Fr. Tanenbaum
Fr. Tanenbaum

• Connection-oriented communication pattern (TCP)
  using sockets
• Synchronous communication

23
TCP/IP Basics

• Computers running on the Internet communicate to
  each other using either the Transmission Control
  Protocol (TCP) or the User Datagram Protocol
  (UDP) in the transport layer

Application (e.g.http, ftp, telnet)
Transport (e.g.TCP, UDP)
Network (e.g.IP)
Link via device driver
24
Using TCP and UDP in Java

• The application programs (i.e., in Java) are
  operating at the application layer
• Typically, applications do not need to handle
  with the TCP and UDP layers directly
• Instead, they use the classes in the java.net
  package
• These classes provide system-independent network
  communication
• Note that TCP and UDP provide different types of
  network services and incur different overheads in
  communication
• I..e, TCP more overheads as it provides a more
  reliable communication

25
TCP and UDP

• TCP
• A connection-based protocol that provides a
  reliable flow of data between two computers
• Two applications can establish a connection via
  TCP and exchange data (back and forth) over that
  connection
• TCP guarantees that data sent from one end of the
  connection actually gets to the other end and in
  the same order it was sent. Otherwise, an error
  is reported
• UDP
• A connectionless-based protocol that sends
  independent packets of data, called datagrams,
  from one computer to another with no guarantees
  about arrival
• The order of datagram delivery is not guaranteed,
  and each datagram is independent of any other
• Examples of applications clock server sends
  current time to its clients

26
UDP Data Communication

• Sending datagram to a receiving process without
  acknowledgement
• Minimizing the communication and setup cost
• If a failure occurs, the message may not arrive
• To send or receive messages, a process must first
  create a socket bound to an Internet address of
  the local host and a local port
• A server binds its socket to a server port that
  it makes known to clients (pre-assigned port)
• A client connects its socket to any free local
  port (at connect time)
• After connection, the submission of a message may
  start (no knowledge about the current status of
  the server. It may not be ready)
• Message size the receiving process needs to
  specify an array of bytes of a particular size to
  receive a message
• Blocking sockets normally provide non-blocking
  sends and blocking receives for datagram
  communication
• Time-out time-out may be specified to unblock
  the receiver
• Failure models are handled by applications, i.e.,
  adding a check sum

27
UDP Data Communication

• DatagramPacket
• This class provides a constructor that makes an
  instance out of an array of bytes comprising a
  message, the length of the message and the
  Internet address and local port number of the
  destination socket
• DatagramPacket array of bytes containing message
  Length of message Internet address Port
  number
• getData for receiving a message
• getPort getting the port number
• getAddress getting the Internet address
• DatagramSocket
• This class supports sockets for sending and
  receiving UDP datagram
• It provides a constructor that takes a port
  number as argument, for use by processes that
  need to use a particular port
• Send and receive for transmitting datagram
  between a pair of sockets
• setSoTimeout set a timeout period
• Connect for connecting to a particular remote
  port and Internet address

28
UDP client sends a message to the server and get
a reply
import java.net. import java.io. public class
UDPClient public static void main(String
args) // args give message contents and
server hostname DatagramSocket aSocket null
try aSocket new DatagramSocket()
byte m args0.getBytes() InetAddress
aHost InetAddress.getByName(args1) int
serverPort 6789
DatagramPacket request
new DatagramPacket(m, args0.length(), aHost,
serverPort) aSocket.send(request)
byte buffer new
byte1000 DatagramPacket reply new
DatagramPacket(buffer, buffer.length) aSocket.
receive(reply) System.out.println("Reply "
new String(reply.getData())) catch
(SocketException e)System.out.println("Socket "
e.getMessage()) catch (IOException
e)System.out.println("IO " e.getMessage())
finally if(aSocket ! null) aSocket.close()

29
UDP server repeatedly receives a request and
sends it back to the client
import java.net. import java.io. public class
UDPServer public static void main(String
args) DatagramSocket aSocket null
try aSocket new DatagramSocket(6789) b
yte buffer new byte1000 while(true)
DatagramPacket request new
DatagramPacket(buffer, buffer.length)
aSocket.receive(request)
DatagramPacket reply new DatagramPacket(request.
getData(), request.getLength(),
request.getAddress(), request.getPort())
aSocket.send(reply) catch
(SocketException e)System.out.println("Socket "
e.getMessage()) catch (IOException e)
System.out.println("IO " e.getMessage()) f
inally if(aSocket ! null) aSocket.close()

30
TCP Stream Communication

• TCP protocol provides the abstraction in
  coordinated communication of a stream of bytes
• Message size The sending end keeps a record of
  each IP packet sent and the receiving end
  acknowledges all the arrivals
• Flow control If the writer is too fast for the
  reader, it is blocked until the reader has
  consumed sufficient data
• Message duplication and ordering message
  identifiers are associated with each IP packet
  which enables the recipients to detect and reject
  duplication and reorder messages
• Message destination a pair of communicating
  processes establish a connection before they can
  communicate over a stream
• In a stream connection, one of them plays the
  client (sending) role and the other plays the
  server (receiving) role
• When the server accepts a connection, a new
  stream socket is created for communicating with
  the client
• The pair of sockets are connected by a pair of
  streams, the input stream and the output stream

31
TCP Stream Communication

• Failure model
• Uses checksum to detect and reject corrupted
  packet
• Uses sequence number to reject duplicated packets
• Uses timeout and retransmissions to deal with
  lost packets
• ServerSocket
• Used by a server to create a socket for listening
  for connect from clients
• The accept method gets a connect request from the
  queue (blocking)
• A socket instance is created for giving access to
  streams for communicating with the client
• Socket
• Used by a pair of processes with a connection
• The client uses a constructor to create a socket
  specifying hostname and a port of a server
• Providing methods getInputStream and
  getOutputStream for accessing the two streams
  associated with a socket

32
TCP client makes connection to server, sends
request and receives reply
import java.net. import java.io. public class
TCPClient public static void main (String
args) // arguments supply message and
hostname of destination Socket s null
try int serverPort 7896 s new
Socket(args1, serverPort)
DataInputStream in new DataInputStream(
s.getInputStream()) DataOutputStream out
new DataOutputStream( s.getOutputStream())
out.writeUTF(args0) // UTF is a
string encoding see Sn 4.3 String data
in.readUTF() System.out.println("Receive
d " data) catch
(UnknownHostException e) System.out.println("S
ock"e.getMessage()) catch (EOFException
e)System.out.println("EOF"e.getMessage())
catch (IOException e)System.out.println("IO
"e.getMessage()) finally if(s!null) try
s.close()catch (IOException e)System.out.print
ln("close"e.getMessage())
33
TCP server makes a connection for each client and
then echoes the clients request
import java.net. import java.io. public class
TCPServer public static void main (String
args) try int serverPort 7896
ServerSocket listenSocket new
ServerSocket(serverPort) while(true)
Socket clientSocket listenSocket.accept()
Connection c new Connection(clientSocket)
catch(IOException e) System.out.println("Lis
ten "e.getMessage()) // this figure
continues on the next slide
34
TCP server makes a connection for each client and
then echoes the clients request
class Connection extends Thread
DataInputStream in DataOutputStream
out Socket clientSocket public Connection
(Socket aClientSocket) try
clientSocket aClientSocket in new
DataInputStream( clientSocket.getInputStream())
out new DataOutputStream( clientSocket.getOutput
Stream()) this.start()
catch(IOException e) System.out.println("Connect
ion"e.getMessage()) public void run()
try // an echo
server String data in.readUTF()
out.writeUTF(data)
catch(EOFException e) System.out.println("EOF"e
.getMessage()) catch(IOException e)
System.out.println("IO"e.getMessage())
finally try clientSocket.close()catch
(IOException e)/close failed/
35
Supporting Multiple Clients

• The server can service them simultaneously
  through the use of threads - one thread per each
  client connection
• The basic flow of logic in such a server is this
  
• while (true)
• accept a connection
• create a thread to deal with the client
• end while
• The thread reads from and writes to the client
  connection as necessary

36

• More on how to send
• Synchronization Problems
• After sending a message, when I can leave?
• Transient Vs. Persistent
• Message-Oriented Message Queues

37
Message-Oriented Communication

• Message-oriented communication concentrates on
  exchange more-or-less independent and complete
  units of information, such as request or reply
  messages etc.
• The other form is stream-oriented communication,
  in which timing plays a crucial role. Two
  successive messages may have a temporal
  relationship such as those in video and audio
  streams in multimedia applications
• What will be the consequence of failing to meet
  the timing requirements, i.e., a packet of a
  stream arrives late?
• Communication system for message-oriented
  communication
• The hosts for application execution
• Communication servers offer an interface for the
  hosts to connect to for passing messages between
  two hosts
• Buffers are placed in hosts and communication
  server
• Note What are the benefits of adding the
  communication server. Making the communication
  transparent and increasing the flexibility in the
  design of communication protocols, i.e.,
  synchronous Vs. asynchronous and transient Vs.
  persistent

38
Persistence and Synchronicity in Communication
Fr. Tanenbaum
2-20

• General organization of a communication system in
  which hosts are connected through a network

39
Message-Oriented Communication

• Transient communication A message is stored by
  communication system only as long as the sending
  and receiving application are executing
• Persistent communication A message that has been
  submitted for transmission is stored by the
  communication system as long as it takes to
  deliver it to the receiver (the sending and
  receiving application may be terminated)
• Asynchronous communication A sender continues
  immediately after it has submitted its message
  for transmission (after executing operation send)
• Synchronous communication The sender is blocked
  until its message is stored in a local buffer at
  receiving host, or to the receiver. The strongest
  form is it is blocked until the receiver has
  processed the message (e.g., in request-reply
  protocol)

40
Message-Oriented Communication

• Persistent asynchronous A message is either
  persistently stored in a buffer at the local host
  or at the first communication server
• Persistent synchronous The sender is blocked
  until its message is stored in the receivers
  buffer/communication server connected to the
  receiving host
• Transient asynchronous Typically offered by
  transport layer datagram services such as UDP.
  When a sending application submits a message, it
  is temporally stored in a local buffer, after
  which the sender continues immediately. In
  parallel, the communication system routes the
  message to the destination
• Transient synchronous A sender continues
  immediately after it has submitted its message
  for transmission (after executing operation send)
• Receipt-based transient synchronous the sender
  is blocked until it receives an acknowledgement
• Delivery-based transient synchronous the sender
  is blocked until the message is delivered to the
  receiver for further processing
• Response-based transient synchronous the sender
  is blocked until it receives a response (most
  common but may not suitable for large networks)

41

• Persistent asynchronous communication
• Persistent synchronous communication

42

• Transient asynchronous communication
• Receipt-based transient synchronous communication

43

• Delivery-based transient synchronous
  communication at message delivery
• Response-based transient synchronous communication

44
Message-Oriented Transient Communication

• Many DSs and applications are built directly on
  top of the simple message-oriented model (as part
  of middleware solution) offered by the transport
  layer
• Until recently, many distributed systems
  supported only response-based transient
  synchronous communication, i.e., remote procedure
  call (RPC) and remote object invocation (RMI)
• Alternatively, some may provide asynchronous RPC
  to reduce the synchronization delay
• Some may take transient asynchronous
  communication as the starting point and then add
  synchronization facilities, how?
• In transient communication, whenever a failure
  occurs, the failure has to be masked immediately
  and a recovery procedure has to be initiated
• Persistent communication is needed for large
  networks with unpredictable performance

45
The interconnection between client and server in
a traditional RPC The interaction using
asynchronous RPC
46

• Transient Communication Model
• (synchronous and asynchronous)
• Persistent Communication Model
• (synchronous and asynchronous)
• Adding a component for communication
• Message-Queuing Model
• Asynchronous Persistent
• Adding a network for message delivery

47
Message-Queuing Model

• Message queuing systems (or message-oriented
  middleware - MOM) are an important class of
  message-oriented middleware services for
  supporting asynchronous persistent communication
  (e.g., for email service)
• Providing persistency of message communication in
  a large network
• The essence of these systems is to offer
  intermediate-term storage capacity for message,
  without requiring either the sender or receiver
  to be active during message passing
• What are the benefits? Separating the send and
  receive operations
• In principle, each application has its own
  private queue to which other applications can
  send messages
• Applications communicate by inserting messages in
  specified queues until they are removed
  (persistent)
• The messages are forwarded through a series of
  connected communication servers before arriving
  the destination
• No guarantees are given about when the messages
  will be arriving at receiver and when they will
  be read
• Loosely-coupled communication
• Sender and receiver can execute independently
  (asynchronous)

48

• Four combinations for loosely-coupled
  communications using queues.

49

• Basic interface to a queue in a message-queuing
  system.

50
Message-Queuing System

• Source queue putting a message into a (local)
  queue with a destination queue address to which
  it should be transferred to
• The queues are manager by queue managers which
  interact with applications and can lookup address
  table (which maintain the connection information)
  for routing purposes
• For large and dynamic networks, routers may be
  used for routing. The routers maintain the latest
  queue connection information (in a database)
• Overlay network the queuing system can be
  considered as a network on top of the physical
  network
• Connection management
• Static Vs. dynamic connection
• Latest connection information is maintained by
  routers
• Various routing schemes, i.e., dynamic routing
  and broadcasting

51

• The relationship between queue-level addressing
  and network-level addressing.

52
Message-Queuing System
2-29
Fr. Tanenbaum

• The general organization of a message-queuing
  system with routers.

53
References

• Dollimore chapters 4.1 and 4.2
• Tanenbaum chapters 2.2 and 2.4