Distributed Computing Class: BIT5

1
Distributed ComputingClass BIT5 6Instructor
Aatif Kamal
Dated Oct 2006

• Chapter 04
• Interprocess Communication

2
Chapter 3

• Networking issues for distributed systems
• Types of network
• Network Principles
• Protocol layers
• Internetworking
• Routing
• Internet protocols
• Case Studies
• Ethernet
• MobileLAN
• ATM

3
Objectives of the lecture

• To study the general characteristics of
  interprocess communication and the particular
  characteristics of both datagram and stream
  communication in the Internet.
• To be able to write Java applications that use
  the Internet protocols and Java serialization.
• To be aware of the design issues for
  Request-Reply protocols and how collections of
  data objects may be represented in messages (RMI
  and language integration are left until Chapter
  5).
• To be able to use the Java API to IP multicast
  and to consider the main options for reliability
  and ordering in group communication.

4
Chapter 4 Interprocess Communication

• Introduction
• The API for the Internet protocols
• External data representation and marshalling
• Client-Server communication
• Group communication
• Case study interprocess communication in UNIX
• Summary

5
Introduction

• Middleware layer
• TCP ( UDP) from a programmers point of view
• TCP two way stream
• UDP datagram, message passing
• Java interface, UNIX Socket
• marshalling and demarshalling
• data form translation
• Client-Server communication
• Request-Reply protocols
• Java RMI, RPC
• Group communication

6
Middleware layer (1)
7
Middleware layer (2)
2-5

• An adapted reference model for networked
  communication.

8
Middleware layer (3)

• A software layer that
• masks the heterogeneity of systems
• provides a convenient programming abstraction
• provides protocols for providing general-purpose
  services to more specific applications, e.g.
  naming, security, transaction, persistent storage
  and event notification
• authentication protocols
• authorization protocols
• distributed commit protocols
• distributed locking protocols
• high-level communication protocols
• remote procedure calls (RPC)
• remote method invocation (RMI)

9
Middleware (4)

• General structure of a distributed system as
  middleware.

1-22
10
Middleware and Openness
1.23

• In an open middleware-based distributed system,
  the protocols used by each middleware layer
  should be the same, as well as the interfaces
  they offer to applications.

11
Middleware programming models

• Remote Calls
• remote Procedure Calls (RPC)
• distributed objects and Remote Method Invocation
  (RMI)
• eg. Java RMI
• Common Object Request Broker Architecture (CORBA)
  
• Other programming models
• remote event notification
• remote SQL access
• distributed transaction processing

12
Chapter 4 Interprocess Communication

• Introduction
• The API for the Internet protocols
• External data representation and marshalling
• Client-Server communication
• Group communication
• Case study interprocess communication in UNIX
• Summary

13
The characteristics of interprocess communication
(1)

• Send and receive
• Synchronous and asynchronous
• a queue associated with message destination,
  Sending process add message to remote queue
• Receiving process remove message from local queue
• Synchronous
• send and receive are blocking operations
• Asynchronous
• send is unblocking,
• receive could be blocking or unblocking
• (receive notification by polling or interrupt)

14
The characteristics of interprocess communication
(2)

• Message destinations
• Internet address local port
• service name names into server locations at run
  time
• location independent identifiers, e.g. in Mach
• Reliability
• validity messages are guaranteed to be delivered
  despite a reasonable number of packets being
  dropped or lost
• Integrity messages arrive uncorrupted and
  without duplication
• Ordering
• the messages be delivered in sender order

15
Socket

• Endpoint for communication between processes
• Both forms of communication (UDP and TCP ) use
  the socket abstraction
• Originate from BSD Unix, be present in Linux,
  Windows NT and Macintosh OS etc
• be bound to a local port (216 possible port
  number) and one of the Internet address
• a process cannot share ports with other processes
  on the same computer

16
Sockets

• The application level API to TCP UDP
• Allows users to write application protocol
  objects which sit on top of TCP and UDP.
• Execution of TCP/UDP/IP are in kernel space.
  Sockets provide the bridge to user space.
• In Java provides 3 types of sockets
• DatagramSocket for UDP
• ServerSocket for TCP server
• Socket endpoint of a TCP stream

App 2
App 1
App 3
socket
socket
socket
TCP
UDP
IP
17
UDP datagram communication

• UDP datagrams are sent without acknowledgement or
  retries
• Issues relating to datagram communication
• Message size not bigger than 64k in size,
  otherwise truncated on arrival
• blocking non-blocking sends (message could be
  discarded at destination if there is not a socket
  bound to the port ) and blocking receives (could
  be timeout)
• Timeout receiver set on socket
• Receive from any not specify an origin for
  messages, but could be set to receive from or
  send to particular remote port by socket
  connection operation

18
UDP datagram communication

• Failure model
• omission failure message be dropped due to
  checksum error or no buffer space at sender side
  or receiver side
• ordering message be delivered out of sender
  order
• application maintains the reliability of UDP
  communication channel by itself (ACK)

19
Java API for UDP datagrams

• DatagramPacket
• DatagramSocket
• send and receive transmit datagram between a
  pair of sockets
• setSoTimeout receive method will block for the
  time specified and then throw an
  InterruptedIOexception
• connect connect to a particular remote port and
  Internet address
• Examples
• be acceptable to services that are liable to
  occasional omission failures, e.g. DNS

20
Datagram Sockets

• Creating a datagram socket
• DatagramSocket soc new DatagramSocket(port-no.)
  
• Creating a datagram packect
• DatagramPacket p new DatagramPacket(byte
  data, int len, InetAddress dest, int dest-port)
• Sending a packet
• soc.send( p )
• Receiving a packet
• q new DatagramPacket(buff, len)soc.receive( q
  )

IP header source-ip-address
destination-ip-address protocol 17 (udp)
UDP header source port-no destination
port-no
Application Data
21
UDP client sends a message to the server and gets
a reply
import java.net. import java.io. public class
UDPClient public static void main(String
args) // args give message contents and
server hostname DatagramSocket aSocket null
try aSocket new DatagramSocket()
byte m args0.getBytes() InetAddress
aHost InetAddress.getByName(args1) int
serverPort 6789
DatagramPacket request
new DatagramPacket(m, args0.length(), aHost,
serverPort) aSocket.send(request)
byte buffer new
byte1000 DatagramPacket reply new
DatagramPacket(buffer, buffer.length) aSocket.
receive(reply) System.out.println("Reply "
new String(reply.getData())) catch
(SocketException e)System.out.println("Socket "
e.getMessage()) catch (IOException
e)System.out.println("IO " e.getMessage())
finally if(aSocket ! null) aSocket.close()

22
UDP server repeatedly receives a request and
sends it back to the client
import java.net. import java.io. public class
UDPServer public static void main(String
args) DatagramSocket aSocket null
try aSocket new DatagramSocket(6789) b
yte buffer new byte1000 while(true)
DatagramPacket request new
DatagramPacket(buffer, buffer.length)
aSocket.receive(request)
DatagramPacket reply new DatagramPacket(request.
getData(), request.getLength(),
request.getAddress(), request.getPort())
aSocket.send(reply) catch
(SocketException e)System.out.println("Socket "
e.getMessage()) catch (IOException e)
System.out.println("IO " e.getMessage()) f
inally if(aSocket ! null) aSocket.close()

23
TCP stream communication (1)

• The API to the TCP
• provide the abstraction of a stream of bytes to
  which data may be written and from which data may
  be read
• Hidden network characteristics
• message sizes (how much data to write/read)
• lost messages (ACK)
• flow control (match the speed)
• message duplication and ordering (identifier with
  each IP Packet)
• message destinations (establish connection) Once
  connection is established, no need of address
  ports
• While connection establishing client/server
• Client ? Connect request, Server ?Accept request,
  becomes Peers
• Client creates stream socket, bind to a port,
  and makes a connect request to server
• Server creates a listening socket, bind to a
  server port, and waits for clients

24
TCP stream communication (2)

• issues related to stream communication
• Matching of data items agree to the contents of
  the transmitted data (int double, write/ read,
  )
• Blocking send blocked until the data is written
  in the receivers buffer, receive blocked until
  the data in the local buffer becomes available
• Threads server create a new thread for every
  connection
• failure model
• integrity and validity have been achieved by
  checksum, sequence number, timeout and
  retransmission in TCP protocol
• connection could be broken due to unknown
  failures
• Cant distinguish between network failure and the
  destination process failure
• Cant tell whether its recent messages have been
  received or not

25
TCP byte stream sockets

• Server socket waits for connections
• soc new ServerSocket( port-no )
• Socket newsoc soc.accept( )
• Client socket connects to server
• client new Socket(server-addr, server-port)
• When connected, get streams
• InputStream in client.getInputStream(
  )OutputStream out client.getOutputStream( )

26
Java API for TCP Streams

• ServerSocket
• accept listen for connect requests from clients
• Socket
• constructor
• not only create a socket associated with a local
  port, but also connect it to the specified remote
  computer and port number
• getInputStream
• getOutputStream

27
TCP client makes connection to server, sends
request and receives reply
import java.net. import java.io. public class
TCPClient public static void main (String
args) // arguments supply message and
hostname of destination Socket s null
try int serverPort 7896 s new
Socket(args1, serverPort)
DataInputStream in new DataInputStream(
s.getInputStream()) DataOutputStream out
new DataOutputStream( s.getOutputStream())
out.writeUTF(args0) // UTF is a
string encoding see Sn 4.3 String data
in.readUTF() System.out.println("Receive
d " data) catch
(UnknownHostException e) System.out.println("S
ock"e.getMessage()) catch (EOFException
e)System.out.println("EOF"e.getMessage())
catch (IOException e)System.out.println("IO
"e.getMessage()) finally if(s!null) try
s.close()catch (IOException e)System.out.print
ln("close"e.getMessage())
28
TCP server makes a connection for each client and
then echoes the clients request (1)
import java.net. import java.io. public class
TCPServer public static void main (String
args) try int serverPort 7896
ServerSocket listenSocket new
ServerSocket(serverPort) while(true)
Socket clientSocket listenSocket.accept()
Connection c new Connection(clientSocket)
catch(IOException e) System.out.println("Lis
ten "e.getMessage()) // this figure
continues on the next slide
29
TCP Server (2)
class Connection extends Thread
DataInputStream in DataOutputStream
out Socket clientSocket public Connection
(Socket aClientSocket) try
clientSocket aClientSocket in new
DataInputStream( clientSocket.getInputStream())
out new DataOutputStream( clientSocket.getOutput
Stream()) this.start()
catch(IOException e) System.out.println("Connect
ion"e.getMessage()) public void run()
try // an echo
server String data in.readUTF()
out.writeUTF(data)
catch(EOFException e) System.out.println("EOF"e
.getMessage()) catch(IOException e)
System.out.println("IO"e.getMessage())
finally try clientSocket.close()catch
(IOException e)/close failed/
30
Chapter 4 Interprocess Communication

• Introduction
• The API for the Internet protocols
• External data representation and marshalling
• Client-Server communication
• Group communication
• Case study interprocess communication in UNIX
• Summary

31
External data representation and marshalling
introduction

• Why does the communication data need external
  data representation and marshalling?
• Different data format on different computers,
  e.g., big-endian/little-endian integer order,
  ASCII (Unix) / Unicode character coding
• How to enable any two computers to exchange data
  values?
• The values be converted to an agreed external
  format before transmission and converted to the
  local form on receipt
• The values are transmitted in the senders
  format, together with an indication of the format
  used, and the receipt converts the value if
  necessary
• External data representation
• An agreed standard for the representation of data
  structures and primitive values
• Marshalling (unmarshalling)
• The process of taking a collection of data items
  and assembling them into a form suitable for
  transmission in a message
• Usage for data transmission or storing in files
• Three alternative approaches
• CORBAs common data representation / Javas
  object serialization XML

32
CORBAs Common Data Representation (CDR)

• Represent all of the data types that can be used
  as arguments and return values in remote
  invocations in CORBA
• 15 primitive types
• Short (16bit), long(32bit), unsigned short,
  unsigned long, float, char,
• Constructed types
• Types that composed by several primitive types
• A message example
• The type of a data item is not given with the
  data representation in message
• It is assumed that the sender and recipient have
  common knowledge of the order and types of the
  data items in a message.
• RMI and RPC are on the contrary

33
CORBA CDR for constructed types
34
CORBA CDR message
Struct Person string name string
place long year
35
External data representation-The SUN XDR standard

• The entire XDR message consists of a sequence of
  4-byte objects
• example Hi, There, 2001

The marshalled message of the example 2 Hi 5
There 2001
36
Java object serialization

• Serialization (deserialization)
• The activity of flattening an object or a
  connected set of objects into a serial form that
  is suitable for storing on the disk or
  transmitting in a message
• Include information about the class of each
  object
• Handles references to other objects are
  serialized as handles
• Each object is written once only
• Example
• Make use of Java serialization
• ObjectOutputStream.writeObject,
  ObjectInputStream.readObject
• The use of reflection
• Reflection The ability to enquire about the
  properties of a class, such as the names and
  types of its instance variables and methods
• Reflection makes it possible to do serialization
  (deserialization) in a completely generic manner

37
Indication of Java serialized form
Public class Person implements Serializable
private String name private String
place private int year public Person (String
aName, String aPlace, int aYear) name
aName place aPlace year aYear //
followed by methods for accessing the instance
variables Person p new Person(Smith,
London, 1934)
38
Representation of a remote object reference

• An identifier for a remote object that is valid
  throughout a distributed system
• Must ensure uniqueness over space and time

• Existence of remote object references
• Life of the remote object references
• Remote object references may not be used for
  relocated objects

39
Chapter 4 Interprocess Communication

• Introduction
• The API for the Internet protocols
• External data representation and marshalling
• Client-Server communication
• Group communication
• Case study interprocess communication in UNIX
• Summary

40
Request-reply protocol - basics

• Client-server communication in terms of the send
  and receive operations in the Java API for UDP
  datagrams.
• Uses remote object references, identifiers for a
  remote object that is valid throughout a
  distributed system.
• Communication primitives
• doOperation.
• getRequest.
• sendReply.

41
The request-reply protocol

• The request-reply protocol
• Normally synchronous
• May be reliable
• Reply is ACK
• Asynchronous may be alternative
• Client may retrieve the reply latter.
• Mostly implemented over UDP but not TCP
• Acknowledgements are redundant since requests are
  followed by replies
• Establishing a connection involves two extra
  pairs of messages in addition to the pair
  required for a request and a reply
• Flow control is redundant for the majority of
  invocations, which pass only small arguments and
  results

42
The request-reply protocol
public byte doOperation (RemoteObjectRef o, int
methodId, byte arguments) sends a request
message to the remote object and receives the
reply. The arguments specify the remote object,
the method to be invoked and the arguments of
that method. public byte getRequest
() acquires a client request via the server
port. public void sendReply (byte reply,
InetAddress clientHost, int clientPort) sends
the reply message reply to the client at its
Internet address and port.
43
Request-reply message structure
44
Request-reply message structure

• Fields
• 1. Message Type
• Request or reply
• 2. A doOperation generates requestId for each
  message
• A server copies in the reply message
• 3 Remote object reference
• marshalled
• 4. methodId

• Message Identifier
• requestId
• Identfier for the sender process, for example its
  port and internet address

45
The request-reply protocol

• Failure model
• Problems
• Omission failures.
• No guarantee of delivery in order.
• Timeout
• doOperation return exception when repeatedly
  issued requests are all timeout
• Duplicate request messages
• filter out duplicates by requestID
• if the server has not yet sent the reply,
  transmit the reply after finishing operation
  execution
• Lost Reply Messages
• If the server has already sent the reply, execute
  the operation again to obtain the result. Note
  idempotent operation, e.g., add an element to a
  set, and a contrary example, append an item to a
  sequence
• History
• History server contains a record of reply
  messages that have been transmitted to avoid
  re-execution of operations

46
The request-reply protocol

• RPC exchange protocol
• Implement the request-reply protocol on TCP
• Costly, but no need for the request-reply
  protocol to deal with retransmission and
  filtering
• Successive requests and replies can use the same
  stream to reduce connection overhead

47
HTTP an example of a request reply protocol

• HTTP used by web browsers to make requests to web
  servers and to receive replies from them.
• Client requests specify a URL that includes the
  DNS hostname of a web server, an optional port
  number and an identification of a resource.
• HTTP specifies
• Messages.
• Methods.
• Arguments.
• Results.
• Rules for representing data in messages.

48
HTTP an example of a request reply protocol

• Over TCP
• Each client-server interaction consists of the
  following steps
• The client requests and the server accepts a
  connection at the default server port or at a
  port specified in the URL
• The client sends a request message to the server
• The server sends a reply message to the client
• The connection is closed
• Persistent connection (http1.1, RFC 2616)
• Connections that remain open over a series of
  request-reply exchanges between client and server
• Closing
• Marshalling
• Request and replies are marshalled into messages
  as ASCII text string
• Resources are represented as byte sequences and
  may be compressed
• MIME (Multipurpose Internet Mail Extensions) to
  send text, images an so on.
• HTTP Methods
• GET, HEAD, POST, PUT, DELETE, OPTIONS, TRACE

49
HTTP - methods

• GET Requests the resource whose URL is given as
  argument.
• HEAD Similar to GET, but no data returned.
• POST Can deal with data supplied with the
  request.
• PUT data supplied in the request is stored with
  the given URL as its identifier either as
  modification of the an existing resource or as a
  new resource.
• DELETE Server deletes resource identified by the
  URL.
• OPTIONS Server supplies client with list of
  methods that it allows.
• TRACE Server sends the request back.

50
HTTP request / reply messages

• Request/Reply
• Header for example acceptable content type
• The message body contains data associated with
  the URL specified in the request

51
Chapter 4 Interprocess Communication

• Introduction
• The API for the Internet protocols
• External data representation and marshalling
• Client-Server communication
• Group communication
• Case study interprocess communication in UNIX
• Summary

52
The usage of Multicast

• Fault tolerance based on replicated services
• Client request are multicast to all the members
  of the group, each of which performs an identical
  operation
• Finding the discovery servers in spontaneous
  networking
• Multicast message can be used by servers and
  clients to locate available discovery services to
  register their interfaces or to look up the
  interfaces of other services
• Better performance through replicated data
• Data are replicated to increase the performance
  of a service, e.g., Web Cache. Each time the data
  changes, the new value is multicast to the
  processes managing the replicas
• Propagation of event notification
• Multicast to a group may be used to notify
  processes when something happens, e.g., the Jini
  system uses multicast to inform interested
  clients when new lookup services advertise their
  existence

53
IP Multicast group communication

• A multicast group is specified by a class D
  Internet address
• Built on top of IP
• Sender is unaware of the identities of the
  individual recipients
• Available only via UDP
• The membership of a group is dynamic
• It is possible to send datagram to a multicast
  group without being a member and join or leave
  the group at any time.
• IPv4
• Multicast routers
• use the broadcast capability of the local network
• MTTL (time to live) - specify the number of
  routers a multicast message is allowed to pass
• Multicast address allocation
• Permanent group exist even without members
  224.0.0.1 to 224.0.0.255
• Temporary group the other addresses, set TTL to
  a small value
• Failure model due to UDP, so it is a unreliable
  multicast
• Java API to IP multicast

54
Multicast peer joins a group and sends and
receives datagrams
import java.net. import java.io. public class
MulticastPeer public static void main(String
args) // args give message contents
destination multicast group (e.g.
"228.5.6.7") MulticastSocket s null try
InetAddress group InetAddress.getByName(a
rgs1) s new MulticastSocket(6789)
s.joinGroup(group) byte m
args0.getBytes() DatagramPacket
messageOut new DatagramPacket(m, m.length,
group, 6789) s.send(messageOut)
// this figure continued on the next slide
55
Multicast peers example continued
// get messages from others in group
byte buffer new byte1000 for(int
i0 ilt 3 i) DatagramPacket messageIn
new DatagramPacket(buffer, buffer.length)
s.receive(messageIn)
System.out.println("Received" new
String(messageIn.getData()))
s.leaveGroup(group) catch
(SocketException e)System.out.println("Socket "
e.getMessage()) catch (IOException
e)System.out.println("IO " e.getMessage())
finally if(s ! null) s.close()
56
Reliability and ordering of multicast

• Failures
• Router failure prevent all recipients beyond it
  receiving the message
• Members of a group receive the same array of
  messages in different orders
• Some examples of the effects of reliability and
  ordering
• Fault tolerance based on replicated services
• if one of them misses a request, it will become
  inconsistent with the others
• Finding the discovery servers in spontaneous
  networking
• an occasional lost request is not an issue when
  locating a discovery server
• Reliable multicast or unreliable multicast?
• According to applications requirement

57
Chapter 4 Interprocess Communication

• Introduction
• The API for the Internet protocols
• External data representation and marshalling
• Client-Server communication
• Group communication
• Case study interprocess communication in UNIX
  FOR SELF STUDY
• Summary

58
UNIX socket

• Datagram communication
• Datagram Socket
• Bind
• Sendto
• recvfrom
• Stream communication
• stream socket , bind
• Accept
• Connect
• Write and read

59
Sockets used for datagrams
60
Sockets used for streams
61
Chapter 4 Interprocess Communication

• Introduction
• The API for the Internet protocols
• External data representation and marshalling
• Client-Server communication
• Group communication
• Case study interprocess communication in UNIX
• Summary

62
Summary

• Two alternative building blocks
• Datagram Socket based on UDP, efficient but
  suffer from failures
• Stream Socket based on TCP, reliable but
  expensive
• Marshalling
• CORBAs CDR and Java serialization
• Request-Reply protocol
• Base on UDP or TCP
• Multicast
• IP multicast is a simple multicast protocol

63
Assignment 3

• Q.1 From the book, answer the following questions
• 4.1
• 4.2
• 4.5
• 4.6
• 4.7
• f) 4.25
• g) 4.27