Remote Procedure Call RPC

1
Remote Procedure Call (RPC)
2
Readings

• Tanenbaum and van Steen 2.2 and 2.3
• Coulouris Chapter 5
• A good book Power Programming with RPC by
  John Bloomer
• Another good book Practical UNIX Programming A
  Guide to Concurrency, Communication and
  Multithreading
• Check out the links on the 402 web page

3
Outline

• Difficulties in Socket Programming
• Remote Procedure Calls (RPC) concepts
• SUN RPC
• Remote Object Invocation

4
Difficulties in Socket Programming

• Data representation
• Binding

5
Data Representation

• Complex data structures must be converted
• Sender must flatten complex data structures
• Receiver must reconstruct them
• Sender and receiver must agree on a common format
  for messages.

6
Example

• typedef struct
• char NameMAXNAMELENGTH
• float Salary
• char JobCodeIDNUMLENGTH
• Employee
• You may want to send this information to the
  server. Using
• send(s, (void ) e, sizeof(e), 0)
• where e is of type Employee is most likely not
  going to work.
• The sender and receiver must agree on a format
  for the message.

7
Data Representation

• Sender and receiver must agree on type of
  message. This can can be quite difficult.
• Sender must convert data to send to the agreed
  upon format. This often requires a flattening
  of the data structure representing this data.
• Receiver should parse the incoming message.

8
Data Representation

• Useful functions
• sprintf() Used to convert data items to
  characters. An example is the following
• sprintf(msg, s f s,name,salary,jobcode)
• sscanf() Retrieves data items from a string. An
  example is the following
• sscanf(msg, s f s,name,salary,jobcode)

9
Data Representation

• Code segment for marshalling the Employee
  structure
• char msg
• char nameMAXNAMELENGTH
• char jobcodeMAXNAMELENGTH
• float salary
• int msglength
• Employee e
• .
• salary GetSalaryFromEmployee(e)
• GetJobCodeFromEmployee(e,jobcode)
• GetNameFromEmployee(e,name)
• msg (char ) malloc(sizeof(Employee))
• sprintf(msg,"s f s",name,salary,jobcode)
• ..
• msglength sizeof(Employee)
• send(s, (void ) msg, msglength))

10
Data Representation

• Code segment for unmarshalling the Employee data
  sent
• char msg
• char nameMAXNAMELENGTH
• char jobcodeMAXNAMELENGTH
• float salary
• int msglength
• Employee e
• msg (char ) malloc(sizeof(Employee))
• msglength sizeof(Employee)
• recv(connectfd, (char ) msg, msglength,0)
• sscanf(msg, s f s, name, salary, jobcode)

11
Other Representational Issues

• Usually in a client-server model, the server
  provides a set of services. Each service can be
  invoked by a procedure. For example, in an
  employee management system you would have
  services that include
• Insert an employee record for a new employee
• Get the salary of an employee
• Etc
• The client must identify the service that it
  wants.
• This is part of the message.

12
Other Representational Issues

• Thus each service must have an identifier e.g.,
  in the previous code segment examples we may have
  something like this
• sprintf(msg,d s f s",methodidentifier,
• name,salary,jobcode)
• sscanf (msg,d s f s",methodidentifier,
• name,salary,jobcode)

13
Other Representational Issues

• What if we have a list of Employees that we want
  to send in a message. We do not know ahead of
  time how many employee records will be sent in a
  message.
• There are different ways to handle this. One way
  is to send a message with the service identifier
  and the number of employee records being sent.
  You then send the number of employee records.

14
Other Representational Issues

• What was just described works fine if the client
  and server machines have similar machine types.
• However, it is common that there are multiple
  machine types.
• IBM mainframes use the EBCDIC character code, but
  IBM PCs use ASCII.
• It would be rather difficult to pass a character
  parameter from an IBM PC client to an IBM
  mainframe server using what has just been
  described.
• Similar problems occur with integers (1s
  complement vs twos complement).

15
Other Representational Issues

• Need a canonical form
• For the UNIX OS there is XDR(eXternal Data
  Representation). For Java RMI, there is Java
  Remote Method Protocol (JRMP).

16
Other Representational Issues

• Converting from local representation to XDR
  representation is called Encoding.
• Converting from XDR representation to local
  representation is called Decoding.
• SUN RPC uses XDR More later.

Sender
Receiver
XDR
ENCODE
DECODE
Functions are provided for the encoding and
decoding Standard representation of datatypes.
17
Other Representational Issues

• XDR Example
• Assume that you want to transmit a long integer.
  You need something as follows
• XDR xdrs
• long i
• xdrstdio_create(xdrs,stdout,XDR_ENCODE)
• if (!xdr_long(xdrs,i))

18
Binding

• Binding refers to determining the location and
  identity (communication identifier) of the callee
• In UNIX, a communication identifier is a socket
  address containing hosts IP address and port
  number.

19
Binding

• Strategies for binding
• Static binding (which binds the host address of a
  server into the client program at compilation
  time) is undesirable.
• The client and server programs are compiled
  separately and often at different times.
• Services may be moved from one host to another.
• Could pass host name and port by reading a file
  or through the command line.
• You do not need to recompile
• You still need to know the location of the server
  ahead of time.

20
Binding

• Strategies for binding (cont)
• Always run the binder on a well-known address
  (i.e., fixed host and port)
• The operating system supplies the current address
  of the binder (e.g., via environment variable in
  UNIX).
• Users need to be informed whenever the binder is
  relocated
• Client and server programs need not be recompiled
• Use a broadcast message to locate the binder
• The binder can be easily relocated
• Client/Server programs need not be recompiled

21
Binding

• Dynamic binding service is desirable. Ideally, a
  binding service would have the following
  characteristics
• Allows servers to register their exporting
  services
• Allows servers to remote services
• Allows clients to lookup the named service
• We will come back to this later when looking at
  specific examples.

22
Difficulties in Socket Programming

• Using sockets does not conceal communication
  which is important to achieve access transparency
  (defined as hiding differences in data
  representation and how a resource is accessed).
• Little was done until a paper by Birell and
  Nelson (1984). They suggested
• Allow programs to call procedures located on
  other machines.
• Sounds simple, but the implementation is actually
  quite difficult.
• This approach is known as Remote Procedure Call
  (RPC).

23
Remote Procedure
Client
Server
blah, blah, blah bar foo(a,b) blah, blah,
blah
int foo(int x, int y ) if (xgt100) return(y-2)
else if (xgt10) return(y-x) else
return(xy)
protocol
24
Introduction to RPC

• An extension of conventional procedure call (used
  for transfer of control and data within a single
  process)
• Allows a client program to call procedures in a
  different address space in the same or remote
  machine.

25
Introduction to RPC

• Ideal for the client-server modeled applications
• Higher level of abstraction for interprocess
  communication
• The goal is to make distributed programming
  easier.
• Want transparent integration with the programming
  language.
• The calling procedure should not be aware that
  the called procedure is executing on a different
  machine.
• We will primarily focus on SUN RPC
• Can use TCP or UDP
• Best known

26
RPC System Components

• Stub procedures
• A stub is a communications interface that
  implements the RPC protocol and specifies how
  messages are constructed and exchanged
• Responsible for packing and unpacking of
  arguments and results this is referred to as
  marshalling.
• Automatically generated by stub generators or
  protocol compilers (more later).

27
RPC System Components

• Client stub
• Marshalling Packs the arguments with the
  procedure name or identifier into a message (this
  is instead of activation records)
• Sends the message to the server and then awaits a
  reply message
• Unpacks the results and returns them to the
  client
• Server Stub
• Receives a request message
• Unmarshalling Unpacks the arguments and calls
  appropriate procedure
• When it returns, packs the result and sends a
  reply message back to client.

28
RPC System Components
IPC runtime
IPC runtime
client
stub
stub
Server
send receive
invoke return
packargs unpackresult
receive send
unpackargs packresult
invoke work return
We do not want to write the stub code ourselves
29
Passing Pointers

• What about pointers?
• Very difficult.
• Pointers are meaningful only within the address
  space of the process in which it is being used.
• Could eliminate all together, but this is not
  necessary.
• One strategy
• Assume a parameter that is a pointer to an array
  of characters and that this arrays size is
  known.
• Copy the array into the message and send to
  server. Server changes the data, sends it back to
  the client and the client copies.
• Cant deal with complex data structures e.g. a
  graph

30
RPC Interface Definition Language and Compiler

• How are stubs created?
• Servers provide one or more services to client
  programs
• Services are encapsulated and their clients
  interact with them only via interfaces
• An interface definition language (IDL) is used to
  define these interfaces which are also known as
  service interfaces.

31
RPCGEN

• There is a tool for automating the creation of
  RPC clients and servers.
• The program rpcgen does most of the work for you.
• The input to rpcgen is an interface
  specification.

32
RPCGEN
Protocol Description
Input File
rpcgen
Server skeleton
XDR filters
header file
Client Stubs
C Source Code
33
Interface Definition Language

• The input file defines an interface
• The language used to describe the interface is
  based on XDR and is called the Interface
  Definition Language.
• An interface contains a program number, a version
  number, procedure definitions and required type
  definitions.
• A procedure definition specifies a procedure
  declaration and a procedure number.

34
Interface Definition Language

• Only a single input and output parameter is
  allowed.
• rpcgen compiles interface definitions into stubs,
  header files and main server source code.

35
Example

• We will illustrate SUN RPC by converting a simple
  local service for generating pseudorandom numbers
  into a remote service.
• This is based on the drand48 and srand48
  functions.
• Prior to invoking drand48, a program must
  initialize a starting value by calling the
  srand48 function with a long parameter value
  called the seed.
• The seed determines the starting position in a
  predetermined sequence of pseudorandom numbers.
• After initializing the generator by invoking
  srand48, call drand48 to return successive values
  in a sequence of pseudorandom values that are
  uniformly distributed in the interval 0,1).

36
Example

• There is one file that we will call
  pseudorandom.c
• include "rand.h"
• void initialize_random(long seed)
• srand48(seed)
• double get_next_random(void)
• return drand48()

37
Example

• A program that uses these functions is in main.c
  and a segment looks like this
• .
• myseed (long)atoi(argv1)
• iters atoi(argv2)
• initialize_random(myseed)
• for (i 0 i lt iters i)
• printf("d f\n",i, get_next_random())
• exit(0)
• Please note that the seed value and the number of
  iterations are command line arguments.

38
Example

• Now let us see how to make initialize_random and
  get_next_random remote functions.
• We first provide a specification (XDR) file for
  the remote service. This specifies the
  interface.
• The XDR file has a .x extension.

39
Example

• rand.x is the following
• program RAND_PROG
• version RAND_VERS
• void INITIALIZE_RANDOM(long) 1
• double GET_NEXT_RANDOM(void) 2
• 1
• 0x31111111

40
Program Identifiers

• The Sun convention for program numbers if the
  following
• 0x00000000 - 0x1fffffff (Sun)
• 0x20000000 - 0x3fffffff (User)
• 0x40000000 - 0x5fffffff (transient)
• 0x60000000 - 0xffffffff (reserved)

41
Procedure Identifiers Program Version Numbers

• Procedure identifiers usually start at 1 and are
  numbered sequentially
• Version numbers typically start at 1 and are
  numbered sequentially.
• The function names are the same as those before
  except that are all in uppercase.
• The functions are numbered so that the
  initialize_random function is service number 1
  and get_next_random is service number 2 in the
  server.

42
Example

• Now to use rpcgen as follows
• rpcgen C a rand.x
• The C option indicates ANSI C and the a option
  tells rpcgen to generate all of the supporting
  files.

43
Example

• Files generated include
• makefile.rand This file is the makefile for
  compiling all of the client and server code.
• rand_clnt.c This file contains the client stub,
  which is usually not motified.
• rand_svc.c This file contains the server stub,
  which is usually not modified.
• rand.h This header file contains all of the XDR
  types from the specification.
• rand_client.c This file contains a skeleton
  client main program with dummy calls to the
  remote service. You insert code to set up the
  argument values for the remote service before the
  dummy call.

44
Example

• Files generated include (continued)
• rand_server.c This file contains the stubs for
  the remote services. Insert the code for the
  local version of the services into these stubs.
• rand_xdr.c If this file is generated, it
  contains XDR filters (routines) needed by the
  client and server stubs.

45
Example

• You can now modify the rand_client.c file to
  contain the client code.
• You then modify the rand_server.c file to contain
  the functions to be called remotely.

46
Example

• This is the rand_client.c generated by rpcgen.
• include "rand.h"
• include ltstdio.hgt
• include ltstdlib.hgt / getenv, exit /
• void
• rand_prog_1(char host)
• CLIENT clnt
• void result_1
• long initialize_random_1_arg
• double result_2
• char get_next_random_1_arg

47
Example

• ifndef DEBUG
• clnt clnt_create(host, RAND_PROG, RAND_VERS,
  "netpath")
• if (clnt (CLIENT ) NULL)
• clnt_pcreateerror(host)
• exit(1)
• endif / DEBUG /
• result_1 initialize_random_1(initialize_random
  _1_arg,clnt)
• if (result_1 (void ) NULL)
• clnt_perror(clnt, "call failed")
• result_2 get_next_random_1((void
  )get_next_random_1_arg,
  clnt)
• if (result_2 (double ) NULL)
• clnt_perror(clnt, "call failed")
• ifndef DEBUG
• clnt_destroy(clnt)
• endif / DEBUG /

48
Example

• main(int argc, char argv)
• char host
• if (argc lt 2)
• printf("usage s server_host\n", argv0)
• exit(1)
• host argv1
• rand_prog_1(host)

49
Example

• In the rand_prog_1, we take note of the
  following
• The clnt_create call generates a handle for the
  remote service. If it fails, a NULL pointer is
  returned. Returns the clnt pointer which is the
  handle (communication information) for the
  remote service.
• The RAND_PROG and RAND_VERS parameters are the
  program and version names specified in rand.x
• The netpath parameter indicates that the
  program should look for an available network
  transport mechanism as specified by the NETPATH
  environment variable.
• The converted remote calls to initialize_random
  and get_next_random have the version number
  appended to the function names i.e.,
  initialize_random is called as initialize_random_1
  .
• In the remote calls, the parameters and return
  values are designated by the pointers.
• The clnt pointer is deallocated by clnt_destroy.

50
Example

• The following is a revised version of the main
  function in rand_client.c
• include "rand.h"
• include ltstdio.hgt
• include ltstdlib.hgt / getenv, exit /
• main(int argc, char argv)
• int iters, i
• long myseed
• CLIENT clnt
• void result_1
• double result_2
• char arg

51
Example

• if (argc ! 4)
• fprintf(stderr, "Usage s host seed
  iterations\n, argv0)
• exit(1)
• clnt clnt_create(argv1,RAND_PROG,RAND_VERS,"ne
  tpath")
• if (clnt (CLIENT ) NULL)
• clnt_pcreaterror(argv1)
• exit(1)

52
Example

• myseed (long)atoi(argv2)
• iters atoi(argv3)
• result_1 initialize_random_1(myseed,clnt)
• if (result_1 (void ) NULL)
• clnt_perror(clnt, "call failed")
• for (i 0 i lt iters i)
• result_2 get_next_random_1((void )arg,
  clnt)
• if (result_2 (double ) NULL)
• clnt_perror(clnt, "call failed")
• else
• printf("5d f\n", i, result_2)
• clnt_destroy(clnt)
• exit(0)

53
Example

• The revised version of rand_client.c is a
  combination of the main program of the generated
  rand_client.c and the main program used in the
  local version.
• Start with the original program for local service
  and insert the call to create_client near the
  beginning and clnt_destroy at the end.
• The host name is passed as the first command line
  argument.
• The main program calls the remote functions
  directly, so there is no need for rand_prog_1.
• The next change is to convert the calls to
  initialize_random and get_next_random from local
  to remote calls.
• The remote functions pass their parameters by
  pointer and return a pointer to the return value.
  The clnt handle is passed as an additional
  parameter in the class.

54
Example

• The skeleton code for rand_server.c is presented
  here.
• include "rand.h"
• include ltstdio.hgt
• include ltstdlib.hgt / getenv, exit /
• include ltsignal.hgt
• void
• initialize_random_1_svc(long argp, struct
  svc_req rqstp)
• static char result
• /
• insert server code here
• /
• return((void ) result)

55
Example

• double
• get_next_random_1_svc(void argp, struct svc_req
  rqstp)
• static double result
• /
• insert server code here
• /
• return (result)

56
Example

• Server code that gets inserted for
  initialize_random_1 is the following
• srand48(argp)
• result (void ) NULL
• Server code that gets inserted for
  get_next_random_1 is the following
• result drand48()

57
Example

• You can now create two executables rand_client
  and rand_server for the client and server
  respectively using the following
• make f makefile.rand
• The following registers the server
• rand_server
• This causes the service to be registered on the
  current host and ready to receive remote
  requests.
• Exactly with what does the server register
  with?

58
Dynamic Port Mapping

• Servers typically do not use well known protocol
  ports!
• Clients know the program identifier (and host IP
  address).
• SUN RPC includes support for looking up the port
  number of a remote program.

59
Port Lookup Service

• A port lookup service runs on each host that
  contains RPC servers.
• RPC servers register themselves with this
  service
• "I'm program 17 and I'm looking for requests on
  port 1736"

60
The portmapper

• Each system which will support RPC servers runs
  a port mapper server that provides a central
  registry for RPC services.
• Servers tell the portmapper what services they
  offer when they register.
• Basically, an RPC server registers itself with
  this service and pass information along the
  following lines
• I am program 17, version 5 and Im looking for
  requests on port 1736
• The port number is often randomly chosen.

61
More on the portmapper

• Clients ask a remote port mapper for the port
  number corresponding to a program identifier and
  version.
• The portmapper is available on a well-known port
  (111).

62
RPC Semantics

• Major difference between an RPC and a
  conventional procedure call is the number of ways
  the RPC may fail.
• During an RPC, problems may occur
• Request message may be lost
• Reply message may be lost
• Server and/or client may crash
• In the last two cases the procedure may have been
  called.

63
RPC Semantics

• Some strategies for different RPC message
  delivery guarantees
• Retry request message -- retransmit the request
  message until either a reply is received or the
  server is assumed to have failed.
• Duplicate filtering -- Filtering duplicate
  requests at the server when retransmissions are
  used.
• Retransmission of replies -- Keep a history of
  reply messages to enable lost replies to be
  retransmitted without re-executing the server
  operations

64
RPC Semantics

• RPC mechanisms usually include timeouts to
  prevent clients waiting indefinitely for reply
  messages
• RPC call semantics
• Semantics achieved depends on how failures are
  dealt with
• maybe call semantics
• at-least-once call semantics
• at-most-once call semantics
• cannot achieve exactly-once call semantics

65
RPC Semantics

• maybe call semantics
• No retransmission of request messages
• Not certain whether the procedure has been
  executed
• No fault-tolerance measures
• Generally not acceptable

66
RPC Semantics

• at-least-once call semantics
• Request messages are repeatedly sent after
  timeouts until it either gets a reply message or
  some maximum number of retries have been made.
• No duplicate request message filtering
• The remote procedure is called at least once if
  server not down
• The client does not know how many times the
  remote procedure has been called
• Unless called procedure is idempotent (i.e.
  repeatable), this could produce undesirable
  results (e.g., money transfer).
• A function such as deposit(DavesAccount,100)is
  not idempotent.

67
RPC Semantics

• at-most-once call semantics
• Retransmission of request messages
• Duplicate request message filtering
• If server does not crash and client receives
  result of call, then the procedure has been
  called exactly once, otherwise an exception is
  reported and the procedure will have been called
  either once or not at all.
• This works for both idempotent and non-idempotent
  procedures.
• Complex support needed.
• Request identifiers ensure only retransmissions
  filtered and not new sendings of the same request.

68
SUN RPC Semantics

• The use of UDP provides maybe semantics.
• The use of TCP provides at most once semantics.

69
DCE RPC

• Distributed Computing Environment (DCE) is from
  OSF.
• DCE is a middleware system that is designed to
  execute as a layer of abstraction between
  existing (network) operating systems and
  distributed applications.
• Initially designed for UNIX it has now been
  ported to all major operating systems.

70
Goals of DCE RPC

• The RPC systems makes it possible for a client to
  access a remote service by simply calling a local
  procedure. This is similar to SUN RPC.
• Semantic options
• At-most-once operation
• Idempotent

71
Binding a Client to a Server in DCE

• Client-to-server binding in DCE.

2-15
72
Remote Object Invocation

• The idea of RPCs applies to invocations on
  objects.
• Object model systems include
• Javas RMI
• CORBA
• DCOM

73
Overview of RMI

• RMI provides a Naming Service through the RMI
  Registry that simplifies how programs specify the
  location of remote objects.
• This naming service is a JDK utility called
  rmiregistry that runs at a well known address (by
  default).

74
Overview of RMI Programming

• Define an interface that declares the methods
  that will be available remotely.
• The server program must include a class that
  implements this interface.
• The server program must create a remote object
  and register it with the naming service.
• The client program creates a remote object by
  asking the naming service for an object
  reference.

75
Server Details Extending Remote

• Create an interface that extends the
  java.rmi.Remote interface.
• This new interface includes all the public
  methods that will be available as remote methods.

76
Sample Interface

• import java.rmi.
• // The interface must extend the Remote interface
  to become something // that RMI can serve up.
• public interface RemoteMath extends Remote
• public int Add(int x, int y) throws
  RemoteException
• public int Sub(int x, int y) throws
  RemoteException
• public int Mult(int x, int y) throws
  RemoteException
• public int Div(int x, int y) throws
  RemoteException

77
Server Details Implementation Class

• Create a class that implements the interface.
• The class should also extend UnicastRemoteObject
• This class needs a constructor that throws
  RemoteException !
• This class is now used by rmic to create the stub
  and skeleton code.

78
Example Interface Implementation

• import java.net.
• import java.rmi.
• import java.rmi.server.UnicastRemoteObject
• public class RemoteMathImpl extends
  UnicastRemoteObject implements RemoteMath
• public RemoteMathImpl() throws RemoteException
• super()

79
Example Interface Implementation

• public static void main(String args)
• try
• // create a RemoteMathImpl object
• RemoteMathImpl r new RemoteMathImpl()
• // register the object with the rmi
  registry
• // (registry must already be running)
• Naming.rebind("ReMath",r)
• System.out.println("Registered
  RemoteMath object\n")
• catch (RemoteException e)
• System.out.println("RemoteException "
  e.getMessage())
• e.printStackTrace()
• catch (MalformedURLException m)
• System.out.println("URL Problem "
  m.getMessage())
• m.printStackTrace()

80
Example Interface Implementation

• public int Add(int x, int y) throws
  RemoteException LogRequest("ADD " x ""
  y)
• return(xy)
• ..

81
Generating stubs and skeleton

• Compile the remote interface and implementation
• gt javac RemoteMath.java RemoteMathImpl.java
• Use rmic to generate RemoteMathImpl_stub.class,
  RemoteMathImpl_skel.class
• gt rmic RemoteMathImpl

82
Server Detail main()

• The server main() needs to
• create a remote object.
• register the object with the Naming service.
• public static void main(String args)
• try
• RemoteMathImpl r new RemoteMathImpl()
• Naming.bind(ReMath,r)
• catch (RemoteException e)
• . . .

83
Client Details

• The client needs to ask the naming service for a
  reference to a remote object.
• The client needs to know the hostname or IP
  address of the machine running the server.
• The client needs to know the name of the remote
  object.
• The naming service uses URLs to identify remote
  objects.

84
Using The Naming service

• Naming.lookup() method takes a string parameter
  that holds a URL indicating the remote object to
  lookup.
• rmi//hostname/objectname
• Naming.lookup() returns an Object!
• Naming.lookup() can throw
• RemoteException
• MalformedURLException

85
Getting a Remote Object

• try
• Object o Naming.lookup(uri//csd.uwo.ca/Re
  Math)
• RemoteMath r (RemoteMath) o
• // . . . Use r like any other Java object!
• catch (RemoteException re)
• . . .
• catch (MalformedURLException up)
• throw up

86
Starting the Server

• First you need to run the Naming service server
• gt rmiregistry
• Now run the server
• gt java RemoteMathImpl

87
Summary

• Potentially the use of remote procedure calls can
  abstract many of the details involved in writing
  networked applications.
• Learning curve is often high.