Making Remote Calls

1
Making Remote Calls

• Remote procedure calls, remote method invocations
  and their infrastructure

2
Calls vs. Messages
Procedure Or Method
Parameter
Message
A
A
B
Call (and wait)
Sends to
Call based middleware hides remote service behind
a programming language call or method. Tight
coupling and synchronous processing are a
consequence of this approach!
Message based middleware creates a new concept
the message and its delivery semantics. A message
system can always simulate a call based system
but not vice versa.
3
In-Process Calls
Application
caller
receiver
Operating System
As long as we stay within one programming
language no special middleware is required. Calls
into the OS are not Inter-process calls in most
systems.
4
In-Process calls are

• Fast
• Performed with exactly once semantics
• Type and link safe (but dll and dynamic loading
  problems)
• Either sequential or concurrent (we decide it!)
• Can assume one name and address space
• Independent of byte ordering
• Controlled in their memory use (e.g. garbage
  collection)
• Can use value or reference parameters (reference
  memory address)
• Programming language calls and not messages

5
An In-process call
1122 1123
Value
Caller pushes return address and parameters on
stack
fff0
stack
Reference
Dff0 (return addr)
Address0xFFF0
receiverss stack
S O M E S T R I N G
Callee de-references character pointer. Result is
stored in register X. After processing jumps back
to caller through return address
data
Charpointer 0xFFF0
Integer intvalue 0x1122
Char charpointer SOMESTRING Int intvalue
0x1122 Main () Int result Callfunction(charpo
inter, intvalue)
int result Callfunction(charpointer, intvalue)
Print(charpointer) Intvalue Return 0 //
store 0 in register X // make a ret
code
ReturnAddress0xDFF0
6
Local Procedure Calls
Application A
Application B
caller
receiver
Find application and function
Calling Layer (RPC)
Calling Layer (RPC)
External Data Repres.
External Data Repres.
Flatten reference parameters
Operating System
Fast IPC
Fast IPC
Some systems use a highly optimized version of
RPC called IPC for local inter-process
communication. See Helen Custer, inside Windows
NT, chapter Message passing with the LPC
Facility
7
Local Procedure calls are

• Pretty fast
• Performed with exactly once semantics
• Type and link safe if both use same static
  libraries (but dll and dynamic loading problems)
• Sequential or concurrent (caller does no longer
  control it! Receiver needs to protect himself)
• Can no longer assume one name and address space
• Still Independent of byte ordering
• Would need cross-process garbage collection
• Can only use value parameters (target process
  cannot access memory in calling process)
• No longer real programming language calls. The
  missing features must be created through messages

8
Inter-process with local calls? No way!
Caller memory
Target memory
dff0
Caller pushes return address and parameters on
stack
1122 1123
stack
Value
fff0
Reference
No access
Address0xFFF0
receivers stack var
S O M E S T R I N G
Receiver de-references character pointer. Result
is stored in register X. After processing jumps
back to caller through return address
data
Charpointer 0xFFF0
Integer intvalue 0x1122
No access
Char charpointer SOMESTRING Int intvalue
0x1122 Main () Int result Callfunction(charpo
inter, intvalue)
int result Callfunction(charpointer, intvalue)
Print(charpointer) Intvalue Return 0 //
store 0 in register X // make a ret
code
No access
ReturnAddress0xDFF0
9
Remote is not local!

• Latency
• Memory access
• Partial failure
• Concurrency

Jim Waldo et al., a note on distributed computing
(please read it for the next session!)
10
Local Inter-process call (1)
Caller memory
stack
Marshalling layer flattens references.Usually
automated using an Interface Definition Language
plus generator. LPC layer selects target process
and function.
data
Integer intvalue 0x1122
Char charpointer SOMESTRING Int intvalue
0x1122 Main () Int result Callfunction(charpo
inter, intvalue)
code
Callfunction(charpointer, intvalue)
createMessage(Callfunction,
SOMESTRING,0x1122) Return Result
sendMessage(targetProcess, Message)
stub
Operating System (sends message to target process)
11
Local Inter-process call (2)
receiver memory
Stack
Integer intvalue 0x1122
data
Char charpointer SOMESTRING Int intvalue
0x1122 Main () Int result Callfunction(charpo
inter, intvalue)
code
Marshalling layer unpacks message and calls real
function.
CallfunctionSkeleton(message) Char
charpointer getArg1(message) intvalue
getArg2(message) Return Callfunction(charpointer,
intvalue)
skeleton
Operating System (sends message to target
process). Returns result to calling process
12
Remote Procedure Calls
Application A
Application B
caller
receiver
Calling Layer (RPC)
Calling Layer (RPC)
Create delivery guarantees, e.g. at most once!
Request/Reply Protocol
Request/Reply Protocol
External Data Repres.
External Data Repres.
Operating System
Speed and reliability issues between systems
require a dedicated request/reply layer. The
external data representation layer becomes more
complicated because it has to deal with
byte-ordering and alignment differences. Most
systems use a Interface Definition Language to
enable generation of stub/skeleton code.
13
Steps of a Remote Procedure Call

• Client procedure calls client stub in normal way
• Client stub builds message, calls local OS
• Client's OS sends message to remote OS
• Remote OS gives message to server stub
• Server stub unpacks parameters, calls server
• Server does work, returns result to the stub
• Server stub packs it in message, calls local OS
• Server's OS sends message to client's OS
• Client's OS gives message to client stub
• Stub unpacks result, returns to client

From van Steen, Tanenbaum, Distributed Systems
14
Remote Inter-Process calls are

• Much slower than both local versions
• No exactly once semantics, perhaps at most once.
• Version mismatches will show up at runtime
• Concurrent (caller does no longer control it!
  Callee needs to protect himself)
• Can no longer assume one name and address space
• Affected by byte ordering
• In need of network garbage collection
• Can only use value parameters (target process
  cannot access memory in calling process)
• No longer programming language calls. The
  missing features must be created through messages

15
Remote Cross Language Calls
Application
Common Type and Behavior Specification
Calling Layer (RPC/IPC)
Request/Reply Protocol
External Data Representation
Operating System
CORBA (Common Object Request Broker Architecture)
supports different languages (OO and Non-OO
e.g.). It uses a least common denominator
approach to achieve this level of
interoperability. Compare this with message based
systems! And with Microsofts new Common Language
Runtime!
16
Remote Cross-Language calls are

• Much slower than both local versions
• No exactly once semantics, perhaps at most once.
• Version mismatches will show up at runtime
• Concurrent (caller does no longer control it!
  Callee needs to protect himself)
• Can no longer assume one name and address space
• Affected by byte ordering
• In need of network garbage collection
• Can only use value parameters (target process
  cannot access memory in calling process)
• No longer programming language calls. The
  missing features must be created through messages

17
Local inter-process vs. remote calls

• A remote calling mechanism does work locally as
  well! But due to marshalling and delivery
  guarantees it is slower!
• A high-speed local inter-process calling
  mechanism is extremely important for an operating
  system

18
Parameter Types and Semantics

• In Parameter will not be returned to caller.
  No need to copy back if reference parameter.
• Out Parameter callers instance of this
  parameter will be overwritten by this parameter
  no need to copy callers version over to receiver.
• In/Out Parameter receiver needs callers value
  to create result value for same parameter copy
  value back and forth.

The copy-back problem is a result of access
transparency. Local call-by-reference semantics
is replaced by copy-restore (copy-back)
semantics. Still, transmitting complex graphs of
objects is not easy!
19
Parameter Types and Semantics(2)
Octet 0x112233
in parameter
receiver
Caller callRemote(in octet)
0x112233
reply
An in parameter is copied over to the receiver.
The result of the remote call will not change the
variable on the callers side. This means the
octet wont be copied back from the receiver to
the caller.
20
Parameter Types and Semantics(3)
Octet empty
Before call
Octet 0x112233
After call
out parameter
receiver
Caller callRemote(out octet)
empty
reply
0x112233
An out parameter is copied from the receiver to
the caller. The caller did allocate an empty
variable before calling the receiver.
21
Parameter Types and Semantics(4)
Octet 998877
Before call
Octet 0x112233
After call
inout parameter
receiver
Caller callRemote(inout octet)
0x998877
reply
0x112233
An inout parameter is copied from the caller to
the receiver AND back. This is necessary to
simulate a call by reference semantics completely
transparent for the caller.
22
Parameter semantics in different RPCs

• SUN RPC/XDR only one in parameter. Only one
  out type which is the result of the procedure.
• Java RMI all parameters are in types. Basic
  types and non-remote, serializable objects are
  transferred by value. The return value of an RMI
  call is an out type Object References are
  inout types.

23
Marshaling
Definition flattening parameters (basic types or
objects) into a common transfer format (message).
The target site will do the transformation from
the transfer format into the original types or
objects

• Binary (sender and receiver know structure of
  every message, I.e. which type/variable is at
  what offset)
• Binary self describing (the transfer format
  contains type and variable information as well.
  Needs some reflective capabilities of the
  involved languages
• Textual, self describing (XML representation of
  types or objects, e.g. using SOAP)

The typical trade-off between speed (binary) and
flexibility (self-describing) which allows e.g.
to skip unknown parts.
24
Request-Reply Message Structure
Needed for request-reply layer and delivery
guarantees
Used by the remote dispatcher to create call to
proper method or function
Optional fields for authentication e.g. client
credentials
25
Byte-Order Problems who converts?
receiver
converts
sender
converts
(little-endian)
message
(big-endian)
(little-endian)
receiver
Use as is
(big-endian)
Using a standard network byte-order (big-endian
here) results in some unnecessary conversions
between little-endian hosts. What is the big
advantage compared with a use sender format
policy? (Hint think about new systems)
26
Delivery guarantees revisited
Re-execute request Re-transmit reply
Local /remote
Retransmit
Filter Duplicates
Semantics
Adapted from Coulouris, Distributed Systems
27
Idempotent operations

• Definition
• If you can send a request a second time without
  breaking application semantics if the request was
  already executed the first time it was sent
  then this operation is idempotent.

Example http get request. (page counter does
NOT break application semantic)
With idempotent operations you can build a
RELIABLE request/reply protocol using only
at-least-once semantics!
28
If operation is NOT idempotent

• Use message ID to filter for duplicate sends
• Keep result of request execution in a history
  list on the server for re-transmit if reply was
  lost.
• Keeping state on the server introduces the
  problem of how long to store old replies and when
  to scrap them.
• Frequently used client leases for server side
  resources

29
SUN-NFS at least once semantics without
idempotent operations
Open(/foo)
client
NFS Server
Open /foo
Error file does not exist!
Error, file does not exist
client
NFS Server
Create(/foo)
Create /foo
Reply lost
OK
(timeout)
/foo
Create /foo
client
Create(/foo)
NFS Server
Error file exists!
Error, file exists!
client
NFS Server
??(censored)!!!
30
IDL Example (Unix RPCs)

• const NL64
• struct Player
• struct DoB int day int month int year
• string nameltNLgt
• program PLAYERPROG
• version PLAYERVERSION
• void PRINT(Player)0
• int STORE(Player)1
• Player LOAD(int)2
• 0
• 105040

From W.Emmerich, Engineering Distributed Objects
Compare with Webservices WSDL format!
31
Writing a Client and a Server
2-14

• The steps in writing a client and a server in DCE
  RPC. (from van Steen, Tanenbaum, Distributed
  Systems)

32
Stubs and Skeletons
Generated in advance from IDL file Generated on
demand from class file Distributed in advance to
all clients/servers Downloaded on demand
33
Marshalling and Unmarshalling
char marshal() char msg msgnew
char4(sizeof(int)1)
strlen(name)1 sprintf(msg,"d d d d s",
dob.day,dob.month,dob.year,
strlen(name),name) return(msg) void
unmarshal(char msg) int name_len
sscanf(msg,"d d d d ",
dob.day,dob.month, dob.year,name_len)
name new charname_len1 sscanf(msg,"d d
d d s", dob.day,dob.month,
dob.year,name_len,name)

• Marshalling Disassemble data structures into
  transmittable form
• Unmarshalling Reassemble the complex data
  structure.

From W.Emmerich
34
Finding a RPC server
server
Ask portmapper for program, version
Portmapper
client
On port X!
Tell portmapper about program, version and port
service
Send procedure call to service
Start listening at port X
X
This is called binding and can be handled in
different ways (inetd, DCE, Unix portmapper)
35
Resources

• Orfali/Harkey, Client/Server programming with
  Java and CORBA
• Wolfgang Emmerich, Engineering Distributed
  Objects
• (www.distributed-objects.com)
• John Bloomer, Power Programming with RPC
• John R.Corbin, The Art of Distributed
  Applications. Programming Techniques for Remote
  Procedure Calls
• Ward Rosenberry, Jim Teague, Distributing
  Applications across DCE and Windows NT
• Helen Custer, Inside Windows NT.
• Ken Birman, Building Secure and reliable network
  applications (Draft at http//www.cs.cornell.edu/
  ken/book.pdf
• http//java.sun.com/j2ee/blueprints/apmIX.html
• Jim Waldo et.al., A note on distributed computing
  (please read for next session!)