Remote Procedure Calls cont'

1
Remote Procedure Calls(cont.)

• Ref Tanenbaum/van Steen, 7.3
• Bershad et al.

2
Outline

• Fault tolerance
• Basic concepts
• Failure models
• RPC failure scenarios
• Fast RPC in SMPs
• User-level IPC
• Thread/Process scheduling

3
Fault Tolerance Concepts

• Availability - Is the system up?
• Metric probability system is up
• Reliability - Will the system remain up while I
  need it?
• Metric mean time between failures
• Safety - If something does go wrong, how bad is
  it?
• Does the entire system fail?
• Maintainability - if something breaks, how hard
  is it to fix it?
• Automatic failure detection and recovery
• Fault tolerance a rich area of research - here,
  just scratch surface

4
Failure Models

• In practice, may be difficult to distinguish
  among these
• Communication Failures
• Protocols such as TCP provide reliable
  transmission (omission failures hidden from
  client)
• Failures due to broken connections (e.g., server
  crashes) not masked - usually cause exception

5
RPC Failures

• Client cannot locate server
• Request message from client to server is lost
• Server crashes after receiving request message
• Reply message from server to client is lost
• Client crashes after sending a request

6
Client/Server Failures

• Client cannot locate server
• Server might be down
• Client stubs may be obsolete
• Can handle by simply flagging error (exception
  like divide by zero)
• Requires signal handler
• Some loss of transparency
• Request message from client to server is lost (or
  appears to be lost)
• Use timer retry request if no response received
  when timer expires
• If message really lost, retry will have right
  effect
• If message wasnt lost, distinguish retry from
  initial request, server can ignore retry message

7
Server Receives Request, Crashes

• Possible scenarios
• Could be the server crashed before the request
  was processed
• Client should simply retry request later
• Could be request processed, but server crashes
  before sending reply
• Should report failure (exception) back to client
• Client cannot tell which one occurred!
• What semantics (guarantees) are provided?
• Approach 0 Operation performed exactly once
• Cannot be implemented in general, cannot
  determine if crash occurred before or after
  requested operation was performed
• Approach 1 At least once semantics
• Retry request until response is received RPC
  done one or more times
• Idempotent request can be done any number of
  times w/o ill effects
• Approach 2 At most once semantics
• Immediately report failure to client
• From clients perspective, RPC done at most once,
  possibly not at all
• Approach 3 Guarantee nothing
• Client gets no guarantees about what happened

8
RPC Failures

• Reply message from server to client is lost
• Again, can just use a timer and retry request
• Similarities to lost request message (same issues
  of duplicate requests)
• Can flag retry requests, like before
• Can restrict to idempotent requests (hard in
  general)
• Client crashes after sending a request
• Orphaned request!
• What if client reboots, reissues request, then
  response from first request shows up?
• In general, a thorny problem
• Can develop schemes to kill orphans, but that may
  have undesirable effects (e.g., if orphan has
  locked files when it is killed)

9
Outline

• Fault tolerance
• Basic concepts
• Failure models
• RPC failure scenarios
• Fast RPC in SMPs
• User-level IPC
• Thread/Process scheduling

10
RPC on SMPs(Bershad, et. al)

• Goal Fast interprocess communication
• Context
• Shared memory multiprocessor
• Each process is a multi-threaded program using
  user-level threads
• Focus on interprocess communication between
  processes (different address spaces) within the
  same SMP
• Traditionally, RPC done through kernel

11
Kernel Based Communication

• Conventional system
• User-level threads done to avoid kernel calls
• But, IPC (e.g., remote procedure calls) done
  through kernel Slow!
• Kernel call
• Context switch overhead (switching memory
  mapping)
• Strong interdependence between user-level thread
  scheduling and kernel IPC
• E.g., RPC results in client blocking, server
  starts executing
• Defeats reasons for going to user-level threads
  in first place
• As a result, applications tend to be monolithic
  (single process) due to performance penalty of
  crossing address space boundaries
• Solution User-level RPC (UPRC) - do both IPC and
  thread management at user-level to avoid kernel
  calls

12
User-Level RPC

• Basically two ideas
• Implement RPC parameter/result passing through
  shared memory
• Manage scheduling of processes/threads to reduce
  how often a processor must switch to a new
  process (try to stay with threads within the same
  process)

13
Shared Memory RPC

• Idea is simple use shared memory between client
  and server to pass parameters and results
• Kernel used to set up shared memory segment
• Pass arguments, results through shared memory w/o
  kernel calls
• Avoids copying data between kernel and process
  (often more expensive than copying data within
  kernel or within process)
• Good idea, even in single CPU implementations
• Safety not compromised
• Uses same type checking mechanisms as before

14
Thread vs. Process Switching

• Switching execution between two threads within
  the same process is more efficient than switching
  between threads in different processes
• Immediate Switching user level threads fast
  switching processes slow because it requires
  kernel operation, changing memory mapping
  register, etc.
• Longer term Better locality for threads within
  same process relative to threads in different
  processes - cache, virtual memory (e.g., TLB)
  work better
• Optimistic reallocation policy do scheduling so
  preference given to switching among threads in
  the same process to take advantage of better
  efficiency
• Tends to give preference to clients (whos
  threads are already running) relative to servers
  (especially idle ones) assumptions
• Client has other work to do (e.g., other threads)
• Server will soon get a processor to run on
• Even so, this approach delays server in
  processing the request
• Note poor performance can result if assumptions
  are violated, e.g., server gets delayed (but
  server might have time consuming operation like
  I/O)

15
Summary

• Fault tolerant RPC
• A rich topic in its own right
• Semantics of service need to be defined
• This is an important problem completely
  satisfactory solutions may not exist or may be
  too difficult to implement
• User-Level IPC
• Shared memory for communication has merit
• Some important assumptions required with the
  proposed optimistic reallocation scheduling
  policy use with care!