Socket based ClientServer Systems

1
Socket based Client/Server Systems
2
Core Elements

• Topologies
• Remoteness
• Concurrency and Independence

These elements are at the core of distributed
computing. No amount of middleware providing
transparency will be able to hide their effects
completely. Today we will cover the first two.
3
Distributed Systems

• Client/Server Systems

server
client
request
Request processing
response
Clients initiate communication and block while
waiting for the server to process the request.
4
Distributed Systems

• Hierarchical Systems

Client and Server
Every node can be both client and server but some
play a special role, e.g. are Domain Name System
(DNS) server.
5
Distributed Systems

• Totally distributed

Client and Server
Every node IS both client and server. Watch out
peer to peer systems need not be totally
distributed!
6
Critical Points in C/S Systems
server
client
Requester locate server? Authenticate?
Synchronous vs asynchr. call?
request
processing single/multithr.? Session state?
Authentication?authorization? Privacy?
response
Scalability, Availability and Security all depend
on the answers to those questions.
7
Queuing Theory
A modern application servers performance largely
relies on the proper configuration of several
queues from network listening to threadpools etc.
Queuing theory lets us determine the proper
configurations (see resources). In general
architectures like above are very sensitive for
saturated queues. Good architectures create a
funnel from left to right and limit resources
like max. threads. Picture from
http//publib.boulder.ibm.com/infocenter/wasinfo/v
5r1//index.jsp?topic/com.ibm.websphere.base.doc/i
nfo/aes/ae/rprf_queue.html
8
Multithreading (blocking calls)
Servers which use blocking threads to handle
complete requests typically need a large number
of those threads. Context switch overhead soon
eliminates all performance gains. see Aruna
Kalaqanan et.al. http//www-128.ibm.com/developerw
orks/java/library/j-javaio
9
Non-Blocking Reactor Pattern
Server applications in a distributed system must
handle multiple clients that send them service
requests. Before invoking a specific service,
however, the server application must demultiplex
and dispatch each incoming request to its
corresponding service provider. The Reactor
pattern serves precisely this function. It allows
event-driven applications to demultiplex and
dispatch service requests, which are then
delivered concurrently to an application from one
or more clients.
The Reactor pattern is closely related to the
Observer pattern in this aspect all dependents
are informed when a single subject changes. The
Observer pattern is associated with a single
source of events, however, whereas the Reactor
pattern is associated with multiple sources of
events.
From Aruna Kalaqanan et.al. http//www-128.ibm.co
m/developerworks/java/library/j-javaio. The
downside all processing needs to be non-blocking
and the threads need to maintain the state of the
processing between handler calls (explicit state
management vs. implicit in normal multi-threaded
designs).
10
Reactor Pattern
From Doug Lea (see resources). Note that this is
the single threaded version. Symbian OS uses a
similiar concept called Active Objects. Handlers
are not allowed to block. In many cases a single
thread may be enough.
11
Handler State Machine
From Nuno Santos (see Resources). The state
machines shows clearly the influence of
non-blocking on the design of the handler which
needs to maintain device or input state by
itself. A regular thread would just handle one
request and as long as input data are incomplete
just sleep (context switch)
12
Communicating Sequential Processes (CSP)
CSP is a concept that can leads to formally
verifiable multi-threaded systems and avoids the
typical problems of multithreaded designs like
deadlock, resource starvation. live-lock and race
conditions. Data flow architectures can be easily
implemented with it. The concept allows nice
composition of parallel working steps. CSP
reduces complexity of shared data multi-threaded
designs.
Hoare, http//www.cs.virginia.edu/evans/crab/hoar
e1978csp.pdf
13
CSP channel concept
message

• channel object

prod-cons / client-server
read/accept
write/call
Process A
Process B
Process A
Process B
(a) Data-flow modeling
(b) Object oriented
channel
read/accept
read/accept
write/call
write/call
Process A
Process B
Process A
Process B
System 1
System 2
Link driver
(c) Hardware (in)dependence
(d) Distributed and heterogeneous
from Hilderink, Uni Twente, http//www.tesi.utwen
te.nl/Workshop02/Hilderink.ppt. A process in CSP
hides all data and runs in a private thread.
Input/output of requests is through queues with
or without buffering.
14
Programming Client/Server Systems with Sockets
15
Overview

• Protocol stack (TCP delivery guarantees)
• Socket primitives
• Process Model with sockets
• Example of server side socket use
• Transparency and socket programming?
• Security, Performance, Availability, Flexibility
  etc. of socket based C/S.
• Typical C/S infrastructure (Proxies, Firewalls,
  LDAP)

16
Protocol Stack for Sockets
Socket host B, port 80, tcp-conn
Socket host A, port 3500, tcp-conn
Reliable comm. channel
Transport/Session
Transport/Session
Tcp connection
Udp connection
Network
Network
Data Link
Data Link
Physical
Physical
17
TCP communication properties

• lost messages retransmitted
• Re-sequencing of out of order messages
• Sender choke back (flow control)
• No message boundary protection

These features form a reliable communication
channel. This does not include proper behavior
in case of connection failures! (timeout
problem). (Ken Birman, building secure and
reliable network applications, chapter 1)
18
TCP implementation properties

• rather expensive (high overhead) especially in
  high-speed networks
• A long path through the OS kernel (compared e.g.
  to a Virtual Interface Architecture (VIA)
  shortcut)
• Hard to configure for high-speed distributed
  applications (see Internet2 architecture)
• poor support for Quality-of-Service demands.

Despite of these points TCP is THE communication
protocol nowadays and proved to scale even on the
Web.
19
Delivery Guarantees

• Best effort (doesnt guarantee anything)
• At least once (same request several times
  received)
• At most once (not more than once but possibly
  not at all)
• Once and only once (simply the best)

In case of channel break-down TCP does NOT make
ANY delivery guarantees. This becomes your job
then (or better a job for your middleware)
Example order handling on the web! Network
break-down or client side re-send of a form cause
problems!
20
Communication Failure timeout
Case Client sends request and receives timeout
Failure Cases a) network problem, server did not
receive request b) Server problem OS did receive
request but server crashed during work c)
OS/Network problem Server finished request but
response got lost or OS crashed during send.
Client options drop request (ok in c), resend
request (ok in a and b), send request to
different server (ok in a and b). Other client
actions lead either to lost or duplicated
requests.
21
At most once implementation
server
A response is stored until client confirms
client
Request number
request
response
By adding a request number to each request the
server can detect duplicate requests and throw
them away. The server itself needs to store a
response until the client acknowledges that it
was received. This creates state on the server!
22
Socket Properties

• Using either tcp or udp connections
• Serving as a programming interface
• A specification of Host, Port, Connection
  type
• A unique address of a channel endpoint.

23
Berkeley Sockets (1)

• Socket primitives for TCP/IP.

From van Steen, Tanenbaum, Distributed Systems
24
Berkeley Sockets (2)

• Connection-oriented communication pattern using
  sockets.

From van Steen, Tanenbaum, Distributed Systems
25
Server Side Processing using Processes
Connecting on arbitrary port C
Listening on port X
Client
Server Dispatcher Process
Accept and spawn process on Port Y
Connection established between client on port C
and server on port Y
Server (process)
After spawning a new process the dispatcher goes
back to listening for new connection requests.
This model scales to some degree (process
creation is expensive and only few processes are
possible). Example traditional CGI processing in
web-server
26
Server Side Processing using Threads
Connecting on arbitrary port C
Listening on port X
Client
Server Dispatcher Process
Accept and spawn thread on Port Y
Connection established between client on port C
and server on port Y
Server (thread)
After spawning a new thread the dispatcher goes
back to listening for new connection requests.
This model scales well (thread creation is
expensive but they can be pooled) and a larger
number of threads are possible). Example servlet
request processing in servlet engine (aka
web-container)
27
Server Side Concurrency
Threaded server
Process per request
Thread
Thread
addMoney(account, value)
addMoney(account, value)
In the case of the threaded server the function
needs to be re-entrant. No unprotected global
variables. Keep state per thread on stack.
28
Designing a socket based service

• Design the message formats to be exchanged (e.g.
  http1.0 200 OK ). Try to avoid data
  representation problems on different hardware.
• Design the protocol between clients and server
• - Will client wait for answer? (asynchronous vs.
  synchr. Comm.)
• - Can server call back? ( client has server
  functionality)
• - Will connection be permanent or closed after
  request?
• - Will server hold client related state (aka
  session)?
• - Will server allow concurrent requests?

29
Stateless or Stateful Service?

• Stateless
• Scales extremely well
• Makes denial of service attacks harder
• Forces new authentication and authorization per
  request

• Stateful
• Allows transactions and delivery guarantees
• Can lead to resource exhaustion (e.g. out of
  sockets) on a server
• Needs somehow reliable hardware and networks to
  succeed.

30
Server Dangers Keeping State and expecting
clients to behave -TCP SYN flooding
client
server
server
client
SYN
SYN
Client info stored
Client info stored
SYN
SYN,ACK(SYN)
Client info stored
request
Client info stored
Client never sends request, only SYN, Server
buffer gets filled and other clients cannot
connect
31
A Client using sockets

• Define hostname and port number of server host
• Allocate a socket with host and port parameters
• Get the input channel from the socket (messages
  from server)
• Get output channel from socket (this is where
  the messages to the server will go)
• Create a message for the server, e.g. GET
  /somefile.html HTTP/1.0
• Write message into output channel (message is
  sent to server)
• Read response from input channel and display it.

A multithreaded client would use one thread to
read e.g. from the console and write to the
output channel while the other thread reads from
the input channel and displays the server
messages on the console (or writes to a file)
32
A server using sockets

• Define port number of service (e.g. 80 for http
  server)
• Allocate a server socket with port parameter.
  Server socket does bind and listen for new
  connections.
• Accept an incoming connection, get a new socket
  for the client connection
• Get the input channel from the socket and parse
  client message
• Get output channel from socket (this is where
  the messages to the client will go)
• Do request processing (or create a new thread to
  do it)
• Create a response message e.g. HTTP/1.0 2000
  \n
• Write message into output channel (message is
  sent to client)
• Read new message from client channel or close the
  connection

A bare bone server. Could be extended through
e.g. a command pattern to match requests with
processing dynamically. New commands could get
loaded dynamically as well. (Application Server)
33
Distribution Transparency with Sockets?

• Invocation The server side function cannot be
  called on the client side. Instead, socket
  operations must be used and messages defined.
• Location/Relocation/Migration If service moves,
  client breaks.
• Replication/Concurrency No support yet
• Failure No support yet
• Persistence No support yet

To be fair socket based services need to deal
with all that but they are still fairly simple to
write!
34
Infrastructure of C/S Systems
Directory
Proxy
Reverse Proxy
Authent. server
Firewalls
Load Balancer
Client (initiate)
Server (process)
Reverse Proxy cache results, end SSL session,
authenticate client Authentication server store
client data, authorize client
Load Balancer distribute
requests across servers
Directory help locate server Proxy check client
authorization, route via firewall
firewall allow outgoing calls only
35
Exercises
Using code pieces from the Java examples book we
will

• Build a generic client
• Build an echo server
• Build a http client and server
• Build a proxy/firewall

We will discuss general design issues as well!
(patterns etc)
36
Resources

• David Flanagan, Java Examples in a Nutshell,
  OReilly, chapter 5. Code www.davidflanagan.com/j
  avaexamples3
• Ted Neward, Server Based Java Programming chapter
  10, Codewww.manning.com/neward3
• Doug Lea, Concurrent Programming in Java
• Pitt, Fundamental Java Networking (Springer).
  Good theory and sources (secure sockets, server
  queuing theory etc.)
• Queuing Theory Portal http//www2.uwindsor.ca/7E
  hlynka/queue.html
• Performance Analysis of networks
  http//www2.sis.pitt.edu/jkabara/syllabus2120.htm
  (with simulation tools etc.)
• Meet the experts Stacy Joines and Gary Hunt on
  WebSphere performance (performance tools, queue
  theory etc.) http//www-128.ibm.com/developerworks
  /websphere/library/techarticles/0507_joines/0507_j
  oines.html
• Doug Lea, Java NIO http//gee.cs.oswego.edu/dl/cpj
  slides/nio.pdf Learn how to handle thousands of
  requests per second in Java with a smaller number
  of threads. Event driven programming, Design
  patterns like reactor, proactor etc.
• Abhijit Belapurkar, CSP for Java programmers part
  1-3. Explains the concept of communicating
  sequential processes used in JCSP library. Learn
  how to avoid shared state multithreading and its
  associated dangers.
• Core tips to Java NIO http//www.javaperformancet
  uning.com/tips/nio.shtml
• Schmidt et.al. POSA2 book on design patterns for
  concurrent systems.
• Nuno Santos, High Peformance servers with Java
  NIO http//www.onjava.com/pub/a/onjava/2004/09/01
  /nio.html?page3 . Explains design alternatives
  for NIO. Gives numbers of requests per second
  possible.