IO Multiplexing

1
IO Multiplexing

• IO Models
• select()

2
I/O Models

• Blocking
• Non-Blocking
• IO Multiplexing
• Signal-driven IO
• Asynchronous IO

3
IO Models

• Two phases
• Waiting for the data
• Copying the data

4
Blocking I/O
application
kernel
System call
recvfrom
No datagram ready
Wait for data
Process blocks in a call to recvfrom
Datagram ready copy datagram
Copy data from kernel to user
Return OK
Process datagram
Copy complete
5
Nonblocking I/O
application
kernel
System call
recvfrom
No datagram ready
EWOULDBLOCK
System call
recvfrom
No datagram ready
Wait for data
EWOULDBLOCK
Process repeatedly call recvfrom wating for an
OK return (polling)
System call
recvfrom
datagram ready
copy datagram
Copy data from kernel to user
Return OK
Process datagram
application
6
I/O multiplexing(select and poll)
application
kernel
System call
select
No datagram ready
Process block in a call to select waiting for one
of possibly many sockets to become readable
Wait for data
Return readable
Datagram ready copy datagram
System call
recvfrom
Copy data from kernel to user
Process blocks while data copied into
application buffer
Return OK
Process datagram
Copy complete
7
signal driven I/O(SIGIO)
application
kernel
Sigaction system call
Establish SIGIO
Process continues executing
Signal handler
Return
Wait for data
Deliver SIGIO
Signal handler
Datagram ready copy datagram
System call
recvfrom
Copy data from kernel to user
Process blocks while data copied into
application buffer
Return OK
Process datagram
Copy complete
8
asynchronous I/O
application
kernel
System call
aio_read
No datagram ready
Return
Wait for data
Process continues executing
Datagram ready copy datagram
Copy data from kernel to user
Delever signal
Signal handler Process datagram
Copy complete
Specified in aio_read
9
Comparison of the I/O Models
asynchronous I/O
I/O multiplexing
signal-driven I/O
blocking
nonblocking
initiate
check
check
initiate
check
check
wait for data
blocked
check
check
check
ready initiate
notification initiate
blocked
blocked
blocked
copy data from kernel to user
notification
complete
complete
complete
complete
ist phase handled differently, 2nd phase handled
the same
handles both phases
10
select()

• int select( int maxfd,
• fd_set readset,
• fd_set writeset,
• fd_set excepset,
• const struct timeval timeout)
• maxfd highest number assigned to a descriptor.
• weadset set of descriptors we want to read from.
• writeset set of descriptors we want to write to.
• excepset set of descriptors to watch for
  exceptions.
• timeout maximum time select should wait

11
struct timeval

• struct timeval
• long tv_usec / seconds /
• long tv_usec / microseconds /
• struct timeval max 1,0

12
Condition of select function

• Wait forever return only when descriptor is
  ready (timeval NULL)
• wait up to a fixed amount of time
• Do not wait at all return immediately after
  checking the descriptors (timeval 0)

13
Select Function

• Readset
• descriptor for checking readable
• Writeset
• descriptor for checking writable
• Exceptset
• descriptor for checking two exception conditions
• arrival of out of band data for a socket
• the presence of control status information to be
  read from the master side of a pseudo terminal

14
Descriptor sets

• fd_set an array of integers, with each bit in
  each integer corresponding to a descriptor.
• Void FD_ZERO(fd_set fdset) / clear
  all bits in fdset /
• Void FD_SET(int fd, fd_set fdset) / turn on
  the bit for fd in fdset /
• Void FD_CLR(int fd, fd_set fdset) / turn off
  the bit for fd in fdset/
• int FD_ISSET(int fd, fd_set fdset)/ is the
  bit for fd on in fdset ? /

15
Example of Descriptor sets

• fd_set rset
• FD_ZERO(rset)/all bits off initiate/
• FD_SET(1, rset)/turn on bit fd 1/
• FD_SET(4, rset) /turn on bit fd 4/
• FD_SET(5, rset) /turn on bit fd 5/

16
Maxfdp1

• specifies the number of descriptors to be tested.
• Its value is the maximum descriptor to be tested,
  plus one
• example fd 1,2,5 ? maxfdp1 6
• constant FD_SETSIZE defined by including
  ltsys/select.hgt, is the number of descriptors in
  the fd_set datatype.(1024)

17
When is the descriptor ready for reading?

• Number of bytes in socket receive buffer gt
  low-water mark of the socket. It defaults to 1
  for TCP and UDP sockets
• The read half of the connection is closed (i.e.,
  a TCP connection that has received a FIN)
• The socket is a listening socket and the number
  of completed connections is nonzero.
• A socket error is pending. A read operation on
  the socket will not block and will return an
  error (1) with errno set to the specific error
  condition.
• These pending errors can also be fetched and
  cleared by calling getsockopt and specifying the
  SO_ERROR socket option.

18
When the socket is ready for writing?

• The number of bytes of available space in the
  socket send buffer is greater than or equal to
  the current size of the low-water mark for the
  socket send buffer and eit
• The write half of the connection is closed. A
  write operation on the socket will generate
  SIGPIPE
• A socket using a non-blocking connect has
  completed the connection, or the connect has
  failed
• A socket error is pending. A write operation on
  the socket will not block and will return an
  error (1) with errno set to the specific error
  condition.
• These pending errors can also be fetched and
  cleared by calling getsockopt with the SO_ERROR
  socket option.

19
When is the socket descriptor returned in
exception list?

• A socket has an exception condition pending if
  there is out-of-band data for the socket
• or the socket is still at the out-of-band mark

20
Condition that cause a socket to be ready for
select
Condition
Readable?
writable?
Exception?
Data to read read-half of the connection
closed new connection ready for listening socket

Space available for writing write-half of the
connection closed

Pending error



TCP out-of-band data
21
Condition handled by select in str_cli
client
select() for readability on either standard input
or socket
Data of EOF
stdin
Socket

error
EOF
TCP
RST
data
FIN
22
Three conditions are handled with the socket

• Peer TCP send a data,the socket becomr readable
  and read returns greater than 0
• Peer TCP send a FIN(peer process terminates), the
  socket become readable and read returns
  0(end-of-file)
• Peer TCP send a RST(peer host has crashed and
  rebooted), the socket become readable and returns
  -1 and errno contains the specific error code

23
Implimentation of str_cli function using select
Void str_cli(FILE fp, int sockfd) int maxfdp1
fd_set rset char sendlineMAXLINE,
recvlineMAXLINE FD_ZERO(rset) for ( )
FD_SET(fileno(fp), rset) FD_SET(sockfd,
rset) maxfdp1 max(fileno(fp), sockfd)
1 Select(maxfdp1, rset, NULL, NULL,
NULL) Continue..
if (FD_ISSET(sockfd, rset)) / socket is
readable / if (Readline(sockfd, recvline,
MAXLINE) 0) err_quit("str_cli server
terminated prematurely") Fputs(recvline,
stdout) if (FD_ISSET(fileno(fp), rset))
/ input is readable / if (Fgets(sendline,
MAXLINE, fp) NULL) return / all done
/ Writen(sockfd, sendline, strlen(sendline))
//for //str_cli
24
Stop and waitsends a line to the server and then
waits for the reply
client
time0
time1
time2
time3
time4
time5
time6
time7
25
Batch input
Time 7
request8
request7
request6
request5
reply1
reply2
reply3
reply4
Time 8
request9
request8
request7
request6
reply2
reply3
reply4
reply5
26
Handling batch input

• The problem with our revised str_cli function
• After the handling of an end-of-file on input,
  the send function returns to the main function,
  that is, the program is terminated.
• However, in batch mode, there are still other
  requests and replies in the pipe.
• A way to close one-half of the TCP connection
• send a FIN to the server, telling it we have
  finished sending data, but leave the socket
  descriptor open for reading lt shutdown function

27
Shutdown function

• Close one half of the TCP connection
• Close function
• decrements the descriptors reference count and
  closes the socket only if the count reaches 0,
  terminate both directions of data
  transfer(reading and writing)
• Shutdown function closes just one of them
  (reading or writing)

28
Calling shutdown to close half of a TCP
connection
29
Shutdown function

• includeltsys/socket.hgt
• int shutdown(int sockfd, int howto)
• / return 0 if OK, -1 on error /
• howto argument
• SHUT_RD read-half of the connection
  closed. No more reads can be issued
• SHUT_WR write-half of the connection
  closed. Also called half-close. Buffered data
  will be sent followed by termination sequence.
• SHUT_RDWR both closed

30
Str_cli function using select and shutdown
include "unp.h" void str_cli(FILE fp, int
sockfd) int maxfdp1, stdineof fd_set rset c
har sendlineMAXLINE, recvlineMAXLINE stdine
of 0 FD_ZERO(rset) for ( ) if
(stdineof 0) // select on standard input for
readability FD_SET(fileno(fp),
rset) FD_SET(sockfd, rset) maxfdp1
max(fileno(fp), sockfd) 1 Select(maxfdp1,
rset, NULL, NULL, NULL) Continue..
31
Str_cli function using select and shutdown
if (FD_ISSET(sockfd, rset)) / socket is
readable / if (Readline(sockfd, recvline,
MAXLINE) 0) if (stdineof 1) return /
normal termination / else err_quit("str_cli
server terminated prematurely") Fputs(recvline,
stdout) if (FD_ISSET(fileno(fp), rset))
/ input is readable / if (Fgets(sendline,
MAXLINE, fp) NULL) stdineof
1 Shutdown(sockfd, SHUT_WR) / send FIN
/ FD_CLR(fileno(fp), rset) continue Writ
en(sockfd, sendline, strlen(sendline))

32
TCP echo server

• Single process server that uses select to handle
  any number of clients, instead of forking one
  child per client.

33
Data structure TCP server(1)
Before first client has established a connection
Client
fd0
fd1
fd2
fd3
0
-1
rset
0
0
0
1
1
-1
Maxfd 1 4
2
-1
fd0(stdin),1(stdout),2(stderr) fd3 gt listening
socket fd
-1
FD_SETSIZE -1
34
Data structure TCP server(2)
After first client connection is established
Client
fd0
fd1
fd2
fd3
fd4
0
4
rset
0
0
0
1
1
1
-1
Maxfd 1 5
2
-1
fd3 gt listening socket fd
-1
FD_SETSIZE -1
fd4 gt client socket fd
35
Data structure TCP server(3)
After second client connection is established
Client
fd0
fd1
fd2
fd3
fd4
fd5
0
4
rset
0
0
0
1
1
1
1
5
Maxfd 1 6
2
-1
fd3 gt listening socket fd
-1
FD_SETSIZE -1
fd4 gt client1 socket fd
fd5 gt client2 socket fd
36
Data structure TCP server(4)
After first client terminates its connection
Client
fd0
fd1
fd2
fd3
fd4
fd5
0
-1
rset
0
0
0
1
0
1
1
5
Maxfd 1 6
2
-1
Maxfd does not change
fd3 gt listening socket fd
-1
FD_SETSIZE -1
fd4 gt client1 socket fd deleted
fd5 gt client2 socket fd
37
TCP echo server using single process
include "unp.h" int main(int argc, char
argv) int i, maxi, maxfd, listenfd,
connfd, sockfd int nready, clientFD_SETSIZE
ssize_t n fd_set rset, allset char lin
eMAXLINE socklen_t clilen struct
sockaddr_in cliaddr, servaddr listenfd
Socket(AF_INET, SOCK_STREAM, 0) bzero(servaddr,
sizeof(servaddr)) servaddr.sin_family
AF_INET servaddr.sin_addr.s_addr
htonl(INADDR_ANY) servaddr.sin_port
htons(SERV_PORT) Bind(listenfd, (SA )
servaddr, sizeof(servaddr)) Listen(listenfd,
LISTENQ)
38
maxfd listenfd / initialize / maxi
-1 / index into client array / for (i
0 i lt FD_SETSIZE i) clienti -1 / -1
indicates available entry / FD_ZERO(allset) F
D_SET(listenfd, allset) for ( ) rset
allset / structure assignment / nready
Select(maxfd1, rset, NULL, NULL, NULL) if
(FD_ISSET(listenfd, rset)) / new client
connection / clilen sizeof(cliaddr) connfd
Accept(listenfd, (SA ) cliaddr,
clilen) for (i 0 i lt FD_SETSIZE i) if
(clienti lt 0) clienti connfd / save
descriptor / break if (i
FD_SETSIZE) err_quit("too many
clients") FD_SET(connfd, allset) / add new
descriptor to set / if (connfd gt maxfd) maxfd
connfd / maxfd for select / if (i gt
maxi) maxi i / max index in client array
/ if (--nready lt 0) continue / no more
readable descriptors /
39
for (i 0 i lt maxi i) / check all
clients for data / if ( (sockfd clienti) lt
0) continue if (FD_ISSET(sockfd, rset))
if ( (n Readline(sockfd, line, MAXLINE))
0) /connection closed by client
/ Close(sockfd) FD_CLR(sockfd,
allset) clienti -1
else Writen(sockfd, line, n) if (--nready lt
0) break / no more readable descriptors
/
40
Denial of service attacks

• If malicious client connect to the server, send 1
  byte of data(other than a newline), and then goes
  to sleep.
• gtcall readline, server is blocked.

41
Denial of service attacks

• Solution
• use nonblocking I/O
• have each client serviced by a separate thread of
  control (spawn a process or a thread to service
  each client)
• place a timeout on the I/O operation

42
pselect function
include ltsys/select.hgt include
ltsignal.hgt include lttime.hgt int pselect(int
maxfdp1, fd_set readset, fd_set writeset,
fd_set exceptset, const struct
timespec timeout, const sigset_t
sigmask)
pselect function was invented by Posix.1g.
43
pselect function

• struct timespec
• time_t tv_sec /seconds/
• long tv_nsec / nanoseconds /
• sigmask gt pointer to a signal mask.

44
Poll function

• Similar to select, but provide additional
  information when dealing with streams devices
• include ltpoll.hgt
• int poll(struct pollfd fdarray, unsigned
  long nfds, int timeout)
• /return count of ready descriptors, 0 on
  timeout, -1 on error/
• Self study