SEDA: An Architecture for WellConditioned, Scaleable Internet Services

1
SEDA An Architecture for Well-Conditioned,
Scaleable Internet Services

• Dave Camarillo

2
Problem Overview

• High loads on web servers expose inefficiencies
  in server applications
• Response time is negatively affected by these
  inefficiencies.
• Need for graceful performance degradation under
  heavy load

3
Current Strategies

• Clustering of web servers
• Load balancers
• DNS Rotation
• Geographic separation of servers
• Multiple application instances per server
• Generally overbuilt configurations
• Sub-division of web site content

4
Thread Based Concurrency

• Provides concurrent execution of multiple client
  requests.
• Allows for concurrent blocking on multiple I/O
  requests, which can be re-ordered and optimized
  by the underlying OS and/or hardware.
• Thread creation and destruction takes time
• Context switching of threads takes time
• How do you determine how many threads to have?
  When to create them?
• When do you spend more time context switching
  then doing work?

5
Event Based Concurrency

• Often a single thread per CPU
• Work-operations are done in full
• Blocking operations are done
• a-synchronously and checked on later (when an
  event occurs or after some type of poll).
• Efficient use of CPU (reduced context switching,
  no thread creation overhead etc).

6
Affects of Current Per-Server Strategies
(Threaded Servers)

• Clients have to re-try connection establishments.
• Client threads may be blocking and not utilizing
  hardware to its fullest.
• Under heavy load, large amount of time are spent
  context switching (thus not servicing the large
  pending load, making things worse)
• Strategies display very high maximum response
  times under load.

7
SEDA, An Alternative Strategy

• SEDA uses a threaded/even-driven hybrid approach
  to concurrency.
• SEDA uses a combination of queues, thread pools
  and events, in a pipe-line structure to maintain
  performance stability under varying loads.
• Clients may connect, but will not be serviced
  until a thread becomes available to service the
  request.

8
Comparison of SEDA to Others (Haboob)

• SEDA design stabilizes loading on server itself
  (i.e. no disk load I/O spike, thread scheduling,
  a-sync I/O)
• Infrastructure can dynamically adjust the
  thread-pool size based on current demands (more
  concurrency in
• those areas that need it).

9
Haboob

• Haboob is a web-server implemented using the SEDA
  strategy.
• Haboob exhibits better network throughput then
  Apache and Flash (in mb/s) as the number of
  concurrent clients grows.

10
Haboob

• Haboob usually exhibits better or comparable
  average response time to queries.
• Haboobs maximum response time is 10x to 30x
  faster then that of Apache or Flash
• Haboobs fairness is 0.98 to 0.99 despite
  significant changes in load. Compared to the 0.80
  to 0.98 fairness of Apache.

11
Haboob Performance
12
SEDA Diagram (Haboob)
13
Conclusion

• Thru alternative concurrent software design
  techniques, greater efficiencies and performance
  can be gained from the same hardware.
• SEDA highlights the non-trivial differences in
  performance and efficiency that arise from the
  concurrency strategy selected.