THREADS

1
THREADS
In its purest form, a thread represents as
little as possible it is an abstraction of only
the agent executing instructions. Thomas W.
Doeppner Jr. et al. Threads - A system for the
Support of Concurrent Programming

• presented by Maciej Janik(6730) Operating
  SystemsSeptember 20th, 2004

2
Outline

• The beginnings.
• Bringing threads to first-class objects.
• Threads Vs. Events.
• Problems with threads
• Highly effective threads.
• Cost of threads

3
The beginnings

• Did you know that Unix was NOT designed to work
  in multithreaded environment?
• not well defined semantics of fork or signals
• standard libraries in static memory
• error reporting by using environment variables
• Coroutines - an early solution
• scheduling difficult to do - use of
  non-preemptive scheduling
• not really paralles coroutines execution
• implemented blocking calls or traps
• Creation of Mach threads and other
  implementations

4
Threads come to life

• Threads make concurrency a practical concept.
• Multithreading improves preformance
• may hide slower device latency
• distribute work among multiple processors
• threads are inexpensive to create and use
• Creating thread environment
• synchronization
• scheduling
• exception handling
• interrupts
• I/O

5
Creating threaded environment

• Synchronization
• achieved by using semaphores and making thread
  sleep/wake up
• used to coordinate work between threads
• happens on user-level
• Scheduling
• the notion of processor does not exist in
  thread abstraction
• it must by preemptive

6
Creating threaded environment

• Exception
• abort execution, call proper handler to do the
  clean-up and restart in safe place
• handle monitors, semaphores
• Interrupts
• make sure that interrupt reach the proper
  addressee
• when interrupt should reach the thread
• I/O wrapper
• convert synchronous calls to asynchronous ones
• separate other threads from I/O effects

7
Creating threaded environment

• Mach thread problems with Unix
• u-area management
• direct references in kernel
• use of fixed memory address
• static data structures
• performance issues
• creating a thread is costly
• improper resource allocation can blow up the
  system
• kernel has to reserve a proper place on stack

main fix for u-area define u (current_thread()-gt
u_address) define u_thread1 uthread-gtuu_thread1
deifne u_task1 utask-gtuu_task1
some performance fixes pool of threads - reuse
instead of create large number of threads will
require hybrid coroutine-thread kernel
implementation
8
Mach kernel support threads

• Created in December 1986
• Implemented in C-Threads package
• run time libraries
• language constructs to manipulate threads
• shared variables
• mutual exclusion for critical sections
• condition variables in synchronization
• Run on uniprocessor and multiprocessor
• VAX, Encore and Sequent machines with CMU.

9
Mach thread results

• Fork/Exit Vs. Thread create/terminate

Avadis Tevanina Jr. et al. - Mach Threads and
the Unix Kernel The Battle for Control
10
First-class user-level threads
Solution ?

• User-level threads
• avoid kernel overhead on operations
• second-class objects
• appropriate scheduling almost impossible
• not properly recognized and supported by kernel
• Implementing user-level threads by kernel-level
  threads
• inflexibility
• poor performance
• blocking system calls
• scheduling and synchronization problems

11
First-class user-level threads

• Psyche threads
• kernel and thread share data structures
• kernel have access to thread state information
• thread has read access to kernel
• shared data asynchronous communication
• kernel informs thread package about scheduling
• thread accepts scheduling signals from kernel
• package can coordinate scheduling with kernel
  (e.g knowing about preemption in advance)
• user-level scheduler allowed with standard
  interface

12
First-class user-level threads
Two-minute warning before preemption
120sec. Implemented on 16MHz machine. Now
machines have 3.4GHz - more than 200 times
faster So, the current warning time should be
around 0.5sec

• Idea of virtual processor that can execute a
  user-level thread.
• Mirror the behavior of physical machine including
  memory and interrupts.
• Blocking system call protected procedure call -
  similar to RPC mechanizm.
• Prior preemption two-minute warning.
• Standard scheduler interfaces - ease of sharing
  data between dissimilar packages.

13
First-class user-level threads

• Typical Psyche thread package
• Brian D. Marsh et al. - First-Class User-Level
  Threads

14
First-class user-level threads

• Evaluation
• semantic flexibility
• kernel leaves lot of space for management on user
  level
• performance
• low operation costs, as not involing kernel
• nonblocking system calls
• blocked in kernel signalling
• coordination of scheduling and synchronization
• enabled by software interrupts
• coordination of preemption and locking
• abstraction shared among different thread packages

15
Threads Vs. Events

• Which model is better - threads or events?
• Threads grew up naturally in OS world.
• Threads are hard to program, even for experienced
  programmer.
• Threads are not well supported - libraries may
  not thread-safe.
• Events are easier alternative that is sufficient
  in most cases.
• Events avoid concurrency and are more portable.

Why events are a bad idea?
Why threads are a bad idea?
16
Threads Vs. Events

• Whats wrong with threads ?

wizards
casual
all programmers
Why events are a bad idea?
Why threads are a bad idea?
John Ousterhout - Why Threads are a Bad Idea
(for most purposes)
17
Threads Vs. Events

• Problems with threads
• synchronization
• deadlocks
• hard to debug because of complicated time/locks
  dependencies
• even simple application faces full complexity
• Events are good because
• one handler for one event
• no concurrency, preemption, synchronization
• one execution stream
• Is concurrency in your code really needed?

Why events are a bad idea?
Why threads are a bad idea?
18
Problems with threads

• Poor performance
• require skills to code efficient threads
• Restrictive control flow
• complicated flows are rare
• Heavyweight synchronization
• do events have if free (on multiple processors)?
• State management - stack overflow
• dynamicly linked stack can do
• Too general scheduling
• it is possible to have your own scheduler

Why events are a bad idea?
Why threads are a bad idea?
19
Highly effective threads

• Fixing thread model for high efficiency
• use lightweight user-level threads, as kernel
  threads are heavy
• reduce thread synchronization moving it to user
  level
• improve memory management for large number of
  threads - stack problem
• more effective asynchronous I/O mechanism
• resource-aware scheduler
• cooperative scheduling

20
Capriccio (for Linux)

• Package provides highly scalable threads -
  designed for internet services.
• Very fast context switches (Edgar Toernig
  coroutine lib) when threads voluntairly yield.
• Override blocking I/O calls in GNU libc.
• Use of latest asynchronous I/O communication
  (epoll)
• Scheduler is resource-aware and prevents
  bottlenecks
• Stack is implemented as dynamic linked stack list
  - solves the memory allocation problem

21
Capriccio

• Scheduler
• monitors application behavior and learn execution
  patterns
• can predict resource consumption as well as
  release
• construct blocking graph

Rob von Behren et al. - Capriccio Scalable
Threads for Internet Services
22
Capriccio

• Stack management
• static program stack analysis - building graphs
• handling dynamic stack growth - even for
  recursive functions
• code checkpoints for optimal memory usage

Rob von Behren et al. - Capriccio Scalable
Threads for Internet Services
23
Capriccio - evaluation

• Can handle 100.000 running threads without
  performance degradation
• Utilizes fastest I/O mechanism in Linux that is
  non-blocking
• Intelligent scheduler dispatches work considering
  resources
• By analysis of graph, it is easy to find threads
  that do not want to yield
• Package optimized for profile of long running
  internet services.

24
Capriccio - test results
Rob von Behren et al. - Capriccio Scalable
Threads for Internet Services
25
Cost of threads

• Minimum thread time
• spawn and join
• Minimum work time
• block and resume on mutex
• condition testing
• Minimum hideable latency
• two context switches

26
Questions?
27
References

• Threads. A System for Support of Concurrent
  Programming - Thomas W. Doeppner Jr. et al.
  (1987)
• Mach Threads and the Unix Kernel The Battle for
  Control - Avadis Tevanian Jr. et al. (1987)
• First-Class User-Level Threads - Brian D. Marsh
  et al. (1991)
• Capriccio Scalable Threads for Internet
  Services - Rob von Behren et al.(2003)
• Why Threads Are a Bad Idea (for most purposes)
  - John Ousterhout (1996)
• Why Events are a Bad Idea (for high-concurrency
  servers) - Rob von Behren et al. (2003)
• Cost of User and Kernel Level Threads Operations
  on Linux - W. Cohen et al. (1998 ?)