Flux Project

1
Flux Project

• University of Utah
• Salt Lake City
• http//www.cs.utah.edu/projects/flux/

2
The Flux Project Team

• Head Jay Lepreau
• Faculty John Carter, Matthew Flat, Bryan Ford,
  
• Wilson Hsieh, Gary Lindstrom, Joe Zachary
• Res.Staff David Andersen, Godmar Back,
• Stephen Clawson, Mike Hibler,
• Sai Susarla, Patrick Tullman
• and many other students

3
Major Projects

• Fluke model, kernel, and operating system
  (started 1995)
• Flask a high security version of Fluke
• Flick flexible, optimizing IDL compiler
  supporting interface annotation
• OS Toolkit modularized infrastructure for OSs
• Janos Java oriented Active Network OS
• Network Testbed A large scale (225 node) testbed
• Alta Fluke architecture in JVM

4
Literature

• Lepreau The Fluke Project, New Results in
  Fluke/Flask
• Spencer et al. The Flask Security Architecture
• Ford, Susarla CPU Inheritance Scheduling, OSDI
  96
• Ford et al. µ-Kernels Meet Recursive Virtual
  Machines,
• OSDI 96
• Ford User-Memory-based Kernel Objects
• Ford et al. Interface and Execution Models in
  the
• Fluke Kernel, OSDI 99
• Tullman et al User-level Checkpointing through
• Exportable Kernel State, 5. Int. Workshop
• on Object Orientation in OS, 1996
• Ford et al. The Flux OS Toolkit Reusable
  Components for OS Implementation

5
Topics of Today

• Architectural Principles
• goals
• µ-kernel
• virtual machine
• nested process model
• Kernel Object Types
• Kernel API and Execution Models
• CPU Inheritance Scheduling

6
Goals

• New approach for modular extensible OS
• Secure but flexible OS components

7
Goals

• New approach for modular extensible OS
• Secure OS components

Design

• Synthesis of µ-Kernel Virtual Machine
• Recursive stackable Virtual Machines
• Hierarchical Resource Management
• Nested Process Model

8
Concepts of µ-Kernel Systems

• Decompose a system horizontally into some servers

Memory Server
µ-Kernel
Hardware
9
Concepts of µ-Kernel Systems

• Decompose a system horizontally into some servers

Memory Server
File Server
µ-Kernel
Hardware
10
Concepts of µ-Kernel Systems

• Decompose a system horizontally into some servers

Memory Server
File Server
Print Server
...
µ-Kernel
Hardware
11
Concepts of µ-Kernel Systems

• Decompose a system horizontally into some servers

...
Application 1
Application n
Memory Server
File Server
Print Server
...
µ-Kernel
Hardware
12
Concepts of µ-Kernel Systems

• Decompose a system horizontally into some servers
• Servers run in m address spaces and in user mode

...
Application 1
Application n
user mode
Memory Server
File Server
Print Server
...
µ-Kernel
kernel mode
Hardware
13
Potential Drawbacks of µ-Kernel Systems

• Low performance due to many and slow IPCs
• Effective mapping of servers onto address spaces
• Either many efficient, specialized servers or
• one general inefficient server per service
• to fulfill the requirements of different
  applications

14
Pros of µ-Kernel Systems

• Flexibility
• build the server you really need, add it and then
  just use it
• Safety
• servers in their own address spaces should not
  harm others

15
Concepts of Virtual Machines

• Traditional Virtual Machines used for
  multiplexing expensive hardware (e.g. for testing
  new Oses)

16
Concepts of Virtual Machines

• Traditional Virtual Machines used for
  multiplexing expensive hardware (e.g. for testing
  new Oses)
• Decompose a system into vertically stackable
• Virtual Machine Monitors
• Simulators (like JAVA Virtual Machine)
• Monitors (like IBMs VM /370)

17
(No Transcript)
18
Java Virtual Machine
19
Java Application
Java Virtual Machine
20
Virtual Machine Monitors
21
The Nested Process Model
Child process is encapsulated in its parent.
Traditional Process Model
Nested Process Model
Parent Process State
Parent Process State
Child State
Child State
Child State
Child State
Parent has complete control over the child
22
The Nested Process Model
23
Benefits of Nested Process Model

• OS Modularity for each service a distinct nester
  (process)
• Extensibility modified service provided by a
  second nester
• Composition nester can be mixed and matched
• Flexible Scope service can be applied to a group
  of processes, too
• Security strong security mechanisms for free
• Strong Resource Control provided by hierarchical
  resource control
• Flexibility strict hierarchy is enabled, not
  enforced

24
State Encapsulating

• Nesting of processes includes
• Hierarchical resource management
• State Visibility
• Relativity of references

25
Hierarchical Resource Management

• Child can obtain resources only through its
  parent
• If child dies all its children die, too
• Following Resources managed directly by the
  µ-kernel
• CPU time hierarchical scheduling
• (see CPU inheritance scheduling)
• Memory (relative address spaces)
• Kernel objects (see threads, ports, etc.)

26
State Visibility
Parent Process
State of Passive Child
Active Child Process
Needed by applications like demand paging,
checkpointing, process migration,
replication,
27
Relativity of References
Parent Process
Process B
Process A

• low-level API includes no absolute
• names, privileges, or resources
• classic kernel-mediated capability model
• gives relativity of cross-domain references
• and short-circuiting of nesting layers

Reference
Object
28
Relativity of References
Parent Process
External references for Parent process
Process B
Process A
Reference
Object
29
Relativity of References
Parent Process
Internal references for Parent process
Process B
Process A
Reference
Object
30
Relative Address Spaces
31
9 Fluke Kernel Object Types

• Mutex enabling mutual exclusion between
  processes
• Cond conditional variable
• Mapping encapsulating an imported memory region
• Region encapsulating an exportable memory region
• Port server-side endpoint of an IPC
• Portset set of ports on which a server waits
• Space associates memory and threads
• Thread thread of control, associated with a
  space
• Reference cross-process handle for objecttypes
  like Mapping, Region, Port, Thread or Space

32
General Specific Interface Operations

• create, destroy, rename, point-a-reference-at
• get_objectstate, set_objectstate
• lock_mutex, unlock_mutex
• A complete description of the Fluke-interface
  see
• Ford,Hibler et al. Flexible µ-kernel environment
• http//www.cs.utah.edu/projects/flux/fluke/ in
  html or ps

33
Atomic versus Conventional Kernel API

• Atomic
• Promptness, i.e. any thread is able to manipulate
  the state of any other thread without being held
  up indefinitely, e.g. the state of a client
  waiting on a reply of a server must be accessible
  without delaying until the server will reply.
• Correctness, any state of an application thread
  Ta extracted at time t0 is independent whether Ta
  meanwhile will be destroyed, recreated,
  reinitialized with this extracted state.
• Interruptibility
• Restartability
• Conventional.

34
Execution Models

• Process Model (kernel stack per thread)
• Interrupt Model (1 kernel stack per processor)

35
API x Execution Models
Process Model
Interrupt Model
Atomic
NT
Conventional
BSD
Linux
36
API x Execution Models
Process Model
Interrupt Model
Atomic
V (Carter Univ. of Utah)
V
Mach
Mach (Draves CMU)
Conventional
BSD
37
API x Execution Models
Process Model
Interrupt Model
Fluke
Fluke
Atomic
V
V
Mach
Mach
Conventional
BSD
38
API x Execution Models
Process Model
Interrupt Model
Fluke
Fluke
L4
Atomic
V
V
Mach
Mach
empty
Conventional
BSD
39
Kernel API Operationtypes

• Trivial thread_myself() 7 completion
• Short mutex-trylock 64 completion or
• in case of exceptions
• rollback and restart
• Long mutex_lock 8 sleep for an
  extended period
• of time
• Multi-stage cond_wait, IPC 22 may sleep
  indefinitely

40
Fluke Kernel Execution Models
41
Performance of the 5 Execution Models
42
Preemption Latency
Fluke Performance
Execution Model
Latency
Schedules
average
maximal
runs
misses
28.9
7430
Process NP
7594
132
18.0
1200
7805
5
Process PP
5.14
19.6
9212
0
Process FP
30.4
7356
7348
141
Interrupt NP
18.7
1272
7531
7
Interrupt PP
43
Prototype Implementation

• µ-kernel in C
• Libraries
• libc provides client side of Common Protocols
• libnest provides server side of Common Protocols
• Nesters
• debugger
• tracer
• process manager
• virtual memory manager
• checkpointer
• file system manager
• Applications

44
Code Size and Percentage Verified
109,000
108,000
100,000
80,000
60,000
26,000
40,000
20,000
9,000
33
FreeBSD
Mach
Fluke
45
CPU Inheritance Scheduling

• Threads schedule each other via a directed
  yield()
• 1 root scheduler thread per processor
• µ-kernel dispatcher manages threads, events, CPU
  donation without making a scheduling decision

46
The Dispatcher

• Performing thread sleep, wakeup, schedule etc.
• Runs in the context of the current thread
• No notion about priority, CPU usage, clocks,
  timers etc.
• Dispatcher wakes up a scheduler thread when
• The schedulers client thread blocks
• Any event of interest to the scheduler occurs
  (i.e. new thread etc.)

47
Scheduling Example
48
Schedule() Operation

• schedule(thread, port, sensitivity)
• Sensitivity levels
• ON_BLOCK Wake scheduler any time its client
  thread blocks
• ON_SWITCH Wake scheduler only when a different
  client is
• requesting the CPU
• ON_CONFLICTWake scheduler only when two or more
  clients
• are runnable at the same time

49
Implicit Donation

• Works like schedule(), except done implicitly,
  e.g.
• thread attempting to lock a held mutex donates
  to current owner
• client donates to server for the duration of an
  RPC

50
Multiprocessor Scheduling
51
Benefits

• Hierarchical, stackable scheduling policies
• Application-specific scheduling policies
• Modular CPU usage control
• Automatic priority inheritance
• Accurate CPU usage accounting
• Naturally extendible to multiprocessors
• Supports processor affinity policies

52
Scheduling Hierarchy
53
(No Transcript)
54
Results

• Three measures
• Scheduling Behavior
• Overhead
• Implementation Complexity

55
(No Transcript)
56
(No Transcript)
57
(No Transcript)
58
Context Switches for Simple Tests
59
6 µsec
600 µsec
60
(No Transcript)