Working of L4 Microkernel

1
Working of L4 Microkernel

• By
• Ashish Srivastava
• Tapas Kundu

2
L4KaPistachio

• Developed at University of Karlsruhe
• Written in C
• Focus on Performance, Portability and Reusability
• Supports Intel's IA32 and IA64, PowerPC 32bit,
  Alpha 21164, and MIPS R4000
• In near future additional support planned for
  AMD64, Power4, ARM and UltraSparc

3
Features

• Separation of API and ABI
• Earlier L4 Kernels ABIs were defined by specific
  implementation, inherently non-portable
• Version 4 strictly separates API ABI, specifies
  a set of HLL bindings which are generic across
  all architectures.
• Virtual Registers
• Notion of a Virtual Register File per thread
• These map to memory or to actual processor
  registers, allowing optimal use of hardware
  registers on all architectures.

4
Features Contd.

• Address Spaces Threads
• In Version 4 the fixed relation between address
  spaces and threads is removed.
• Kernel Interface Page
• Contains function entry points for all system
  calls as well as frequently accessed system
  information.
• New architectural features added to next
  generation processors can be directly utilized by
  replacing the page.
• Overhead is negligible when using an optimized
  dynamic linker.

5
Features Contd.

• SuperFast IPC
• L4 provides Very Limited, Extremely Flexible and
  Powerful set of mechanisms
• Threads, Address Spaces, IPC, Mapping
• Other primitives built upon these four
• Still limited by the hardware architecture
• Invocation of IPC requires change of privilege
  mode
• However, IPC is mainly used for synchronization
  and signalling of threads executing within the
  same address space. IPC can then be performed
  completely in user mode avoiding the overhead
  induced by the two unnecessary privilege level
  changes.

6
Features Contd.

• Multiprocessor Support
• Load balancing, Scheduling decisions are purely
  user-level based, thus supports implementation of
  arbitrary processor allocation policies.
• Prerequisite for SMP scalability is the
  preservation of parallelism of user applications
  within the kernel. This design rule is strictly
  followed by providing powerful orthogonal
  interfaces.
• L4KaPistachio's SMP support is experimental and
  not fully scalable yet due to a few
  coarse-grained locks, in particular in the memory
  subsystem.
• However, the critical IPC path is fully lock-free
  giving a good first performance indication.
• Multiprocessor support exists for IA32, IA64
  systems.

7
Features Contd.

• Interrupts
• Abstracts system interrupts as kernel threads and
  interrupt delivery as IPC.
• V4 completely abstracts the first-level interrupt
  controller and only provides basic primitives for
  interrupt association
• Allows for a higher level of parallelism, more
  efficient synchronization, and caching of
  interrupt controller state thereby increasing
  overall performance
• Abstracting interrupt hardware was also a
  portability requirement, since some architectures
  allow access to interrupt controllers only in
  privileged mode

8
Drivers at User Level
9
(No Transcript)
10
Interrupt Association

• Association is done via the privileged thread
  (root task) using ThreadControl.
• To associate a thread to an interrupt
• Set the pager of the hardware thread ID to the
  thread ID of the interrupt handler
• To disassociate the thread from an interrupt
• Set the pager of the hardware thread ID to the
  hardware thread ID itself

11
Features Contd.

• Privileged Threads
• 3 privileged tasks sigma0, sigma1, the root task
• All threads of a privileged task are allowed to
  execute certain system calls prohibited to
  non-privileged tasks.
• This scheme exports rights delegation completely
  to user level and supports implementation of
  arbitrary policies, such as access control lists,
  or capabilities using remote procedure calls

12
Kernel Interface Page

• Contains API and Kernel Version data, system
  descriptors including memory descriptors, and
  system-call links.
• A microkernel object
• Directly mapped through microkernel into each
  address space upon address space creation
• Cant be granted to another space, or unmapped
• Creator of address space can specify where KIP is
  to be mapped, this address will remain same
  throughout the lifetime of that address space
• Any thread can obtain KIPs address though the
  KERNELINTERFACE system call

13
Virtual Registers

• Static per-thread objects.
• 3 Classes
• Thread Control Registers (TCRs)
• Used to share information about threads between
  the kernel and user level
• Message Registers (MRs)
• Used to send messages between threads. Contains
  the message
• Buffer Registers (BRs)
• Used to specify where messages (other than MRs
  themselves) are received
• Safe from kernels point of view
• Illegal Accesses, Programming Bugs can only
  compromise the originators address space
• Generic API offers no operations for indirect
  virtual-register access

14
(No Transcript)
15
(No Transcript)
16
Thread Control Blocks (TCBs)

• State of a thread is stored in its thread control
  block
• Some state can only be modified via a controlled
  interface (system calls) (e.g. address space
  associated with the thread)
• Some state can be freely visible and modifiable
  by user-level applications without compromising
  the system
• For efficiency of access this information is put
  in a user-level TCB (UTCB)

17
(No Transcript)
18
Thread Identifiers

• Global Identifiers
• Identify a thread uniquely within the system
• Thread number and version number assigned by root
  task according to selected policy like for
  example
• Version numbers are unique to give unique Ids
• Threads numbers are grouped into tasks to allow
  upper bits to be task numbers
• Local Identifiers
• Identify a thread within an address space
• Only unique and useable within an address space
• Used for some optimisations
• Typically the address of the threads UTCB.
• Assigned by the system
• Can translate one to another

19
ThreadControl System Call

• Used to create, destroy, and modify threads
• Determines
• The global thread identifier associated with the
  thread
• The address space the thread is associated with
• The thread permitted to control scheduling
  parameters of the new thread
• The pager
• Location of the UTCB within the address spaces
  allotted UTCB area (See SpaceControl later)
• Threads can be created active or inactive.
• Inactive is used to create and manipulate a new
  address space, or allocate a new thread to an
  existing address space.

20
Task

• Task Address Space Thread
• A Task has
• Thread
• Identifier, IP, SP, Pager, Scheduler, UTCB
  Location
• Address Space
• UTCB Area, Kernel Info Page Area, Redirector
• Code, Data and Stack mapped to address space

21
IPC

• Messages
• Sent and received using the IPC system call
• Sender writes into its MR and receiver reads from
  its MR. Each Thread has 64 MRs
• A message can use some or all MRs to transfer
  untyped words, can include memory strings and
  fpages

22
L4 IPC

• Message passing is the basic IPC mechanism in L4.
• All L4 IPC is Synchronous Unbuffered
• Synchronous IPC requires an agreement between
  both the sender and the receiver. The main
  implications of this agreement is that the
  receiver is expecting an IPC and provides the
  necessary buffers. If either the sender or the
  receiver is not ready, then the other party must
  wait.
• Unbuffered IPC reduces the amount of copying
  involved and is thus the key to high performance
  IPC.

23
L4 IPC Messages

• Data can be transferred in three ways using L4
  IPC.
• In-line by-value data. A limited amount of such
  data is passed directly in registers (first 8
  words in MIPS R4k) with any remainder in a
  message buffer.
• Strings. Arbitrary out-of-line buffers which are
  copied to the receiver.
• Virtual memory mappings (by-reference data).
• Data transfer via mappings is described by
  flex-pages (a contiguous region of virtual
  address space). Alternatively, virtual memory can
  be granted mapped to the receiver and unmapped
  from the sender simultaneously.

24
(No Transcript)
25
Flex Pages (fpages)

• A contiguous region of virtual address space.
• Contains all the mapped pages within the region
  described by the flex-page.
• Fpages are required for mapping and granting
  virtual memory.

26
Sending/Receiving IPC Messages

• Necessary to decide the form of data to be sent.
• In-line data needs to be copied to a buffer first
  and must also be aligned.
• Best for small amounts of data and is useful when
  some marshalling is required anyway
• Strings avoid extra copying and can be located
  anywhere in memory (no alignment necessary) but
  require buffers to be specified (via string
  dopes).
• Buffer specifications must also be consistent.
• The receiver must specify and expect to receive
  the (at least) same number and size strings that
  the sender sends

27
Sending/Receiving IPC Messages

• IPC operations provided by C Interface
• Single send or receive versus a combined send and
  receive.
• Receive from specific sender (closed) or any
  sender (open).
• Deceiving or non-deceiving IPC.
• Certain parameters must be provided
• dest/src Identifier of message destination/source
  thread respectively.
• snd msg/rcv msg Descriptor for long part of
  message, for send/receive part of IPC
  respectively.
• snd reg/rcv reg Short part of message (artifact
  of C interface) for send/receive part of IPC
  respectively.
• timeout Timeout specification. Used to ensure
  that a thread need not be blocked indefinitely.
• result Variable to store the result status of the
  IPC operation.

28
Task Creation and Deletion
29
(No Transcript)
30
(No Transcript)
31
(No Transcript)
32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
(No Transcript)
36
(No Transcript)
37
Steps in Creating a New Task

• Create an inactive thread in a new address space
• L4_ThreadControl (task, / new tid /
• task, / new space identifier /
• me, / scheduler of new thread /
• L4_nilthread, / pager nil, inactive,
• (void ) -1) / NOP Utcb location /

38
Steps in Creating a New Task

• 2. Initialise location of KIP and UTCB area in
  new address space
• L4_SpaceControl (task,
• 0, / control (ignored on mips) /
• kip_area,
• utcb_area,
• L4_anythread, / redirector /
• control)

39
Steps in Creating a New Task

• 3. Specify the utcb location and assign a pager
  to the new thread to activate it.
• L4_ThreadControl (task, task, me,
• pager, / new pager /
• (void ) utcb_base) / utcb location /
• This results in the thread immediately waiting
  for an IPC containing the IP and SP of the new
  thread.

40
Steps in Creating a New Task

• 4. Send an IPC to the new thread with the IP and
  SP in the first two words of the message.
• This results in the thread starting at the
  received IP with the SP set as received.

41
Adding extra inactive threads toa task

• Use ThreadControl to assign new inactive
  threads to an existing address space.
• L4_ThreadControl (newtid, / new thread id /
  ExistingId, / address space identifier / me, /
  scheduler of new thread / L4_nilthread, / pager
  nil, inactive, (void ) -1) / NOP Utcb
  location /
• Note Can also add active threads

42
ExchangeRegisters

• Used to activate or modify an existing thread
  within an address space

43
Threads

• The microkernel only manages (preserves) the
  user-level IP and SP
• (and registers if preempted)
• The following is managed by user-level
  applications
• User stack location,allocation, size,deallocation
• Thread allocation,deallocation
• Entry point
• Stack corruption is very difficult to diagnose
  and debug

44
IPC Overview

• Single system call that implements several
  variants of synchronous unbuffered IPC
• Arguments determine IPC system call behaviour
• Operations are
• Send() send a message to a specified thread
• Receive() closed receive from a specific sender
  (might be an interrupt)
• Wait() open receive from any sender (incl.
  Interrupt)
• Call() send and wait for reply (usual RPC
  operation)
• Reply_and_Wait() send to a specific thread and
  wait for any new message (typical server
  operation)

45
(No Transcript)
46
IPC Registers

• Message Registers
• 64 registers
• Form a message
• Used to transfer typed items and untyped words
• Typed items
• MapItem
• GrantItem
• StringItem
• Buffer Registers
• 34 registers
• Specify where
• MapItems and GrantItems are received
• StringItems are received
• if any are permitted to be in the message

47
(No Transcript)
48
Overview of IPC operations

• L4_Ipc system call performs all IPC operations
  (both sending and receiving)
• Helper functions for frequent operations (see
  ltl4/ipc.hgt)
• L4_Send Send a message to a thread (blocking)
• L4_Receive Receive a message from a specified
  thread
• L4_Wait Receive a message from any sender
• L4_ReplyWait Send a response to a thread and
  wait for the next message
• L4_Call Send a message to a particular thread
  and wait for it to respond (usual RPC operation)

49
(No Transcript)
50
(No Transcript)
51
Example IPC code

• L4_Msg_t msg
• L4_MsgTag_t tag
• L4_MsgClear(msg)
• L4_MsgAppendWord(msg, word1)
• L4_MsgAppendWord(msg, word2)
• L4_MsgAppendWord(msg, word3)
• L4_MsgAppendWord(msg, word4)
• L4_MsgLoad(msg)
• tag L4_Send(tid)

52
(No Transcript)
53
IPC Timeouts

• Used to control the duration of IPC
• Two timeout types
• Rcv/Snd Timeouts
• Used to control how long the IPC syscall will
  block prior to
• The send phase beginning (SndTimeout)
• The receive phase beginning (RcvTimeout)
• XferTimeouts (Snd/Rcv)
• Used to limit how long the IPC transfer takes
• Only used for StringItems (I.e. you can ignore
  them on MIPS-64)
• Needed to limit time waiting for sender/receiver
  pagefaults on memory.

54
(No Transcript)
55
(No Transcript)
56
(No Transcript)
57
(No Transcript)
58
(No Transcript)
59
(No Transcript)
60
Microkernels

• The microkernel is the only software executing at
  the most privileged level (supervisor or kernel
  mode).
• Actual operating system services, such as device
  drivers, protocol stacks, file systems and user
  interface code are contained in user space.

61
Essential features Minimality

• A Microkernel must include some core features
• some mechanisms for dealing with address spaces
• managing memory protection
• execution abstraction to manage CPU allocation
• typically threads or scheduler activations
• inter-process communication
• required to invoke servers running in their own
  address spaces.
• Liedtke's minimality principle
• A concept is tolerated inside the microkernel
  only if moving it outside the kernel, i.e.,
  permitting competing implementations, would
  prevent the implementation of the system's
  required functionality.