Code Aware Resource Management

1
Code Aware Resource Management

• Luca de Alfaro Marco Faella
• Rupak Majumdar Vishwanath Raman

UC Santa Cruz
UC Santa Cruz Universita di Napoli
UC Los Angeles
UC Santa Cruz
2
A Network Protocol Implementation
User
User
Router
Driver
From n/w
Broadcast
Delay
Input
Sender
Output
To n/w
3
A Network Protocol Program
User
User
Multithreaded program with shared data structures
Router
Driver
From n/w
Broadcast
Delay
Input
Sender
Output
Guard shared queues with mutexes and
semaphores to avoid race conditions
To n/w
4
A Network Protocol Implementation

• Locks ensure no race conditions
• But every once in a while, the system freezes
• Deadlock!

5
A Network Protocol Implementation
User
User
Router
Driver
From n/w
Broadcast
Delay
This paper What can the compiler do to help?
Input
Sender
Output
To n/w
6
Resource Management

• Simple policy
• Universally applicable
• But leads to deadlocks

Application Program
Application Program
Thread Library

Rsrc Mgr
lock(b) lock(a) unlock(a) unlock(b)
lock(a) lock(b) unlock(b) unlock(a)
Implements resource allocation policy

Usual (simple) policy If resource (mutex,
semaphore) is available, grant the resource
7
Deadlock

• Not a new problem!
• Various techniques to detect and avoid deadlocks
• Bankers algorithm
• Priority ceiling
• Monotonic locking
• Methods require either additional bookkeeping by
  the programmer, or restrictive
• E.g., monotonic locking does not deal with
  semaphores

8
Code aware resource managers

• Guarantee deadlock freedom while maintaining
  resources efficiently
• Automatic, static tool to synthesize code-aware
  resource managers
• Related work KNY03

Application Program 1
Rsrc Mgr 1
Thread Library

Application Program 2
Rsrc Mgr 2
9
Custom Resource Manager

• Thread 2 requests lock b
• If Thread 1 does not hold lock a, then grant
• Else if Thread 1 is in the else branch then grant
• Else block the request

If() lock(a) lock(b) unlock(b)
unlock(a) else lock(a) lock(c)
unlock(c) unlock(a)
lock(b) lock(a) unlock(a) unlock(b)

10
Cynthesis Design Flow
Embedded Application (C source)
Annotate calls with thread id, location
Cynthesis
Code Aware Resource manager (C code)
Annotated embedded application (C code)
C compiler
C compiler
Linker
Executable with Custom resource manager
11
How does it work?

• Formulate the resource scheduling problem as a
  2-person game between program and resource
  manager
• Goal of resource manager is to avoid deadlocks
  while ensuring all threads make progress
• Winning strategy Code aware resource manager
• Caveat We assume threads are correct, except
  possibly w.r.t. resource interaction
• Progress cannot be guaranteed if thread enters
  infinite loop while holding a lock

12
Resources

• Non-sharable, reusable quantity
• Model Integer variable with actions
• wx! (want x), gx? (get x), rx! (release x)
• Mutex Values in 0,1, initially 1
• Counting semaphores Non-negative numbers,
  initialized to some constant

13
Thread Resource Interfaces

• Abstraction of C threads
• Only models resource interaction
• Control flow graph, edges labeled with resource
  actions

0
wb!
rb!
while(1) lock(b) lock(a) // critical
section unlock(a) unlock(b)
1

• Each wx! edge goes to a
• state whose only edge is
• gx?
• Each gx? edge starts
• from a state whose
• incoming edges are all
• wx!

gb?
2
wa!
3
ga?
4
ra!
5
14
Joint Interfaces The Arena

• A thread resource interface for each thread
• Values for each resource

0
0
a
1
wb!
wa!
rb!
ra!
1
1
b
1
gb?
ga?
2
2

• Three players
• Resource manager
• Some input action
• 2. Scheduler Which thread next?
• 3. Thread Which action next?

wa!
wb!
3
3
ga?
gb?
4
4
ra!
rb!
5
5
15
Joint Interfaces The Arena

• A thread resource interface for each thread
• Values for each resource

0
0
a
1
wb!
wa!
rb!
ra!
1
1
b
1
gb?
ga?
2
2
wa!
wb!
3
3
ga?
gb?
4
4
ra!
rb!
5
5
16
Joint Interfaces The Arena

• A thread resource interface for each thread
• Values for each resource

0
0
a
0
wb!
wa!
rb!
ra!
1
1
b
1
gb?
ga?
2
2
wa!
wb!
3
3
ga?
gb?
4
4
ra!
rb!
5
5
17
Joint Interfaces The Arena

• A thread resource interface for each thread
• Values for each resource

0
0
a
0
wb!
wa!
rb!
ra!
1
1
b
1
gb?
ga?
2
2
wa!
wb!
3
3
ga?
gb?
4
4
ra!
rb!
5
5
18
Joint Interfaces The Arena

• A thread resource interface for each thread
• Values for each resource

0
0
a
0
wb!
wa!
rb!
ra!
1
1
b
0
gb?
ga?
2
2
wa!
wb!
3
3
ga?
gb?
4
4
ra!
rb!
5
5
19
Joint Interfaces The Arena

• A thread resource interface for each thread
• Values for each resource

0
0
a
0
wb!
wa!
rb!
ra!
1
1
b
0
gb?
ga?
2
2
wa!
wb!
3
3
ga?
gb?
Stuck!
4
4
ra!
rb!
5
5
20
The Winning Condition

• Ensure that all threads make progress
• Æi1n ? progressi
• where progressi is true over an edge if thread i
  moves in that edge
• Sources of non-determinism
• Resource manager can decide to grant a request or
  not
• System scheduler can decide which thread to run
  next
• Thread decides which of several possible moves to
  play

21
Fairness Assumptions

• Assume underlying system scheduler is fair if a
  thread is ready to execute infinitely often, it
  will make progress infinitely often
• Assume strongly fair threads if a choice is
  presented infinitely often, it is taken
  infinitely often
• Takes care of loop termination

0
2
1
22
The Scheduling Game SG

• Given a joint interface, find a (possibly
  randomized) strategy of the resource manager for
  the winning condition
• If fair scheduler and fair thread then
• all threads make progress
• Bad news Best known algorithm is exponential in
  the size of the winning condition
• Winning condition is proportional to size of
  threads

23
Toward Efficient Solutions

• Exploit structure of the scheduling game
• Consider two simplified versions
• SGintra All intra-thread non-determinism
  resolved randomly (A stochastic game)
• SGintra,inter Both intra-thread and
  inter-thread non-determinism resolved randomly (A
  Markov Decision process)

0
½
½
2
1
24
Efficient Computation of Solutions

• Theorem A most liberal randomized winning
  strategy for the resource manager can be computed
  for SGintra,inter in quadratic time in the size
  of the joint interface
• Theorem A winning strategy in SGintra,inter is
  also winning in SGintra
• So SGintra games can be solved quickly
• Randomization plays a key role in getting
  efficient solutions!

25
Justifying Randomness

• Solutions for SGintra lead to solutions for SG
• Theorem There is a quadratic algorithm to check
  that the most liberal winning strategy for
  SGintra,inter is also winning for SG
• Thus, use randomization as a surrogate to solve
  the original problem in polynomial time
• Practical experience The winning strategy for
  SGintra,inter is invariably winning for SG

26
Periodically Mutex Free Systems

• Only mutexes (no semaphores)
• If intra-thread non-determinism is resolved
  fairly, then the thread does not hold any mutex
  infinitely often
• Invariably satisfied by any real system
• For PMF systems, a winning strategy in
  SGintra,inter is winning for SG

27
Efficient Resource Managers

• The strategy constructed may not utilize
  resources effectively
• May decide to wait (not grant resource) even when
  there are other moves that grant resource and win
• Decreases CPU utilization
• Should grant as many requests as possible for
  better system utilization

28
Maximal Progress Strategies

• Play wait only in states where wait is the only
  winning move
• Unfortunately, this may not be winning
• Example in the paper
• It is sometimes useful to wait, even when there
  are resources ready to be granted

29
Critical Progress Strategies

• For mutex-only systems
• Play wait only on states where some thread is
  holding a mutex (or if wait is the only winning
  move)
• If no thread is holding a mutex, play uniformly
  over non-wait winning moves
• This strategy is winning for PMF systems

30
When there are semaphores

• Take the winning strategy from SGintra,inter
• Skip wait moves only when no thread is blocked
  waiting for a resource
• Again, this strategy is winning in SG iff the
  original strategy was winning in SG

31
Implementation

• Cynthesis implemented in Ocaml, using Cil
• Generates code for Ecos OS, but can be configured
  for other OS
• Outputs resource manager as compact C code
• Manager maintains separate thread interfaces and
  resource values
• Encodes losing states as a BDD

32
Code Aware Resource Manager
Slow down the router Disallow router to add
packets to broadcast if output is full, and
vice versa
User
User
Router
Driver
From n/w
Broadcast
Delay
Input
Sender
Output
To n/w
33

• Thank You!
• Questions?

34
A Network Protocol Implementation
User
User
Router
Driver
From n/w
Broadcast
Delay
Input
Sender
Output
To n/w