Locking in Operating Systems for Multiprocessor Machines

1
Locking in Operating Systems forMultiprocessor
Machines

• Sitaram Iyer

2
Outline

• background
• a brief look at current operating systems
• thus motivation
• some clever ideas to achieve some goals
• conclusion

3
Why Locking?

• To protect kernel data structures.

4
Model of OS

• Modified uniprocessor OS
• interrupt-driven architecture
• scheduler chooses threads by some policy and
  throws them onto idle processors
• Switch to kernel mode via
• system call usually a software interrupt
• hardware interrupt to any processor

5
Popular OSs Linux

• syscalls acquire coarse-grained Kernel Lock
• release only when they block for something
• hence little concurrency among system calls
• interrupts cannot disable on other processors
• so fine grained locks
• exclusive
• shared

6
Lessons learned

• Would perform real badly for system call
  intensive applications, e.g. Web servers
• Absolutely wouldnt scale to monsters like NUMA
• fine grained locking gets complex

7
Coarse vs. Fine grained

• Fine grained locks because
• concurrency
• less contention
• scalability
• Coarse grained locks because
• low locking overhead
• simple locking architecture
• no deadlock concerns

8
Hurricane OS NUMA

• Bunch of techniques
• hybrid coarse and fine grained locks
• clustering and replication
• RPC between clusters
• deadlock avoidance
• pessimistic
• optimistic
• Distributed locks

9
Hybrid locks

• Coarse-grained lock
• over several data structures
• can be held for a very short time only
• acquire a fine grained lock and release this
• Fine-grained lock
• reserve bit
• locking mechanism

10
Clustering

• To address the problem of scalability
• readmost data is replicated across clusters
• some data is replicated across all processors
• write-shared data is migrated around via remote
  procedure calls across clusters
• locking within a cluster standard locks, or
  Distributed Locks

11
Deadlock management

• Deadlock when
• cluster 1 makes a RPC call to cluster 2
• and vice versa
• holding say the coarse lock
• Pessimistic
• release all locks -- will always work.
  Inefficient.
• Optimistic
• mark the reserve bit and try. If fail,
  pessimistic.

12
Distributed locks

• Simple idea locking processors are put on a
  queue rather than arbitrarily spinlocking
• They loop over their own queue elements
• Cache lines dont get tossed around badly
• On unlock, they do something to next-in-line
• FIFO ordering guaranteed, of course
• Modifications 32 improvement

13
Nonblocking synchronization

• Central idea
• read version of data element into local var
• make a local copy of the element
• do all processing in local copy
• atomically
• check version to see if it has changed
• if not, version and update pointer to element
• else (if it has changed), repeat entire procedure

14
So?

• Most data structures lists, queues.. can be
  implemented with a non-blocking lock inside the
  C object
• TSM type-stable memory management
• addition/deletion from list instead of updation
• guarantee that the type of the data element
  wouldnt change for at least time T
• hidden inside C classes

15
Implementation

• Atomic Double Compare and Swap (DCAS)
• hardware support (very few processors), or
• implement using blocking
• atomic
• if (version oldversion ptr oldptr)
• version (version 1) ptr newptr
• return TRUE
• else return FALSE

16
Spinlock (implementation)

• Essentially while (x 1) x 1
• Lock the system bus
• atomic instruction to test-and-set bit
• (in contrast, freebsd compare-and-exchange)
• loop until it becomes available
• aside software locking algorithms

17
Evaluation

• Good
• it is the ultimate fine-grained lock
• deadlocks never happen
• prioritize actions advisory locks
• only a very tiny window of inconsistency
• Bad
• potentially a lot to undo, bad under contention
• atomically updating multiple lists

18
Conclusions

• Hybrids of coarse and fine grained locks are a
  good idea
• Clustering seems to be the best way for scalable
  locking mechanisms
• Distributed locks reduce second-order effects
• Design simplicity hasnt been improved much
• Nonblocking synchronization is attractive