Modular Verification of Multithreaded Software

1
Modular Verification of Multithreaded Software

• Shaz Qadeer
• Compaq Systems Research Center

Joint work with Cormac Flanagan, Stephen Freund,
Sanjit Seshia
2
Multithreaded software

• Examples operating systems, databases, web
  servers, Java libraries
• Verification problem
• for all inputs and interleavings, program
  satisfies specification
• Verifying multithreaded programs is difficult
• large and complex

3
Modular verification of multithreaded programs

• thread verify each thread separately using
  specifications of other threads
• procedure in each thread, verify each procedure
  separately using specifications of called
  procedures

4
Method

• Reduce verification of multithreaded programs to
  verification of sequential programs
• Analysis of sequential programs well-studied
• dataflow analysis
• abstract interpretation
• theorem proving with verification conditions

5
Action

• predicate on unprimed and primed variables
• (x y1 ? y y) ? x y1
• (p ? x x) ? py
• true? ? skip

6
Atomic operation

• actions p and q
• p?q all transitions allowed by p must also be
  allowed by q
• (x y1 ? y y) ? x ? y succeeds
• (x y1 ? y y) ? x y fails
• skip?p ? assert p
• p abbreviates p?true

7
Outline

• modular verification
• without procedure calls
• with procedure calls
• verification of Mercator

8
Simple multithreaded program
Init x 0
Thread1 acq x x1 assert x gt 0 rel
Thread2 acq x 0 rel
9
Simple multithreaded program
Init x 0
Thread1 (m0?m1)m x x1 assert x gt
0 (m0)m
Thread2 (m0?m2)m x 0 (m0)m
10
Owicki-Gries-Lamport method
Thread1 L1 m0 ? x?0 (m0?m1)m L2
m1 ? x?0 x x1 L3 m1 ? x?1
assert x gt 0 L4 m1 ? x?1 (m0)m
L5 m0 ? x?0
Thread2 M1 m0 ? x?0 (m0?m2)m M2
m2 ? x?0 x 0 M3 m2 ? x?0
(m0)m M4 m0 ? x?0

• Sequential correctness
• Non-interference

11
Why is Thread1 correct?
Thread1 view
(m0?m1)m
x x1
assert x gt 0
(m0)m
12
Why is Thread1 correct?
Thread1 view
1-abstraction
A1 (m0?m1)m?A2 A1 x x1 ?A2 A1
assert x gt 0 ?A2 A1 (m0)m ?A2
(m0?m1)m
x x1
assert x gt 0
(m0)m
13
Summary of method
Thread1
Sequential checker
1-abs
Thread2
Sequential checker
2-abs
...
...
...
Threadn
Sequential checker
n-abs
14
Assume-guarantee decomposition

• Misra-Chandy 81
• processes communicating over channels
• Jones 83
• parallel shared-memory programs
• Abadi-Lamport 85
• temporal logic
• Alur-Henzinger 96, McMillan 97
• Mealy machines (hardware)
• . . .

15
Outline

• modular verification
• without procedure calls
• with procedure calls
• verification of Mercator

16
Simple multithreaded program
Init x 0
Thread1 acq( ) x x1 assert x gt 0 rel( )
Thread2 acq( ) x 0 rel( )
17
Mutex implementation is not atomic !
ACS(m,v,t) ? if (mv) then m,t t,m
acq( ) t i while (t ? 0)
ACS(m,0,t)
rel( ) m 0
What is the specification of acq( ) and rel( )?
18
Modular verification of sequential programs

• verify each procedure separately using
  specifications of called procedures

requires x gt 0 modifies x ensures x x1 inc( )
x x1
. . assert x gt 0 (x
x1)ltxgt . .
assume x gt 0 x_pre x y_pre y x
x1 assert x x_pre1 assert y y_pre
19
Pre/post-conditions not enough !
Thread1 acq( ) x x1 assert x gt 0 rel( )
modifies x, m ensures m1 ? x?0
acq( ) t 1 while (t ? 0)
ACS(m,0,t)
20
Specification of acq() for Thread i
Spec skip (m0 ? mi)m
Need witness to synchronize spec actions with
impl actions !
Impl acq( ) t i while (t ? 0)
ACS(m,0,t)
Often, witness can be guessed automatically !
21
Verification of acq() for Thread 1
Spec skip (m0 ? m1)m
true t 1 ? skip true while (t ? 0)
true ACS(m,0,t) ?

• m?0 ? skip
• m0 ? (m0 ? m1)m

In general, assumption could involve variables in
scope.
22
Specification of rel() for Thread i
Spec (m0)m
Impl rel( ) m 0
(m0)m
23
Generalization of pre/post-conditions
requires p modifies U ensures q f( ) . .
.
assert p trueU? assume q f( ) . . .
24
Why is Thread1 correct?
Thread1 skip (m0?m1)m x x1 assert x
gt 0 (m0)m
Thread1 acq( ) x x1 assert x gt 0 rel( )
25
Summary

• for each thread i
• write assumption
• write specifications for procedures
• check each specification by inlining
  specifications of called procedures
• check assumptions of other threads

26
Outline

• modular verification
• without procedure calls
• with procedure calls
• verification of Mercator

27
Implementation

• Extended Static Checker for Java
  (Detlefs-Leino-Nelson-Saxe 98)
• verification conditions proved by Simplify
  (Nelson 81)

28
More details

• atomic operation is read/write of a heap variable
• optimizations for reducing assumptions between
  atomic operations
• assume sequential consistency
• witness guessed automatically

29
Mercator (Heydon-Najork 99)

• multithreaded web crawler written in Java
• in production use at Altavista
• synchronization based on Java monitors and
  reader-writer locks

30
Mercator architecture
Checkpointing Thread
Worker Threads
. . .
TC
T1
TNUM
RAM
DISK
URL set
URL queue
31
Reader-writer locks

• lock L1
• checkpointing thread acquires in write mode
• worker threads acquire in read mode
• lock L2
• protects data structure on disk
• worker thread acquires in write mode to write and
  in read mode to read

32
Race detection

• checkpointing thread
• must hold L1 in write mode
• worker thread
• must hold L1 in read mode
• reading must hold L2 in read mode
• writing must hold L2 in write mode

33
Verification

• analyzed 1500 LOC
• Mercators reader-writer locks implement
  specification
• acq_read( ), acq_write( ), rel_read(), rel_write(
  )
• absence of races
• threads obey protocol for accessing shared data
• found bug introduced by us

34
Annotation cost

• 55 annotations
• Scales with complexity of synchronization
  patterns, not size of program

35
Verification of java.util.vector

• synchronization with Java monitors
• 450 LOC
• 4 annotations
• found known race (Flanagan-Freund 00)

36
Annotation overhead
37
Web server
File system
Disk
Buffer cache
Mutex
38
module Mutex / Data / boolean b int
m invariant ?b ? m 0 / Code /
spec acq() . assert b
. spec rel()
39
Programs
S ? SeqProg ??? atomic
operation S1
S2 composition
S1 ? S2 choice
S1? iteration
call p procedure call P ? ParProg
S1 . . . Sn
40
Modular verification of multithreaded programs

• one procedure active for each thread
• interleaving of atomic actions of active
  procedures
• atomic action is unit of induction
• pre/post-conditions not enough
• need abstraction of each atomic action of a
  procedure
• specification is an abstract program