Runtime Refinement Checking of Concurrent Data Structures the VYRD project

1
Runtime Refinement Checking of Concurrent Data
Structures(the VYRD project)

• Serdar Tasiran
• Koç University, Istanbul, TurkeyShaz Qadeer
• Microsoft Research, Redmond, USA

2
The Problem

• Given
• a specification for a data structure
• high-level, executable
• sequential, with atomic operations
• an implementation
• concurrent
• not atomic
• not necessarily linearizable
• may have race conditions
• Verify that
• (All) executions of the implementation are
  consistent with the specification

3
Why runtime verification?

• Static techniques
• Must reason about entire state space of
  implementation
• Theorem proving techniques
• Need representation invariant for implementation
• Difficult for multi-threaded implementations
• Compositional methods
• Better-suited for correctness proofs
• We want to catch concurrency bugs
• Proofs difficult in practice
• Need abstract model for each components
  environment
• Coordination of proof sub-tasks difficult for
  large programs
• Runtime verification
• No need for component environment abstractions
• Gives up exhaustiveness, but can handle practical
  programs

4
Outline

• Preliminaries
• Atomicity
• Refinement
• I/O refinement
• The multiset example
• Checking I/O refinement
• Improving I/O refinement
• Comparison with other correctness criteria

5
Semantics State Transition Systems

• A set of program variables
• The set of program states
• A transition function
• Given
• current state
• action performed
• specifies next state
• Actions
• Visible e.g. method calls, returns
• Invisible e.g. updates to method local variables

6
More preliminaries

• Call action (thread, Call, methodName,
  args)
• Return action (thread, Return, methodName,
  rets)
• A run
• Corresponding trace The sequence of visible
  actions along the run
• ?2 ?3 ?n-1

?1
?2
?3
?n-1
?n
s0
s1
. . .
s2
Sn-1
sn
7
Well-formed atomic traces

• No visible actions by any other thread or by
  another method between matching call and
  return actions

Call2
Return2
c1
c0
Call0
Return0
a1
a0
Call1
Return1
b0
Matching call and return
Matching call and return
Matching call and return
8
Well-formed atomic traces
Call2
Return2
c1
c0
Call0
Return0
a1
a0
Call1
Return1
b0
Atomicfragment
Call0
Return0
Fragment signature
9
Determinism
Call0
Return0
Fragment signature

• Recall Return action contains return value
• Atomic state transition system deterministic iff
• same initial state and
• same fragment signature
• imply same final state
• We require specs to be atomic and deterministic

10
Outline

• Preliminaries
• Atomicity
• Refinement
• I/O refinement
• The multiset example
• Checking I/O refinement
• Improving I/O refinement
• Comparison with other correctness criteria

11
Refinement

• Projection of traces (visible actions) to
  application threads match
• S State transition system for specification
• I State transition system for implementation
• I refines S iff
• for every trace ?I of I, there is a trace ?S
  of S such that, for every application thread
  t ?It ?St

12
Refinement
Specification trace ?S
Call2
Return2
c1
c0
Call0
Return0
a1
a0
Call1
Return1
b0
Implementation trace ?I
Call2
c1
c0
Return2
Call0
b0
a0
Call1
a1
Return1
Return0
13
I/O Refinement

• Refinement ??I ??S ?It ?St
• Notion of refinement Choice of what actions are
  visible
• I/O Refinement
• Define only call and return actions as
  visible
• ? sequence of calls and returns in the
  implementation ? a spec run in with matching
  calls and return values
• Spec atomic and deterministic
• Spec run gives a linear order of method
  executions
• Called witness interleaving
• Practical issue Too many possible interleavings
• Idea Infer witness interleaving from runtime
  analysis

14
Commit actions and witness interleavings
Specification trace ?S
Call2
Return2
c1
c0
Call0
Return0
a1
a0
Call1
Return1
b0
Implementation trace ?I
Call2
c1
c0
Return2
Call0
b0
a0
Call1
a1
Return1
Return0
Commitaction
Commitaction
Commitaction
15
Commit action

• Intuition
• The first line of code that makes visible to
  other threads the modified view of the data
  structure state
• Ordering of commit actions Applications view
  of ordering of operations
• Can be seen as simplified way to construct
  abstraction map
• User specifies line of code that corresponds to
  the commit action

16
Outline

• Preliminaries
• Atomicity
• Refinement
• I/O refinement
• The multiset example
• Checking I/O refinement
• Improving I/O refinement
• Comparison with other correctness criteria

17
Example Multiset

• Multiset data structure
• Two operations
• INSERTPAIR(x,y)
• If it succeeds, inserts integers x and y into the
  multiset
• If it fails, it leaves multiset unmodified
• LOOKUP(x)
• Returns true iff x is in the multiset

18
Multiset Specification

• INSERTPAIR(x,y)
• status ? success or failure
• if (status success)
• M ? M U x,y
• return status

LOOKUP(x) return (x ? M)

• INSERTPAIR allows non-deterministic failure or
  success
• Makes choice visible via return value

19
Multiset Implementation

• Implementation uses array to represent multiset
• Only elements with valid bit set to true in
  multiset

1
2
3
4
A
20
Multiset Implementation
?
LOOKUP(x) for i ? 1 to n ACQUIRE(Li) if
(Ai.valid and Ai.content x)
RELEASE(Li) return true else
RELEASE(Li) return false
?
content
A
valid
21
Implementation helper method FindSlot(x)

• FINDSLOT(x)
• for i ? 1 to n
• ACQUIRE(Li)
• if (Ai.content null)
• Ai.content x
• RELEASE(Li)
• return i
• else
• RELEASE(Li)
• return 0

• Finds free slot in array to insert x
• Returns index if slot found
• Returns 0 otherwise

22
Multiset Implementation

• INSERTPAIR(x,y)
• i ? FINDSLOT(x)
• if (i0)
• return failure
• j ? FINDSLOT(y)
• if (j0)
• Ai.content ? null
• return failure
• ACQUIRE(Li)
• ACQUIRE(Lj)
• Ai.valid ? true
• Aj.valid ? true
• RELEASE(Li)
• RELEASE(Lj)
• return success

23

• INSERTPAIR(x,y)
• i ? FINDSLOT(x)
• if (i0)
• return failure
• j ? FINDSLOT(y)
• if (j0)
• Ai.content ? null
• return failure
• ACQUIRE(Li)
• ACQUIRE(Lj)
• Ai.valid ? true
• Aj.valid ? true
• RELEASE(Li)
• RELEASE(Lj)
• return success

LOOKUP(x) ?
LOOKUP(y) ?
Commit
LOOKUP(x) ?
LOOKUP(y) ?
24
Outline

• Preliminaries
• Atomicity
• Refinement
• I/O refinement
• The multiset example
• Checking I/O refinement
• Improving I/O refinement
• Comparison with other correctness criteria

25
Runtime Checking of I/O Refinement

• Spec atomic and deterministic
• Given a sequence of method calls and return
  values,there is at most one run
• Checking procedure
• Execute implementation
• Record
• order of commit points
• method calls and return values
• Execute spec methods in the order they committed
• Check fails if spec is not able to execute method
  with given return value

26
Runtime Checking of I/O Refinement

• I/O refinement check may fail because
• Implementation is wrong
• The selection of commit points is wrong
• Can tell which is the case by comparing witness
  interleaving with implementation trace
• Improves testing
• In multi-threaded tests, without witness
  interleaving, difficult to decide expected return
  value of method or final state at end of test
• Must consider all interleavings, or
• Forced to be too permissive

27
Off-line checking using a log

• Avoid overhead and concurrency impact
• Write actions of implementation into log
• Verification Separate thread
• Only reads from the log
• Log Sequence of visible actions and commit
  actions
• Actions appear in log in the order they happen in
  implementation
• Logged actions (not operations) serialized by log
• Initial part of log in memory
• ? Low impact of contention for log
• Must perform action atomically with log entry
  insertion
• Optimizations possible

28
Outline

• Preliminaries
• Atomicity
• Refinement
• I/O refinement
• The multiset example
• Checking I/O refinement
• Improving I/O refinement
• Comparison with other correctness criteria

29
Improving I/O refinement

• Why
• Test program may not perform observations
  frequently enough
• Inserting them may affect concurrency too much
• Observations may not get interleaved at the most
  interesting places
• Example If a multiset test program does no
  LookUps, all I/O refinement tests will pass

30
Improving I/O refinement The view

• Include a state-based condition into the
  definition of refinement
• Define auxiliary, hypothetical variable view
• Both spec and implementation have their copy of
  view
• User indicates how view gets updated in both
• view represents only the abstract state of the
  data structure
• Abstract away information not relevant to data
  structure state
• From both spec and implementation
• Method return values determined uniquely by view
• Initialized to same value in spec and
  implementation
• In spec Updated once, atomically between call
  and return
• In implementation Updated once, atomically with
  commit action
• During runtime checking, check that views match
  for each method

31
Definition of view for multiset

• Specs definition of view
• Contents of multiset in spec
• Implementations definition of view
• Computed atomically with commit action

view ? ? for i ? 1 to n lockOK (Li not
held by any thread) or (Li held by
thread currently committing) or (Li
held by thread executing LOOKUP) if
(Ai.valid or lockOK) view ? view U
Ai.content
32
Off-line checking for view

• Implementations version of view constructed from
  log, off-line
• Must log all variable updates that affect view
• can look forward in log, past commit action, to
  compute view for implementation
• May have to log certain lock acquisitions and
  releases in log
• Compute, compare view incrementally for
  efficiency
• Overhead, impact on concurrency increases
• But programs we are working on keep similar logs
  for recovery purposes anyway
• Performance overhead tolerable for these

33
What does view buy?

• Examining program and spec state provides more
  observability
• Imagine multiset with REMOVE operation
• Suppose application executes INSERTPAIR(a,a)
• But implementation erroneously inserts a once
• To expose this error through I/O refinement or
  testing, error must show up in return value of
  LOOKUP(a)
• Need execution that
• inserts a pair of as when there are no as in
  the multiset
• removes a
• looks a up
• before more as get inserted
• Checking view catches bug right after
  INSERTPAIR(a,a)

34
Non-trivial spec view

• Imagine multiset spec given as a binary search
  tree
• Executable spec contains detail not part of
  abstract data structure state
• Must abstract structure of search tree
• view just takes union of tree node contents
• Abstracts away parent-child, left-right
  relationships

35
Outline

• Preliminaries
• Atomicity
• Refinement
• I/O refinement
• The multiset example
• Checking I/O refinement
• Improving I/O refinement
• Comparison with other correctness criteria

36
Multiset is not atomic

• INSERTPAIR(x,y)
• i ? FINDSLOT(x)
• if (i0)
• return failure
• j ? FINDSLOT(y)
• if (j0)
• Ai.content ? null
• return failure
• ACQUIRE(Li)
• ACQUIRE(Lj)
• Ai.valid ? true
• Aj.valid ? true
• RELEASE(Li)
• RELEASE(Lj)
• return success

• Not possible to re-order different threads
  executions of FINDSLOT

37
Multiset has a race condition

• INSERTPAIR(x,y)
• i ? FINDSLOT(x)
• if (i0)
• return failure
• j ? FINDSLOT(y)
• if (j0)
• Ai.content ? null
• return failure
• ACQUIRE(Li)
• ACQUIRE(Lj)
• Ai.valid ? true
• Aj.valid ? true
• RELEASE(Li)
• RELEASE(Lj)
• return success

LOOKUP(x) for i ? 1 to n ACQUIRE(Li) if
(Ai.valid and Ai.content x)
RELEASE(Li) return true else
RELEASE(Li) return false
38
Multiset is not linearizable

• thread t1 executing INSERTPAIR(1,2) concurrently
  with thread t2 executing INSERTPAIR(5,6)

FINDSLOT(2),INSERTPAIR(1,2) by t1
fail FINDSLOT(6),INSERTPAIR(5,6) by t2 fail

• No linearized execution of implementation fails
  first call to INSERTPAIR(5,6)
• Multiset implementation not linearizable

39
Conclusions, future work

• Run-time refinement checking a promising approach
• Caught artificial bugs in Multiset
• Caught real bugs in Scan file system for
  WindowsNT
• In the process of applying it to Boxwood
• A concurrent implementation of a B-link tree
• When no formal, executable spec exists, we use
  atomic version of implementation as spec
• Lowers barrier to application of
  refinement-based methods
• Concentrates on concurrency bugs