Tayfun Elmas, Serdar Tasiran Ko University, Istanbul, Turkey

1
VYRD VerifYing Concurrent Programs by Runtime
Refinement-Violation Detection

• Tayfun Elmas, Serdar TasiranKoç University,
  Istanbul, Turkey
• Shaz QadeerMicrosoft Research, Redmond, WA

2
Motivation
Verifying ConcurrentData Structures

• Widely-used software systems are built on
  concurrent data structures
• File systems, databases, internet services
• Standard Java and C class libraries
• Intricate synchronization mechanisms to improve
  performance
• Prone to concurrency errors
• Concurrency errors
• Data loss/corruption
• Difficult to detect, reproduce through testing

3
Our Approach
Refinement asCorrectness Criterion

• Refinement
• For each execution of the implementation (Impl)
• there exists an equivalent, atomic execution of
  Spec
• Linearizability, atomicity (by reduction)
• For each execution of Impl
• there exists an equivalent atomic execution of
  Impl
• Refinement less restrictive
• Rules out fewer implementations
• Example more permissive Spec allows exceptional
  method termination
• in a way not possible in an atomic execution of
  Impl

4
Our Approach
Runtime Checking of Refinement

• Refinement
• For each execution of Impl
• there exists an equivalent, atomic execution of
  Spec
• Use refinement as correctness criterion
• More thorough than assertions
• More observability than pure testing
• Runtime verification Check refinement using
  execution traces
• Can handle industrial-scale programs
• Intermediate between testing exhaustive
  verification

5
Outline

• Example
• Refinement
• I/O-refinement
• View-refinement
• The VYRD tool
• Experience
• Conclusions

6
Multiset
Implementation LookUp

• Multiset data structure
• M 2, 3, 3, 3, 9, 8, 8, 5
• Represented by A1..n
• content the element
• valid Is it in the set?

LookUp (x) for i 1 to n acquire(Ai) if
(Ai.contentx Ai.valid)
release(Ai) return true else
release(Ai) return false
7
Multiset
Implementation FindSlot

• FindSlot Helper routinefor InsertPair
• For space allocation
• Does not set valid field
• x not in multiset yet

FindSlot (x) for i 1 to n acquire(Ai) if
(Ai.contentnull) Ai.content x
release(Ai) return i else
release(Ai) return 0
8
Multiset
Implementation InsertPair

• InsertPair(x,y)
• Refinement violationif only one of x, y inserted
• Two separate calls to FindSlot
• To allocate space for x and y
• InsertPair allows exceptional
• termination
• Example MS array of size 2
• 2 concurrent InsertPairs
• both find slots for xs
• both fail to find slots for ys
• Not possible in atomic execution

9
Multiset
Specification

• Spec state
• M set of integers
• Each method
• Atomic deterministic state update/observation
• Given
• current state, arguments and
• method return value (if one exists)
• specifies new Spec state

10
Outline

• Example
• Refinement
• I/O-refinement
• View-refinement
• The VYRD tool
• Experience
• Conclusions

11
Multiset
I/O Refinement
12
I/O-refinement
Selecting Commit Actions

• Commit points Determines witness ordering
• Drives Spec
• Hints to refinement checking tools
• For each method
• Designate lines in source code
• Multiple lines annotated as commit
• For each method execution
• Only one line should get executed as commit
  action
• No formal procedure
• Intuitively, where new data structure state
  becomes visible to other threads
• Example InsertPair

release(Ai) // commit
13
Outline

• Example
• Refinement
• I/O-refinement
• View-refinement
• The VYRD tool
• Experience
• Conclusions

14
View-refinement
Need for more observability
T1 InsertPair(5,7)
T2 InsertPair(6,8)
Read A0.elt null
FINDSLOT (x) // Buggy for i ? 1 to n if
(Ai.content null) acquire(Ai)
Ai.content x release(Ai) return
i return 0
Read A0.elt null
Read A1.elt null
0
1
2
3
?
?
?
?
elt
valid
F
F
F
F
elt
5
7
?
?
valid
F
F
F
F
5
7
?
?
elt
Overwrites 5!
valid
T
T
F
F
LookUp(5)true, LookUp(7)true
0
1
2
3
6
7
?
?
elt
valid
T
T
F
F
Read A2.elt null
6
7
8
?
elt
valid
6
7
8
?
elt
valid
LookUp(6)true, LookUp(8)true
LookUp(5)false
15
View-refinement
I/O-refinementmay miss errors
T1 InsertPair(5,7)
T2 InsertPair(6,8)

• If observer methods dont get interleavedin the
  right place
• Source of bug too far in the past when I/O
  refinement violation happens

Read A0.elt null
Read A0.elt null
Read A1.elt null
0
1
2
3
?
?
?
?
elt
valid
F
F
F
F
elt
5
7
?
?
valid
F
F
F
F
5
7
?
?
elt
Overwrites 5!
valid
T
T
F
F
LookUp(5)true, LookUp(7)true
0
1
2
3
6
7
?
?
elt
valid
T
T
F
F
Read A2.elt null
6
7
8
?
elt
valid
6
7
8
?
elt
valid
LookUp(6)true, LookUp(8)true
16
View-refinement
More Observability

• I/O-refinement may miss errors
• Our solution View-refinement
• I/O-refinement correspondence between states
  of Impl and Spec at commit points
• Catches state discrepancy right when it happens
• Early warnings for possible I/O refinement
  violations

17
View-refinement
View Variables

• State correspondence
• Hypothetical view variables must match at
  commit points
• view variable
• Extracts abstract data structure state
• Updated atomically once by each method
• viewImpl state information for Impl
• For A1..n
• Extract contentif validtrue
• viewSpec state information for Spec
• Elements of the multiset
• viewSpec ? M (nothing to abstract)
• Other Specs may have state to be abstracted

18
View-refinement
View Variables for Multiset

• viewImpl Computed using abstraction function
• View is a canonical representation
• Canonizes state for viewExact match not required

viewImpl1, 3, 5, 6
19
View-refinement
Checking Refinement
20
View-refinement
Catching FindSlot Bug
T1 InsertPair(5,7)
T2 InsertPair(6,8)
Read A0.elt null
FINDSLOT (x) // Buggy for i ? 1 to n if
(Ai.content null) acquire(Ai)
Ai.content x release(Ai) return
i return 0
Read A0.elt null
Read A1.elt null
0
1
2
3
?
?
?
?
elt
valid
F
F
F
F
elt
5
7
?
?
valid
F
F
F
F
5
7
?
?
elt
Overwrites 5!
valid
T
T
F
F
LookUp(5)true, LookUp(7)true
0
1
2
3
6
7
?
?
elt
valid
T
T
F
F
Read A2.elt null
6
7
8
?
elt
valid
6
7
8
?
elt
valid
LookUp(6)true, LookUp(8)true
21
View-refinement
Catching FindSlot Bug
InsertP(5,7)Returns success
InsertP(6,8)Returns success
Call InsertP(5,7)
Call InsertP(6,8)
Read A0.elt
A0.content5
Read A0.elt
A0.validtrue
A0.validtrue
Read A0.elt
A1.content7
A1.validtrue
A0.content6
A2.content8
A0.validtrue
A2.validtrue
Read A2.elt
Implementation
22
Outline

• Example
• Refinement
• I/O-refinement
• View-refinement
• The VYRD tool
• Experience
• Conclusions

23
The VYRD Tool
Architecture

• Instrument Impl
• in order to log actionsin the order they happen
• Commit actions annotated by user
• Write abstraction function

Test harness
Impl
Write to log

• Enablesonline/offline checking

...
Returnsuccess
Call Insert(3)
Unlock A0
A0.elt3
Call LookUp(3)
Returnsuccess
Unlock A1
A1.elt4
Returnsuccess
read A0
Return true
A0.eltnull
Unlock A0
Call Insert(4)
Call Delete(3)
Read from log
Execute logged actions
Run methods inwitness ordering
Replay Mechanism
Implreplay
Spec
Refinement Checker
traceImpl
traceSpec
24
The VYRD Tool
Atomized Impl as Spec

• Spec atomized version of Impl
• Fully synchronized methods
• Use single global lock
• Separates checking concurrency errors from
  sequential verification
• Slight modification
• Return value from Impl method additional argument
  to Spec methods
• More permissive than Impl
• Can handle failure return values
• Exact state match at commit pointsnot required
• Match view variables only
• Different from commit atomicity

25
Outline

• Example
• Refinement
• I/O-refinement
• View-refinement
• The VYRD tool
• Experience
• Conclusions

26
Experience
The Boxwood Project
27
Experience
Experimental Results

• Scalable method Caught bugs in industrial-scale
  designs
• Boxwood (30K LOC)
• Scan Filesystem (Windows NT)
• Java Libraries with known bugs
• Moderate instrumentation effort
• Several lines for each method
• I/O-refinement
• Low logging and verification overhead
• BLinkTree Logging 17 over testing, refinement
  check 27
• View-refinement
• BLinkTree Logging 20 over testing, refinement
  check 137
• More effective in catching errors
• Cache
• View-refinement 26 random methods before error
• I/O-refinement 539 random methods before error

28
Experience
Concurrency Bug in Cache

• Very similar to bug found in Scan file system
• Had not been caught by developers
• Current version does not contain bug
• Bug manifestation
• Cache entry is correct
• Permanent storage has corrupted data
• Cause of bug Concurrent execution of Write and
  Flush on the same entry
• Write to a dirty entry not locked properly
• Flush writes corrupted data to Chunk Manager
• Marks entry clean
• Hard to catch through testing
• As long as Reads hit in Cache, return value
  correct
• Caught through testing only if
• Cache fills, clean entry in Cache is evicted
• No Writes to entry in the meantime
• Entry read after eviction
• Very unlikely

29
Conclusions

• Runtime refinement checking
• Powerful technique with reasonable computational
  cost
• Effective for complex industrial-scale software
• Key novelty Improves observability of testing
• Future work
• Improving coverage/controllability
• Reducing manual instrumentation
• by limited use of model checking
• Tayfun Elmas, Serdar TasiranVyrdMC Driving
  Runtime Refinement Checking with Model Checkers
  (To appear in) Fifth Workshop on Runtime
  Verification (RV'05).The University of
  Edinburgh, Scotland, UK. July 12, 2005.

30
Questions
VYRD VerifYing Concurrent Programs by Runtime
Refinement-Violation Detection Tayfun Elmas,
Serdar Tasiran College of Engineering Koç
University, Istanbul, Turkey Shaz
Qadeer Microsoft Research Redmond, U.S.