Employing decision procedures for automatic analysis and verification of heapmanipulating programs

1
Employing decision procedures for automatic
analysis and verification of heap-manipulating
programs

• Greta Yorsh

under the supervision of
Mooly Sagiv and Alexander Rabinovich
2
Introduction
Its not gonna crash because..
programmer
3
source code
programmer
4
source code
compiler
programmer
5
source code
executable
compiler
programmer
test inputs
tester
6
source code
executable
compiler
programmer
test inputs
tester
client
7
Let X be the set of all inputs
verifier
source code
verifier
compiler
programmer
8
Let X be the set of all inputs
source code
verifier
compiler
programmer

• What is software verification?
• Why do we need software verification?
• Why is software verification difficult?
• How can we verify software?

9
Software Verification Techniques

• Deductive verification Hoare76
• ask the programmer
• use specifications
• preconditions, postconditions, loop invariants
• use (automatic/interactive) theorem prover
• Abstract interpretation CoutsotCousot77
• automatically infers loop invariants
• use abstraction to over-approximate programs
  behavior

9
10
Abstraction

• Choose abstraction an abstract value represent
  sets of program states
• Over-approximate all reachable program states
• Check property over abstract values
• Proof
• Potential error
• Most-precise abstract value
• Trade-off precision and cost

10
11
Leverage Specifications

• Specifications provide insights from the
  programmer

source code
?
verifier
?
programmer

• _at_requires precondition
• _at_ensures postcondition

12
Example
class List List n null ... List
append(List a, List b) if (a null)
return b List d a while (d.n !
null) d d.n d.n b return a
List test(List z) List x, y, t z x
new List() y new List() while (t !
null) x.n y y new List() t
t.n t append(x,y) x append(t,z)
return x
_at_requires acyclic(z)
_at_ensures acyclic(ret)
_at_requires acyclic(a) disjoint(a,b) _at_ensures
acyclic(ret)
13
Modular Analysis

• Each procedure is analyzed only once, in
  isolation
• Allows more precise analysis of each procedure

class List List n null ... List
append(List a, List b) ...
List test(List z) ... t
append(x,y) ... return x
_at_requires acyclic(z)
_at_ensures acyclic(ret)
_at_requires acyclic(a) disjoint(a,b) _at_ensur
es acyclic(ret)
assert acyclic(x) disjoint(x,y)
assume acyclic(t)
14
Challenge

• Abstract interpretation is not designed to take
  specifications into account
• operates on abstract values
• does not support symbolic operations assert,
  assume
• How to implement these operations ?
• Which specification language to use?

15
Results

• Assist program analysis tools to employ formal
  specifications
• algorithms for computing symbolic operations
  TACAS04, VMCAI04
• leverage testing to reduce cost of symbolic
  operations ISSTA06
• case-study on modular analysis AIOOL05
• Develop specification language for recursive
  data-structures that can be incorporated in
  program analysis
• Decidability and undecidability for transitive
  closure logics CSL04
• Verification via structure simulation CAV04
• LRP decidable logic for reasoning about linked
  data-structures FoSSaCS06, JLAP

15
16
Outline

• Two algorithms for computing assume
• Logic of Reachable Patterns

17
Computing assume

• What is the result of assume ? ?
• The most-precise abstract value that represent
  all states that satisfy ?

• Why is assume difficult?
• Formal specification often more expressive than
  abstract values (describes more sets of
  concrete states)

18
Naïve Algorithm for assume ?

• Requires
• finite abstract domain A
• ? is a formula in L
• for every abstract value a,
• the set of states represented by a is expressed
  by the formula ?(a) in L
• theorem prover for L

• Choose the least abstract value a s.t.
• ? ? ?(a) is valid
• Number of calls ? A

19
Better Algorithms

• Goal number of calls ltlt A
• Two algorithms and their tradeoffs
• number of calls ? height(A) ltlt A
• usually, A is at least 2height(A)

20
Computing assume from below
???
21
Computing assume from below
???

• Extension to infinite-height domains
• guarantee termination using widening
• no guarantee of precision
• Leveraging concrete execution to reduce number
  of calls ISSTA06

is ? ? ?(a) valid ?
22
Computing assume from below
???

• Extension to infinite-height domains
• guarantee termination using widening
• no guarantee of precision
• Leveraging concrete execution to reduce number
  of calls ISSTA06

• Intermediate result is not sound

is ? ? ?(a) valid ?
23
Computing assume from above
???
24
Computing assume
???
???
25
Employing Theorem Provers

• Theorem prover or model generation might fail
• no guarantee of precision
• we can guarantee soundness and termination
• Theorem prover should match the expressive power
  of the abstraction and the properties of interest

26
Choosing Specification Language

• Expressive
• specifications assertions, postconditions...
• characterization the abstraction ?(a)
• queries from the algorithms
• Support for automatic reasoning
• theorem prover
• model generator (for assume from below)
• Predictable
• guarantee termination of the analysis
• guarantee the most-precise result w.r.t. to the
  abstraction

27
Goal

• Specification language for reasoning about
  programs that manipulate dynamically allocated
  memory (heap) and recursive data-structures
• Expressive
• Natural for writing specifications
• Decidable

27
28
Heap-Manipulating Programs

• Imperative programs (in C, C, Java)
• dynamic memory allocation
• recursive data-structures
• destructive updates of fields x.next y
• Unbounded size of the heap
• Arbitrary structure of the heap
• objects can be heap shared
• Heap structure can be mutated
• Invariants involve unbounded objects and
  reachability between objects
• Temporary violations of data-structure invariants

29
Decidability with Reachability

• Allow regular properties of arbitrary heaps
• disjointness
• cyclicity
• reversal
• sharing

29
30
Logic of Reachable Patterns (LRP)

• Expressive
• regular properties of arbitrary heaps
• invariants involve reachability
• temporary violations of data-structure invariants
• Decidable
• validity, satisfiability, model generation
• non-trivial proof of tree-like model property
• Useful for verification

30
31
Interesting Heap Properties

• x? ?y
• y is reachable from x
• x? ?x
• x is cyclic

32
Unshared Lists

• List pointed-to by x is not shared
• x unsf
• where unsf is a pattern
• unsf (v0) ? (v1 v0) ? (v2 v0) ? (v1
  v2)

33
Doubly-linked Lists

• Doubly-linked list pointed-to by x
• x invf,b
• where invf,b is a pattern
• invf,b (v0) ? (v0 v1) ? (v1 v0)

33
34
List with head pointers

• x p
• where p is a pattern
• p (v0) ? (v1 v0) ? (v1 x)

34
35
Reversal of Singly-linked List

• precondition
• x? ?null
• postcondition
• y? ?null ? x invn,n

36
LRP Principles

• Arbitrary structure of the heap
• Allow arbitrary boolean combinations of
  reachability constraints
• Restrict quantification no alternations
• Use regular expressions to define reachability
• Syntactically limit the patterns

36
37
Summary

• Approach to software verification
• Prove properties of programs using abstraction
• Trade-off between precision and cost of
  abstraction
• Modular program analysis
• Thesis results
• Novel algorithms enable program analysis
  withuser-provided specifications
• LRP decidable logic for reasoning about
  recursive data-structures
• Foundations of modular analysis of
  heap-manipulating programs

38
Future Directions

• Reasoning about data-structures
• develop specification languages
• develop decision procedures
• Modular analysis of real programs
• using user-provided specifications
• specification inference

38
39
Research Topics and Collaborators

• Advisors
• Mooly Sagiv
• Alexander Rabinovich
• Thesis Results
• Symbolic implementation of abstract operations
  for shape analysis TACAS04
• with Tom Reps and Mooly Sagiv
• Combining testing and abstraction ISSTA06
• with Tom Ball and Mooly Sagiv
• Logic of Reachable Patterns FoSSaCS06,JLAP
• with Alexander Rabinovich, Mooly Sagiv, Antoine
  Meyer, Ahmed Bouajjani

• Additional Results
• Symbolic implementation of best transformer
  VMCAI04
• with Tom Reps and Mooly Sagiv
• Case-study on assume-guarantee AIOOL05
• with Alexey Skidanov, Tom Reps, Mooly Sagiv
• Verificiation via Structure Simulation CAV04
• with Neil Immerman, Tom Reps, Mooly Sagiv
• Decidability and undecidability for transitive
  closure logics CSL04
• with Neil Immerman, Tom Reps, Mooly Sagiv
• Abstraction for falsification CAV05
• with Tom Ball, Orna Kupferman
• Combination method for generating interpolants
  CADE05
• with Madan Musuvathi
• Modular typestate verification POPL08
• with Eran Yahav, Satish Chandra

• Ongoing work
• Decision procedure for LRP, extensions
• with Mooly Sagiv, Alexander Rabinovich
• Decidability and model generation for fragments
  of separation logic
• with Josh Berdine, Byron Cook, Mooly Sagiv

39