Java PathFinder and Model Checking of Programs

1
Java PathFinder andModel Checking of Programs

• Guillaume Brat, Dimitra Giannakopoulou, Klaus
  Havelund, Mike Lowry, Phil Oh, Corina Pasareanu,
  Charles Pecheur, John Penix, Willem Visser
• NASA Ames Research Center
• Automated Software Engineering Group

Matt Dwyer, John Hatcliff Kansas State
University Department of Computing and
Information Sciences
Alex Groce, Flavio Lerda Carnegie Mellon
University School of Computer Science
2
Outline

• Motivation
• Model Checking and Testing
• Java PathFinder
• Program Model Checking

3
Motivation

• Software errors are expensive
• Mars Polar Lander
• Ariane 501

4
Model Checking

• Verification and Validation are crucial
• Model checking has been shown effective

5
The dream

• Model Check Programs

6
Some of the Issues

• Semantics Gap
• Programming Languages
• vs.
• Modeling Languages
• Complexity
• Not Automated

Gap
7
Outline

• Motivation
• Model Checking and Testing
• Java PathFinder
• Program Model Checking

8
Model Checking and Testing

• Software complexity is too high
• Some of the presented methods are not sound
• This is not model checking anymore
• It is automated testing

9
The assumption

• Programs have bugs
• Knowing that there are doesnt mean knowing where
  they are
• Testing is not always effective
• Requires a lot of knowledge of the system
• Model checking can be used to find bugs
  systematically
• If no bug is found we have a non-result

10
Coverage Metrics

• Testing has coverage metrics
• They tell you how good your testing is
• They can be used to measure confidence
• Testing is not very effective for concurrent
  systems
• You dont just have to guess the inputs but also
  the timing of the inputs and the scheduling
• Model checking can address these issues
• We are still missing metrics for concurrent
  programs

11
Bug hunting

• Bug hunting instead of trying to prove something
  correct
• We can accept unsound methods
• We may be able to handle real world examples
• If we allow for modeling we are still not
  checking the correctness of the system itself

12
Outline

• Motivation
• Model Checking and Testing
• Java PathFinder
• Program Model Checking

13
Model Checking for Java

• Explicit State Model Checker
• Java Bytecode as Input Language
• Assertions, Deadlock Freedom,
• LTL Properties
• Source Level Error Trace
• Special JVM
• Allows guided execution

14
Architecture
Generic Verification Environment
Generic
C
C
Search Algorithms (model checking, testing)
Java
Special JVM
Class Loader
Storage Subsystem (hash table, bitstate hashing)
Expression Evaluator
15
Outline

• Motivation
• Model Checking and Testing
• Java PathFinder
• Program Model Checking

16
Programs are complex

• Enabling Technologies
• Slicing
• Abstractions
• State Compression
• Partial Order Reduction
• Heuristic Search

17
Property-directed Slicing

• Slicing criterion automatically generated
• Backwards slicing automatically finds dependencies

18
Abstractions

• Remove behaviors but preserve errors
• manual or partially automated
• Over-approximation
• Preserve correctness
• Type-based abstractions
• Predicate abstraction
• Semi-automated

19
JPF Predicate Abstraction

• Annotation used to indicate abstractions
• Source-to-source translation
• Java PathFinder can find abstract error traces

Abstract.remove(x) Abstract.remove(y) Abstract
.addBoolean(EQ, xy)
20
Choice-bounded Search

• An abstract trace that does not contain any
  non-deterministic choice correspond to at least
  one concrete trace
• Bias the model checker to look only choice-free
  traces

21
Storing the States

• States are complex objects
• Classes, Instances, Threads, Stack Frames

Classes
Threads
Objects
22
State Compression

• Instructions modify only part of a state
• Different states share common subparts

X X 1
X?0
X?1
23
State Compression
State
Pools
Array
Compression is very effective up to 94!
24
Partial Order Reduction

• Do not explore equivalent traces
• Requires analysis before model checking

Access to local variable is perfect candidate for
partial order reduction. Java does not provide
enough information. Assume that every access to a
shared object is made in mutual
exclusion. Massive use of partial order
reduction. Use lockset algorithm to check that
mutual exclusion is actually present.
X
Y
X11 Y28
X12 Y27
Y
X
25
Heuristic Search

• Depth first search leads to very long counter
  examples
• Reactive system often exhibit periodic behavior
• It is possible to discover errors at a shorter
  depth
• Heuristic Search
• Breadth first like state generation
• Priority queue for the states based on some
  heuristic
• The challenge
• Find good heuristics
• Based on the property being checked
• Based on the program structure
• JPF offers an API for user-defined heuristics

26
An example

• DEOS
• Real time OS from Honeywell
• 1500 lines of code
• Subtle concurrency error
• Testing did not reveal it
• We (re)discovered the bug!
• Dependency analysis
• Type abstraction
• Choice-free heuristic

27
Conclusion

• Model check programs poses some specific issues
• Some we can deal with
• Some we looked for a way around
• Model checking can be used for systematic testing
• Can be automated
• Can handle concurrent systems
• This is still work in progress!

28
Future directions

• Apply the same techniques to C/C
• Next summer internship proposal
• Combine property and heuristic specification
• Allow the model checker to direct the search
• Combine coverage, model checking and runtime
  analysis
• Develop metrics
• Check the system under certain assumptions