Towards a Compositional SPIN

1
Towards a Compositional SPIN

• Corina Pasareanu
• QSS, NASA Ames Research Center
• Dimitra Giannakopoulou
• RIACS/USRA, NASA Ames Research Center

2
Project Overview

• Main objective
• An integrated environment that supports software
  development and verification/validation
  throughout the lifecycle detect integration
  problems early, prior to coding
• Approach
• Compositional (divide and conquer)
  verification, for increased scalability, at
  design level
• Use design level artifacts to improve/aid coding,
  testing and fault containment

Compositional Verification
Testing
Design
Coding
Requirements
Deployment
implementations
3
Towards a Compositional SPIN

• Learning based compositional analysis for
  increased scalability TACAS03 LTSA tool
• Contributions
• Generic tool architecture for learning based
  assume guarantee reasoning
• Handles multiple components
• Uses SPIN to answer model checking questions
• Other tools can be used (e.g. Java PathFinder)
• Heuristic for automated generation of interface
  specifications
• Application to realistic resource arbiter for a
  spacecraft
• Significant memory gains over traditional
  non-compositional model checking

4
Outline

• Introduction
• Background assume-guarantee reasoning and
  learning
• Tool architecture
• Implementation
• Using SPIN for answering learning questions
• Promela subset
• Case study MER Arbiter model
• Description, results, discussion
• Conclusions and future work

5
Compositional Verification
Does system made up of M1 and M2 satisfy property
P?
M1

• Check P on entire system too many states!
• Use the natural decomposition of the system into
  its components to break-up the verification task
• Check components in isolation
• Does M1 satisfy P?
• Typically a component is designed to satisfy its
  requirements in specific contexts / environments
• Assume-guarantee reasoning introduces assumption
  A representing M1s context

A
M2
6
Assume-Guarantee Rules

• Reason about triples
• ?A? M ?P?
• The formula is true if whenever M is part of a
  system that satisfies A, then the system must
  also guarantee P

M1

• Simplest assume-guarantee rule
• We can also handle symmetric rules

A
M2
How do we come up with the assumption? (usually a
difficult manual process)
7
Approaches

• Infer assumptions automatically
• Two solutions developed
• Algorithmic generation of assumption
  (controller) knowledge of environment is not
  required ASE02
• Incremental assumption computation based on
  counterexamples, learning and knowledge of
  environment TACAS03, SAVCBS03 (symmetric
  rules)

8
Formalisms

• Components modeled as finite state machines (FSM)
• FSMs assembled with parallel composition operator
  
• Synchronizes shared actions, interleaves
  remaining actions
• A property P is a FSM
• P describes all legal behaviors
• Perr determinize complete P bad behaviors
  lead to error
• Component M satisfies P iff error state
  unreachable in (M Perr)
• Assume-guarantee reasoning
• Assumptions and guarantees are FSMs
• ?A? M ?P? holds iff error state unreachable in (A
  M Perr)
• The alphabet of A, ?A, contains all environment
  actions that appear in P

9
Example
Input
Ordererr
in
send
in
ack

out
in
out
Output
out
send
ack
10
Computing the Weakest Assumption

• Given component M, property P, and the interface
  of M with its environment, generate the weakest
  environment assumption A such that assuming A, M
  P
• Weakest means that for all environments E
• (E M P) IFF E A

11
Learning for Assume-guarantee Reasoning

• Use an off-the-shelf learning algorithm to build
  appropriate assumption for the rule
• Process is iterative
• Assumptions are generated by querying the system,
  and are gradually refined
• Queries are answered by model checking
• Refinement is based on counterexamples obtained
  by model checking
• Termination is guaranteed
• Extended with symmetric rules

12
Learning with L

• L algorithm by Angluin, improved by Rivest
  Schapire
• Learns an unknown regular language U (over
  alphabet ?) and produces a DFA A such that L(A)
  U
• Uses a teacher to answer two types of questions

true
L
query string s
is s in U?
false
remove string t
false
conjecture Ai
true
output DFA A such that L(A) U
is L(Ai)U?
false
add string t
13
Learning Assumptions

1. ?A? M1 ?P?
2. ?true? M2 ?A?
3. ?true? M1 M2 ?P?

• Use L to generate candidate assumptions
• ?A (?M1 ? ?P) ? ?M2

true
Model Checking
L
query string s
false
remove counterex.
false
conjecture Ai
true
true
P holds in M1 M2
false
yes
P violated
add counterex.
no
14
Characteristics

• Terminates with minimal automaton A for U
• Generates DFA candidates Ai A1 lt A2 lt lt
  A
• Produces at most n candidates, where n A
• queries ?(kn2 n logm),
• m is size of largest counterexample, k is size of
  alphabet

• For n components (M1 M2 Mn) apply
  algorithm recursively
• ?Ai? M1 ?P? as before
• ?true? M2 Mn?Ai? invokes the framework

15
Learning Interface Specifications

• Compute an assumption (as weak as possible) for a
  component M1 to guarantee some property P, when
  environment is not available
• ?A ?P

L
counterexample
false
conjecture Ai
?Ai? M1 ?P?
true
true
return Ai
?true? P ?Ai?
counterexample
not accurate with respect to !P !M1
16
Implementation in SPIN

• Stand-alone application
• Invokes SPIN for queries and conjectures
• Handles only a Promela subset
• Components are processes
• Communication through rendezvous channels
• Safety properties
• SPIN trace assertions
• Checking assume-guarantee triples
• Encode assumptions into environment processes
  that run in parallel with components

17
Case Study

• Model derived from MER Resource Arbiter
• Local management of resource contention between
  resource consumers (e.g. science instruments,
  communication systems)
• Avoid simultaneous conflicts (e.g. driving while
  capturing a camera image are mutually
  incompatible)
• Enforce orderly activity in accordance with
  predefined priorities encoded in a lookup table
• 5 resources
• Comm, Drive, PanCam, Arm, Rat
• 5 User threads
• Non-deterministically decide to use any of the 5
  resources
• 3000 LOC
• Checked several safety properties

18
Arbiter Architecture

• Property P
• Mutual exclusion between resources
• Comm and Drive can not be used at the same time

19
MER Arbiter Property

• Mutual exclusion between resources
• Comm and Drive can not be used at the same time
• PanCam and Arm can not be used at the same time
• Rat and Arm can not be used at the same time

20
Incremental Property Checking

• Compute A1 A5 s.t.
• ?A1? U1 ?P?
• ?A2? U2 ?A1?
• ?A3? U3 ?A2?
• ?A4? U4 ?A3?
• ?A5? U5 ?A4?
• ?true? ARB ?A5?
• Result
• P holds on U1 .. U5 ARB
• Two techniques
• Recursive invocation
• Interface specification
• Comparison with non-compositional analysis

21
Analysis Results
Analysis Memory State Space Time tmodel tcompile trun Assumption Size
Monolithic 544 MB 3.9e06 0.02s 0.8s 33.7s N/A
Recursive 2.6 MB 1002 0.03s 1.1s 0.03s 6 .. 12
Interface 2.6 MB 2941 0.04s 1.3s 0.02s 12
Results do not reflect the cost of generating the
assumptions
22
Cost of Assumption Generation
Analysis queries oracle1 oracle2 tSPINtLearn MemLearn tLTSA MemLTSA
Recursive 4884 48 1 5646.3s 1743K 42.8s 20400K
Interface (A1) 852 12 1 818.2s 508K 3.07s 4845K
LTSA tool
23
Discussion

• Significant memory savings
• Serious time overhead
• Experimented with SPINs feature for separate
  compilation
• Encode assumptions as never claims
• Restrict search of the verifier
• New encoding for queries
• Significant improvement
• Cost of interface generation reduced from
  818.213s to 185.185s

24
Conclusion and Future Work

• Tool architecture for compositional verification
• Uses L for automatic derivation of assumptions
• Automated derivation of interface specification
• Loose integration with SPIN
• Model checking for queries and conjectures
• Application to Arbiter case study
• Significant memory gains
• Serious time overhead ? techniques to address
  this issue
• Future work
• Tighter integration with SPIN
• Parallel checks for queries
• Investigate buffered message passing
  communication
• Liveness (learning for infinitary regular sets)
• Distinguish between read and write operations