How ModelChecking Can Help Model Exploration

1
How Model-Checking Can Help Model Exploration

• Marsha Chechik
• Dept of Computer Science
• University of Toronto

Joint work with Arie Gurfinkel, Benet Devereux
2
Overview of Automated Verification
Correct?
3
Correctness properties CTL

• propositional temporal logic
• branching-time logic, allowing explicit
  quantification over possible futures
• Syntax
• True and False are CTL formulas
• if p and q are CTL formulae, then so are ? p,
  p? q, p? q
• EX p - p is true in some next states
• EF p - along some path, p is true in some future
  state
• Ep U q - along some path, p holds until q
  holds
• EG p - along some path, p holds in every state
• Universal quantification AX p , AF p , Ap U
  q, AG p
• ECTL EX, EF, EU, EG, disjunction, negation
  atomic
• ACTL AX, AF, AU, AG, conjunction, negation
  atomic

4
ModelsKripke Structures

• Conventional state machines
• M ltS, A, s0, I , Rgt
• S is a (finite) set of states
• A is a (finite) set of propositional
• variables
• s0 is a unique initial state (s0 ? S)
• I S ? 2A is a labelling function that maps each
  state to the set of propositional variables that
  hold in it
• R ? S ? S is a (total) transition relation

5
Counterexamples and Witnesses

• Goal
• explain why the answer is as given
• counterexample why ? (s0) F
• i.e., why the property fails to hold
• witness why ? (s0) T
• i.e., why the property holds

• Counterexamples and mathematical proofs
• to disprove that ? holds on all elements of S,
  produce a single element s ? S s.t. ?? holds on
  s.
• counterexamples are restricted to
  universally-quantified formulas
• counterexamples are paths (trees) from initial
  state illustrating the failure of property

6
Examples

• Witnesses
• EG p

• Counterexamples
• AG p

7
Witnesses/Counterexamples

• Counterexamples (SMV)
• AG(Running ? AF Idle) is false
• long path leading to state Running
• followed by a counterexample for AF
• EX p is false
• nothing given even though it is false!
• AG (Running ? EF Idle)
• no feedback given when the result is either true
  or false
• Witnesses (SMV)
• similar -- just negate the properties!
• A witness to ? is a counterexample to ??

8
So, what do we want?

• Witnesses/counterexamples
• Good
• short -- contain only as much information as
  necessary
• correspond to the model
• Bad
• not available in all cases
• often hard to navigate to an interesting part

• Proofs
• Good
• complete
• (available for all temporal properties)
• all information is here
• Bad
• too verbose
• not particularly intuitive
• Where is the model?

Want proof-like counter-examples TACAS03
9
Example Cruise Control System (CCS)

• For keeping an automobile running at a certain
  speed
• Driver accelerates to desired speed and then
  presses a button on steering wheel (Button
  bCruise)
• System maintains car speed until
• The driver presses the brake pedal (Break)
• The driver presses the accelerator (Accel)
• The driver turns the cruise control off
  (ButtonbOff)
• The driver turns the ignition off (Ignition)
• The cars speed becomes uncontrollable (Toofast)
• The system can be reactivated by pressing a
  resume button (ButtonbResume)
• The controlled variable is Throttle

10
Model-Checking Example
11
Proof View Example
12
Proof-Like Counter-Example (part 1)
Property AG((CCCruise) -gt Brake)
13
Proof-Like Counter-Example (part 2)
Property AG((CCCruise) -gt Brake)
14
Proof-Like Counter-Example (part 3)
Property AG((CCCruise) -gt Brake)
15
Mixed Quantifier Formula Example
16
Mixed Quantifier Formula Proof View
Property EF (AX Brake)
17
Mixed Quantifier Property Example
Property EF (AX Brake)
18
The Framework
Temporal logic property (CTL)
SMV model (with fairness)
Model- checker engine
solution proof-like witness
KEGVis
Navigation/ exploration strategies
partial witness/ counterexample
19
Exploration

• Witness/Counterexample exploration
• dealing with large witnesses
• How?
• how to fast-forward to interesting parts
• Specify a starting condition, e.g., navigate by a
  formula
• Example AG(Running ? AF Idle)
• not interested in how to get to Running
• Specify stopping condition
• Pick direction (forward/backward)
• how to limit information given to user so that
  interesting cases are easier to find

20
Navigation

• Witness/Counterexample Navigation
• making the most interesting choice
• Sources of choices
• explicit (disjunction)
• which part of property to consider
• Example (EF p) ? (EG q)

• implicit (via EX)
• which state to pick as a witness?
• Example EX p

21
Bounds in Property-Based Navigation

• Depth of exploration
• Example witness for AF ?
• feasible (AF p)(s0) (AF1 p)(s0)
• unfeasible when bound ? S
• (witness is as big as the model!)

• Can get partial knowledge using depth of
  exploration
• Example (AF p)(s0), set depth to 3

• what if depth is smaller than expected?

22
State-based navigation

• Pick successor in which ? (some propositional
  formula) holds
• EX p

• Pick a state using number of successors
• least -- linear
• greatest -- branching

• Attempt to maintain largest common prefix
• Example (EX p) ? (EX q)
• try to pick next state where p and q hold
• greedy approximation

23
Example shortest counterexample
Strategy A combination of navigation and
exploration to ensure that some user-specified
goal is met

• traditionally,
• counterexample generators always do shortest
  (greedy) counterexample

• shortest witness not necessarily the most
  interesting!
• Example (EF Good ? EF Error)(s0)
• (ET U Good ) (s0) ? (ET U Error) (s0)
• (E T U7 Good ) (s0) ? (E T U3 Error) (s0)

• picking the shortest counter-example
• automatically
• manually based on size or additional information

24
Strategies

• Choices
• anything based on paths, states, etc.
• including depth of expansion, history, longest
  common prefix, shortest counter-example...
• decision procedure always greedy (and thus
  non-optimal)
• if a strategy was not followed, does not mean
  that it could not be followed!
• Examples
• User has complete control.
• whenever there is a choice, always ask user
• Always attempt to go through Idle state.
• Always choose state s over t otherwise, ask user

25
Part II

• Finding Properties

26
Model-Checking

• Typically used for verification

Correct?

• Where do the properties come from?

• What to do when they do not hold?

So, goal is not just verification but discovery
of properties that the system should have!
27
Query Checking Chan, CAV00

• Goal speed-up design understanding
• discover properties not known a priori

• Temporal logic query
• temporal logic formula with placeholders
  (unknowns)
• e.g., AG ?x, AG (p ? ?x)
• evaluates to strongest propositional formula that
  makes query true.

• Some applications
• provide partial explanation when property holds
• e.g. instead of AG (a ? b), ask AG ?xa, b
• answer a ? b is stronger!
• provide diagnostic information when property
  fails
• e.g. if AG (req ? AF ack) fails - ask AG (req ?
  AF ?x)

28
Types of queries

• number of placeholders
• e.g., AG ?x, AG (?x ? EX ?y)
• positive vs negative
• positive queries - placeholder under even of
  negations
• e.g. AG ?x
• then look for strongest solutions
• negative queries - placeholder under odd of
  negations
• e.g. AG (?x ? p) AG (??x ? p)
• then look for weakest solutions
• mixed queries - neither positive nor negative
• number of maximally strong solutions
• valid queries - one strongest solution
• arbitrary queries - several strongest solutions

29
Deciding TLQ Problem

• Related work on TLQ
• Original definition Chan, CAV00
• algorithm for valid (single strongest solution)
  subset of CTL
• Extended by Bruns Godefroid, LICS01
• arbitrary temporal logic formulae
• via extended alternating automata
• Our work
• Gurfinkel, Chechik, Devereux, FSE02, TSE03
• reduction to multi-valued model-checking
• implementation that deals with arbitrary temporal
  logic formulae
• with arbitrary number of unknowns
• answers include witnesses

30
Example Cruise Control System (CCS)

• For keeping an automobile running at a certain
  speed
• Driver accelerates to desired speed and then
  presses a button on steering wheel (Button
  bCruise)
• System maintains car speed until
• The driver presses the brake pedal (Break)
• The driver presses the accelerator (Accel)
• The driver turns the cruise control off
  (ButtonbOff)
• The driver turns the ignition off (Ignition)
• The cars speed becomes uncontrollable (Toofast)
• The system can be reactivated by pressing a
  resume button (ButtonbResume)
• The controlled variable is Throttle

31
Specification of CCS

• Specified using SCR method

• Input monitored variables
• Output controlled variables
• System state modeclasses
• Sets of states (modes) that partition the state
  space
• The system is in exactly one mode of each
  modeclass at any point

• State changes in response to events (changes in
  environment)

_at_T(a) WHEN b ?a ? b ? a
32
Mode TransitionTable for CCS
33
Event Table for Throttle
34
Applications of TLQ
35
Query-Checking Witnesses

• A witness
• a subtree explaining why ECTL formula holds
• produced automatically by model-checker

• solutions ?x p
• ?x ?p

Same strategies as for exploring model-checking
witnesses available
36
TLQ Applications Testing

• Current approach (Gargantini, Heitmeyer FSE99)
• branch coverage
• for each mode in mode transition table, test each
  event at least once
• for each mode, test every no-change at least once

• e.g. for mode Off
• need _at_T(Ignition) and no-change

• form CTL properties
• EF((CCOff) ? EX(CCInactive))
• EF((CCOff) ? EX(CCOff))
• witness produced by model-checker is the test
  case!

37
Query-Checking for Testing

• our approach
• witness to single query
• EF((CCOff) ? EX ?xCC)

38
Testing Transitions from Mode Off
39
Generated Witness
Property EF((CCOff) ? EX ?xCC)
40
Query Checking for Testing

• An even better approach

• coverage of the entire mode transition table
• EF(?xCC ? EX ?yCC)

41
Testing All Transitions from Table
42
Witness
43
Implementation

• Our framework is named XChek CAV02
• multi-valued model-checking engine
• TLQSolver CAV03
• query checker built on top of it
• KEGVis TACAS03,FME03
• tool for visualization and exploration of count.
  ex. and witnesses
• Input
• XML models (based on GXL derived language)
• SMV-like input language (synchronous product of
  simple state machines)
• fairness condition (e.g., p is true infinitely
  often)
• CTL property or query
• For more info
• To obtain a prototype version
• send e-mail to xchek_at_cs.toronto.edu
• http//www.cs.toronto.edu/chechik/publications.ht
  ml

44
Questions?Comments?Concerns?Suggestions?
THANKS FOR YOUR ATTENTION!
45
Overview of Multi-Valued Model-Checking
How Correct?
46
Multi-Valued Algebras

• Use additional truth values to represent levels
  of contradiction, uncertainty or anything else
• e.g., True, False, Maybe, Likely, etc.
• Can be defined on a (finite) distributive lattice
  of truth values
• with True at the top and False at the bottom
• using lattice meet as conjunction
• and lattice join as disjunction
• Negation is defined to preserve involution, i.e.
• ??A A
• Then get preservation of associativity,
  idempotency, distributivity, and De Morgans laws
  
• These algebras are called quasi-boolean
• (optional) may also add a refinement operator

47
Multi-Valued Algebras Examples
TT
MT
TM
T
TF
FT
MM
M
(Maybe)
FM
MF
F
FF
Representing disagreement and uncertainty
Classical logic
Representing uncertainty Uses reasoning about
abstraction and partial systems
48
Multi-valued state machines Xkripke structures

• Extension of conventional state machines (Kripke
  structures)
• variables take any value from the logic
• transitions between states take any value from
  the logic
• False transitions are not shown (by convention)

49
Partial information

• Algebra
• use three-valued algebra (Kleene)
• intermediate value represents incomplete
  information or uncertainty

• compact representation for all possible
  refinements of this model
• if a property is True/False on the partial model,
  it is True/False on a refined one
• initial theory developed by Bruns Godefroid,
  CAV99

50
Reasoning about Abstraction

• a way to overcome the state-space explosion
  problem in classical model-checking
• collapses sets of concrete states into a single
  abstract state
• thus indicating that any differences between
  the concrete states within a single abstracts
  state are ignored
• Goal state-wise preservation
• if a formula evaluates to True (False) in an
  abstract state, it evaluates to True (False) in
  the corresponding concrete state

?
51
Complexity

• Running time of the model checker is O(S ?? ?
  k), where
• S - size of state space
• ? - the size of the XCTL formula
• k - time to compute EX

52
Solving Query-Checking

• arbitrary temporal logic formulae
• not necessarily valid
• positive, negative, mixed queries
• any number of placeholders
• output includes reason why the answer is as
  given
• extend language of queries
• e.g. EF ?xp, q, r, where p and q are not true
  simultaneously

53
Some formalism

• A - set of prop. variables
• e.g., p
• set of prop. formulas over A forms lattice
  ordered by ?

• ?B c ?b?B s.t. b ? c
• ?p, ?p p, ?p, true
• X is an upset if ?XX
• p, ?p not an upset, p, ?p, true is

• result - upset lattice, ordered by set inclusion

• each upset can be represented by a set of minimal
  elements
• ?false can represent p, ?p, true, false

So, if X is a solution to a query, all elements
of ?X are solutions
54
Reasoning with Colors

• Given a non-temporal formula with colors, how to
  evaluate in a state?
• e.g. ? (p?q ? red) ? (?p?q ? green) ? (?p? ? q
  ? yellow) ? (p? ? q ? blue) in state where p?q
  holds

• answer red

• Given a temporal formula, how does it evaluate?

• (EX ?) (s) ?t ?succ(s) ? (t)

• e.g., evaluate (EX ?) (s0)
• (p?q ? red) ? (?p?q ? green) ? (?p? ? q ? yellow)
  ? (p? ? q ? blue)

red ? green
yellow
55
Encoding TLQ

• Encoding non-temporal formula with ?x
• if p?q holds in s0, then ?xp,q(s0) ?(p?q)

• in general,
• ?xp (p??p) ? (?p ? ?(?p))
• (p? green) ?(?p? red)

• each color is strongest possible solution
• always get exactly one color per state!

• Temporal formula
• (EX ?x )(s) ?t ?succ(s) ?x (t)

• e.g., (EX ?xp) (s0)

red ? green
yellow
?p, ?p
56
Queries with Multiple Placeholders

• Solution
• Li - lattice of propositional formulas over i th
  placeholder
• solution - from upset lattice over L1 ? ? Ln
• Example
• ?x ? (EX ?x ? AX ?y)
• solution in (B? L2) ? ((C? L2) ? (L1? D)) (B?
  L2) ? (C? D)
• i.e., (x,y) x ? B ? (x ? C ? y ? D)

• Another example
• ?xp,q ? EX ?yp,q in state s0

• ?x(s0) ?p ? ?q
• EX?y(s0) ?p ? q, ?p ? q

• ?x(s0) ?p ? ?q ? ?false
• EX?y(s0) ?false ? ?p ? q, ?p ? q

• solution
• ?p ? ?q ? ?p ? q, ?p ? q
• answers (p ? ?q, p ? q), (p ? ?q, ?p ? q)

57
Negation

• All occurrences of placeholder are either
  negative or positive
• Example AG ? ?x
• solve for AG ?x, choose ? from it
• AG ? and thus AG ? (? ? ) hold
• so ? ? is in solution-set for AG ? ?x
• A given placeholder (?x) appears in both negative
  and positive forms
• replace each positive occurrence with ?x
• replace each negative occurrence with ?x-
• solve
• the set of all solutions to ?x is intersection of
  solutions to ?x and ?x-

58
Multi-Valued Model-Checking

• Multi-Valued CTL (XCTL)
• introduce new constants that interact with true
  and false
• e.g., true ? red red
• allow to include these constants into CTL
  formulas
• Multi-Valued Models
• include colors in transition relations and/or in
  states
• XChek symbolic model-checker
• receives
• a lattice of colors, describing how to compose
  them
• multi-valued model XCTL property
• fairness requirements (e.g., assume p holds
  infinitely often)
• returns
• appropriate color, counter-example/witness

59
Running time

• Naïve query-checking algorithm O(S ? ? ? 22n)
• n - number of atomic propositions of interest
• S - size of state space
• ? - the size of the CTL formula
• Query-checking for valid queries O(S ? ? ?
  2n)
• Running time of our model checker O(S ? ? ?
  dd)
• dd - time to compute EX symbolically
• depends on
• size of lattice (number of join-irreducible
  elements in it)
• cost of performing union and intersections

60
Running time of Query Checker

• Preliminaries
• CT(n) - complexity of performing an operation on
  terminal nodes of ADD
• n -- number of propositions restricting the
  placeholder
• complexity of performing any operation on entire
  ADD is linear in CT(n) and exponential in A
• Theorem
• Complexity of solving a query ? with one
  placeholder is linear in S, ?, CT(n) and
  exponential in A
• Theorem
• Complexity of solving a query ? with multiple
  placeholders (?1(n1)?k(nk)) is same as with
  single placeholder, where CT(?i1k ni)
• but how many terminal nodes are there?

61
Running time (Contd)

• CT(n) is quadratic in the number of strongest
  solutions to ? !
• How many solutions are there in a query?

• Queries about states
• Example AG (?q ? AX ?xp)
• Number of solutions O(2n lt S)
• Query-checking - same complexity class as
  model-checking
• Queries about paths
• Example EG (?x)
• Number of solutions in worst case hornus02
  O(22n lt 2S)
• Query-checking can be infeasible even for small
  problems!
• but not always!
• Ex EF EG ?xCC in a Cruise Control System
  (later in the talk)

62
Improvements to Running Time

• Query-checking and model-checking
• query built of some query-checking and some
  model-checking parts
• AG(?x ? AF p) CTL model-checking of AF p, result
  used for query-checking
• Theorem if placeholder occurs in scope of V
  temporal quantifiers is V ? Q (? ? V) ? M
• Q - worst case complexity of query checking a
  formula with one temporal quantifier
• M - worst case complexity of query checking a
  formula with one temporal quantifier

63
Improvements (Contd)

• Heuristics
• constructing decision diagram for ?x is
• but can be avoided in most queries
• Transition relation is boolean so cost of
  performing conjunction and disjunction on
  terminal nodes is O(1) and does not depend on CT.