Introduction to Formal Methods for SW and HW Development 09: SAT Based Abstraction/Refinement in Model-Checking

1
Introduction to Formal Methods for SW and HW
Development09 SAT Based Abstraction/Refinement
in Model-Checking
Roberto Sebastiani Based on work and slides by
E. Clarke, A. Gupta, J. Kukula, O. Strichman
(CAV02) revisions by M. Roveri, R. Sebastiani
and S. Tonetta
2
Outline

• Preliminaries
• Notion of Abstraction
• Over and under-approximation, simulation,
  bisimulation
• Counter-example based abstraction refinement

3
Model Checking

• Given a
• Finite transition system M (S, I, R, L)
• A temporal property p
• The model checking problem
• Does M satisfy p?

4
Model Checking

• Temporal properties
• Always when a train arrives
• the bar is not up G (train_arriving ?
  !bar_up)
• Every Send is followed immediately
• by Ack G(Send ? X Ack)
• Reset can always be reached GF Reset
• From some point on, always switch_on FG
  switch_on

Safety properties
Liveness properties
5
Model Checking (safety)
Add reachable states until reaching a fixed-point
bad state
6
Model Checking (safety)
PROBLEM Too many states to handle !
bad state
7
Abstraction
S
S
Abstraction Function h S ! S
8
Abstraction Function

• Partition variables into visible(V) and
  invisible(I) variables.

• The abstract model consists of V variables. I
  variables are made inputs.

• The abstraction function maps each state to its
  projection over V.

9
Abstraction Function
x1 x2 x3 x4
0 0 0 0 0 0 0 1 0 0 1
0 0 0 1 1
Group concrete states with identical visible part
to a single abstract state.
10
Building an Abstract Model
M can be computed efficiently if M is in
functional form, e.g. sequential circuits.
11
Building an Abstract Model (cont.)

• next(x1) f1(x1,x2,x3,x4,i1,i2)
• next(x2) f2(x1,x2,x3,x4,i1,i2)
• next(x3) f3(x1,x2,x3,x4,i1,i2)
• next(x4) f4(x1,x2,x3,x4,i1,i2)

• next(x1) f1(x1,x2,x3,x4,i1,i2)
• next(x2) f2(x1,x2,x3,x4,i1,i2)

abstract
12
Computing Abstractions
?
S
S

• S concrete state space
• S abstract state space
• ? S ? S - abstraction
• ? S ? 2S ?(s) s sh(s)
  -concretization (refinement)
• Properties of ? and ?
• ?(?(A)) A, for A in S
• ?(?(B)) ? B, for B in S
• The above properties mean that ? and ? are
  Galois-connected

?
13
Aside simulations

• M (s0, S, R, L)
• M (t0, S, R, L)
• Definition p is a simulation between M and M
  if
• (s0, t0) ? p
• ? (t, t1) ? R ?(s, s1) ? R s.t. (s, t) ? p and
  (s1, t1) ? p
• (We say that M simulates M.)
• Intuitively, for every transition in M there is
  a corresponding transition in M

14
Aside bisimulation

• M (s0, S, R, L)
• M (t0, S, R, L)
• Definition p is a bisimulation between M and M
  if
• p is a simulation between M and M and
• p is a simulation between M and M

15
Existential Abstraction (Over-Approximation) M
simulates M
I
I
16
Model Checking Abstract Model

• Let f be a universally-quantified property
  (i.e., expressed in LTL or ACTL) and M
  simulates M

• Preservation Theorem
• M ? f ? M ? f
• Intuition if M has a countermodel,M simulates it

• The converse does not hold
• M ? f ? M ? f

• The counterexample may be spurious

17
Universal Abstraction (Under-Approximation) M
simulates M
I
I
18
Model Checking Abstract Model

• Let f be a existential-quantified property
  (i.e., expressed in ECTL) and M simulates M

• Preservation Theorem
• M ? f ? M ? f
• Intuition if M has a model, M simulates it

• Converse does not hold
• M ? f ? M ? f

19
Model Checking Abstract Model

• M (s0, S, R, L) and M (t0, S, R, L)
  related by bisimulation
• Then, for every CTL/LTL property f
• M ? f ? M ? f
• M ? f ? M ? f

20
Our specific problem

• Let f be a universally-quantified property
  (i.e., expressed in LTL or ACTL) and M
  simulates M

• Preservation Theorem
• M ? f ? M ? f
• Converse does not hold
• M ? f ? M ? f
• Counter-examples may be spurious

21
Checking the Counterexample

• Counterexample (c1, ,cm)
• Each ci is an assignment to V.
• Simulate the counterexample on the concrete
  model.

22
Checking the Counterexample
Concrete traces corresponding to the
counterexample (c1, ,cm)
(Initial State)
(Unrolled Transition Relation)
(Restriction of V to Counterexample)
23
Abstraction-Refinement Loop
Model Check
Abstract
M, p
M, p, h
Pass
No Bug
Fail
h
Check Counterexample
Refine
Spurious
Real
Bug
24
Refinement methods
Localization
(R. Kurshan, 80s)
25
Refinement methods
Intels refinement heuristic
(Glusman et al., 2002)

• Generate all counterexamples.
• Prioritize variables according to their
  consistency in the counterexamples.

X1 x2 x3 x4
26
Refinement methods
Abstraction/refinement with conflict analysis
(Chauhan, Clarke, Kukula, Sapra, Veith, Wang,
FMCAD 2002)

• Simulate counterexample on concrete model with
  SAT
• If the instance is unsatisfiable, analyze
  conflict
• Make visible one of the variables in the clauses
  that lead to the conflict

27
Why spurious counterexample?
28
Refinement

• Problem Deadend and Bad States are in the same
  abstract state.
• Solution Refine abstraction function.
• The sets of Deadend and Bad states should be
  separated into different abstract states.
• ?D to represent the set of Deadend states
• ?B to represent the set of Bad states

29
Refinement
h
h
h
h
h
h
Refinement h
30
Refinement
Let f be the maximum value by which the following
formula is satisfiable
31
Refinement
32
Refinement as Separation

• The state separation problem
• Input Sets D, B of states (assignments)
• Output U (minimal) subset of I s.t.
• ? d ?D, ? b ?B, ?u? U. d(u) ? b(u)

The refinement h is obtained by adding U to V.
33
Refinement as Separation
0 1 0 1
0 1 0
d1
I
b1
V
b2

• Refinement Find subset U of I that separates
  between all pairs of deadend and bad states. Make
  them visible.
• Keep U small !

34
Refinement as Separation
d1
I
b1
V
b2

• Refinement Find subset U of I that separates
  between all pairs of deadend and bad states. Make
  them visible.
• Keep U small !

35
Two separation methods

• ILP-based separation
• Minimal separating set.
• Computationally expensive.
• Decision Tree Learning based separation.
• Not optimal.
• Polynomial.

36
Separation with Decision Tree learning (Example)
D
B
Classification
Separating Set v1,v2,v4
37
Separation with 0-1 ILP

• One constraint per pair of states.
• vi 1 iff vi is in the separating set.

38
Separation with 0-1 ILP (Example)
39
Refinement as Learning

• For systems of realistic size
• Not possible to generate D and B.
• Expensive to separate D and B.
• Solution
• Sample D and B
• Infer separating variables from the samples.
• The method is still complete
• counterexample will eventually be eliminated.

40
Efficient Sampling
D
B

• Let ?(D,B) be the smallest separating set of D
  and B.
• Q Can we find it without deriving D and B ?

• A Search for smallest d,b such that ?(d,b)
  ?(D,B)

41
Efficient Sampling

• Direct search towards samples that contain more
  information.
• How? Find samples not separated by the current
  separating set (Sep).

42
Efficient Sampling

• Recall
• ?D characterizes the deadend states
• ?B characterizes the bad states
• ?D ??B is unsatisfiable
• Samples that agree on the sep variables

43
Efficient Sampling
Run SAT solver on W(Sep)
Sep d,b
STOP
unsat
sat
Add samples to d and b
Compute Sep ?(d,b)
Sep is the minimal separating set of D and B
44
The Tool
Sep
MC
LpSolve
NuSMV
Cadence SMV
Dec Tree
SAT
Chaff
45
Results
Property 1
46
Results
Property 2

• Efficient Sampling together with Decision Tree
  Learning performs best.
• Machine Learning techniques are useful in
  computing good refinements.

47
Current trends (3/3)
Abstraction/refinement with conflict analysis
(Chauhan, Clarke, Kukula, Sapra, Veith, Wang,
FMCAD 2002)
48
The End