Abstraction for Falsification

1
Abstraction for Falsification

• Thomas Ball
• Orna Kupferman
• Greta Yorsh

Microsoft Research, Redmond, US Hebrew
University, Jerusalem, Israel Tel Aviv
University, Israel
CAV05
2
Abstraction for Verification

• Goal prove properties
• Sound abstraction for verification
• properties of abstract system hold for
  corresponding concrete system
• ? C ? A
• if abstract state a satisfies property P then all
  concrete states represented by a satisfy P

3
Abstraction for Verification

• Goal prove properties
• Sound abstraction for verification
• properties of abstract system hold for
  corresponding concrete system
• ? C ? A
• ? a ? A if a ? P
• then ? c ? C . ?(c)a ? c ? P

4
Abstraction for Verification

• Goal prove properties
• Sound abstraction for verification
• properties of abstract system hold for
  corresponding concrete system
• ? C ? A
• ? a ? A if a ? P
• then ? c ? C . ?(c)a ? c ? P

5
Abstraction for Verification

• Goal prove properties
• Sound abstraction for verification
• errors of the abstract system exist in
  corresponding concrete system
• ? C ? A
• ? a ? A if a ? P
• then ? c ? C . ?(c)a ? c ? P

6
Abstraction for Verification

• Goal prove properties
• Sound abstraction for verification
• errors of the abstract system exist in
  corresponding concrete system
• ? C ? A
• ? a ? A if a ? P
• then ? c ? C . ?(c)a ? c ? P

? c ? C . ?(c)a ? c ? P
7
Motivation

• An abstraction that is sound for falsification
  need not be sound for verification.
• Existing frameworks for abstraction for
  verification
• Modal Transition System (MTS)
• MTS, PKS,KMTS - equivalent in expressive power
  Godefroid,Jagadessan VMCAI03
• can be too restrictive for falsification

8
Main Results

• New framework for abstraction
• Ternary Modal Transition System (TMTS)
• TMTS is stronger than MTS
• Semantics of ?-calculus for TMTS
• Weak reachability
• TMTS with parameterized transitions gives tighter
  underapproximation
• TMTS with assume-guarantee transitions for
  complete reasoning

9
Modal Transition Systems
overapproximation
?
total
underapproximation
?
must ? may
?
?
must and must are incomparable
10
TMTS strictly more expressive than MTS

• MTS
• may and must transitions
• precision preorder is logically characterized by
  PML
• ? p AX ? ? ? ? ? ?
• TMTS
• may, must and must transitions
• precision preorder is logically characterized by
  full-PML
• ? p AX ? AY ? ?? ? ? ?
• full-PML is strictly more expressive than PML
• Pinter,Wolper - PODC84 Kupferman,Pnueli -
  LICS95

11
full-PML is strictly more expressive than PML
?p
?p
K1
K2
unwind
p
p
p
K1 ? ?
K2 ? ?
? EX( (EYp) ? (EY ?p) )

• PML is insensitive to unwinding
• no PML formula can distinguish between K1 and K2

12
TMTS what does it buy us?

• Verifying specifications with past operators
• Reasoning about specifications in falsification
  setting
• must for verification and must- for
  falsification
• Tighter weak reachability in abstract system
• combine must and must- along the path

13
Semantics of ?-calculus for TMTS

• ? C ? A
• (C, c1) ? ?
• (A, a1) ? ? - the value of the ?-calculus
  formula ? in state a1 of TMTS A

14
Semantics of ?-calculus for TMTS

• (A, a) ? ? T
• for all concrete state c with ?(c) a, (C, c) ?
  ?
• (A, a) ? ? T?
• there exists a concrete state c with ?(c) a and
  (C, c) ? ?
• (A, a) ? ? F
• for all concrete state c with ?(c) a, (C, c) ?
  ?
• (A, a) ? ? F?
• there exists a concrete state c with ?(c) a and
  (C, c) ? ?
• (A, a) ? ? M
• there exist concrete states c and c such that
• ?(c) ?(c) a and (C, c) ? ? and (C, c) ?
  ?
• (A, a) ? ? ?

15
Information Lattice
Truth Lattice
T
T
F
?
?
F
16
Information Lattice
Truth Lattice
T
T
F
M
T?
?
M
T?
F?
F?
?
F
17
Truth Lattice
(A,a) ? ? ?
C ? ? ? ?-1(a) c1,c2,c3
AL
Abstraction function ? P(CL) ?
AL Concretization function ? AL ?P(CL)
?(T) c1,c2,c3 ?(T?) c1, c2,
c3, c1,c2,c2,c3,c1,c3, c1,c2,c3 ?(M)
c1, c2, c3, c1,c2,c2,c3,c1,c3
?(F?) , c1, c2, c3,
c1,c2,c2,c3,c1,c3 ?(F)
18
Truth Lattice
(A,a) ? ? ?
C ? ? ? ?-1(a) c1,c2,c3
AL
Abstraction function ? P(CL) ?
AL Concretization function ? AL ?P(CL)
Order in the abstract lattice (induced by the
concrete order and ?) ? v1, v2 ? AL v1 ? v2
? ?(v1) ? ?(v2) Order in the concrete powerset
lattice (Hoare order with set inclusion) ? D1,
D2 ? P(CL) . D1 ? D2 ? ?d1 ? D1 . ? d2 ? D2
. d1 ? d2
19
Truth Lattice

• Abstraction function ? P(CL) ? AL
• ?( d1, ... , dk ) ? ?(d1), ... , ?(dk)
• ? CL ? AL
• ?(d) ?( d ) , ?( ?-1(a) - d ) ? T,
  F, M
• ?(n) ? n 0, n ?-1(a), 0 lt n lt ?-1(a)
• ?(n) n if n 0 or n ?-1(a)
• ?(n) (gt0) otherwise
• Join operator t1 , f1 ? t2 , f2
• (t1 t2) ? t1 ? , (f1 f2) ? f1 ?

20
Semantics of ?1? ?2

• Semantics of conjunction in the concrete powerset
  lattice
• ?D1, D2 ? P(CL).
• D1 ? D2 D1?D2
• D1 ? D2 d1 ? d2 ?d1 ? D1 ?d2 ? D2
• Semantics of conjunction in the abstract lattice
  is conservative
• ?v1, v2 ? AL.
• ? ( ?(v1) ? ?(v1) ) ? v1 ? v2

21
Semantics of ?-calculus for TMTS

• (A, a) ? ?1 ? ?2
• (A, a) ? EX ?
• (A, a) ? ? ?

22
6-valued Semantics of ?1? ?2

• (A, a) ? ?1 ? ?2
• (A, a) ? ?1 ? (A, a) ? ?2

23
6-valued Semantics of ?1? ?2
24
6-valued Semantics of ?1? ?2
25
6-valued Semantics of ?1? ?2 Example
??1
?1 T?
c1
? ?2
?2 T?
a
?
?1 ? ?2 ?
?
c2
??2
? ?1
26
6-valued Semantics of ?1? ?2
27
Information Lattice
Truth Lattice
T
T
F
M
T?
?
M
T?
F?
F?
?
F
28
6-valued Semantics of ?1? ?2
29
6-valued Semantics of ?1? ?2
30
Semantics of EX?

• (A, a) ? EX?

F if for all a, if may(a,a) then (A, a)
? ? F T if exists a s.t. must(a,a) and
(A,a) ? ? T T? if exists a s.t.
must(a,a) and (A,a) ? ? ? T? ? otherwise
31
if (A, a) ? EX? T? then there exists c
with ?(c) a and c ? EX?

• (A, a) ? EX? T?
• exists a s.t. must(a,a) and (A,a) ? ? T?
• exists c such that ?(c)a and c ? ?
• for all c with ?(c)a there is c with ?(c)a
  such that c?c

? EX?
? ?
32
Semantics of ??

• The semantics of PML operators is monotonic
• Least fixpoint operator can be computed by
  iterations from F is the usual way
• (A,a)? ? Z . ?(Z) (A, a) ? ?(F)

33
Semantics of ?-calculus for TMTS

• The 6-valued semantics is at least as precise as
  the standard 3-valued semantics of ?-calculus for
  MTS
• (A,a) ? ? ?
• 3-valued abstraction refinement of must
  transitions Shoham,Grumberg CAV03 adapt for
  must-
• Hypermust transitions
• Larsen,Xinxin-LICS90 Shoham,Grumberg
  CAV04
• adapt for must
• MTS with hypermust is incomparable with TMTS

x 7
x 10
? EX(xgt6) ? T
? EX(xgt6) ? F
? EX(xgt6) T?
? EX(xgt6) ?
must
may
34
Semantics of ?-calculus for TMTS

• The 6-valued semantics is at least as precise as
  the standard 3-valued semantics of ?-calculus for
  MTS
• (A,a) ? ? ?
• 3-valued abstraction refinement of must
  transitions Shoham,Grumberg CAV03 adapt for
  must-
• Hypermust transitions
• Larsen,Xinxin-LICS90 Shoham,Grumberg
  CAV04
• adapt for must
• MTS with hypermust is incomparable with TMTS

35
Weak Reachability
initial state
error trace
error state

• a is weakly-reachable from a
• ?c, c . ?(c)a ? ?(c)a ? c ? c

Related to testing
36
Example
Predicates (x lt 6) (x gt 7)
xlt6
xgt7
(x6)?(x7)
L1 TF
L0 FT
L0 FF

• L0 if xlt6 then
• L1 x x 3
• L2 if x gt 7 then
• L3 x x 3
• L4

must
may
must
L2 TF
L3 FT
L2 FF
may
must
must
L4 TF
L4 FT
L4 FF
37
Example
Predicates (x lt 6) (x gt 7)
xlt6
xgt7
(x6)?(x7)
L1 TF
L0 FT
L0 FF

• L0 if xlt6 then
• L1 x x 3
• L2 if x gt 7 then
• L3 x x 3
• L4

must
may
must
L2 TF
L3 FT
L2 FF
may
must
must
L4 TF
L4 FT
L4 FF
x 5
38
Underapproximation of Weak Reachability

• if must(a,a) then a is weakly reachable
  from a
• Arbitrary combinations of must and must
  transitions do not preserve weak reachability
• Find a tighter underapproximation of
  weak-reachability

39
Example
Predicates (x lt 6) (x gt 7)
xlt6
xgt7
(x6)?(x7)
L1 TF
L0 FT
L0 FF

• L0 if xlt6 then
• L1 x x 3
• L2 if x gt 7 then
• L3 x x 3
• L4

must
may
must
must ?
must ?
L2 TF
L3 FT
L2 FF
may
must
must
L4 TF
L4 FT
L4 FF
40
Underapproximation of Weak Reachability

• if must(a,a) then a is weakly reachable
  from a
• Arbitrary combinations of must and must
  transitions do not preserve weak reachability
• Find a tighter underapproximation of
  weak-reachability

41
Observations

• a3 is weakly reachable from a1
• if there exists a2 such that
• must(a1,a2) and must(a2,a3)
• Onto nature of must is preserved by must-
• Total nature of must is preserved by must

a1
?
must
a2
?
must
a3
?
T.Ball FMCO04
42
Underapproximation

• If there exists a1, a2, a3 such that
• must(a1,a2) and
• must(a2,a3)
• then a3 is weakly-reachable from a1

T.Ball FMCO04
43
Example
Predicates (x lt 6) (x gt 7)
xlt6
xgt7
(x6)?(x7)
L1 TF
L0 FT
L0 FF

• L0 if xlt6 then
• L1 x x 3
• L2 if x gt 7 then
• L3 x x 3
• L4

must
may
must
L2 TF
L3 FT
L2 FF
may
must
must
L4 TF
L4 FT
L4 FF
44
Parameterized Transitions
NO
a
?
NO
a
?
MAY
45
Parameterized Transitions
a
?
must(?)
a
?
if ? is TRUE then must(?) is must and must(?)
is must
46
Observation
a1

• a3 is weakly reachable from a1
• if there exists a2 such that
• must(?1)(a1,a2)
• must(?2) (a2,a3)
• ?1? ?2 ? a2 is satisfiable

?
must(?1)
a2
?1
?
?2
must(?2)
a3
?
47
Observation
a1

• a3 is weakly reachable from a1
• if there exists a2 such that
• must(?1)(a1,a2)
• must(?2) (a2,a3)
• ?1? ?2 ? a2 is satisfiable
• Strongest parameters ?1 and ?2

?
must(?1)
a2
?1
?
?2
must(?2)
a3
?
48
Strongest Parameters
MUST ( WP(s,a) )
?
a
?
s
?c. ?(c) a ? c ? ? ? ?c . ?(c) a ? c ? c
if must(?) then a ? (? ? WP(s,a))
a
?
Generated automatically as part of the
construction of TMTS
49
Example
Predicates (x lt 6) (x gt 7)
xlt6
xgt7
(x6)?(x7)
L1 TF
L0 FT
L0 FF

• L0 if xlt6 then
• L1 x x 3
• L2 if x gt 7 then
• L3 x x 3
• L4

must
may
must
L2 TF
L3 FT
L2 FF
SP(xx3, xlt6) x lt 9 WP(xx-3, xlt6) x lt 9
may
must
must
L4 TF
L4 FT
L4 FF
50
Example
Predicates (x lt 6) (x gt 7)
xlt6
xgt7
(x6)?(x7)
L1 TF
L0 FT
L0 FF

• L0 if xlt6 then
• L1 x x 3
• L2 if x gt 7 then
• L3 x x 3
• L4

must
must
must(xlt9)
L2 TF
L3 FT
L2 FF
must(xlt9)
SP(xx3, xlt6) x lt 9 WP(xx-3, xlt6) x lt 9
must
must
? must (x lt 9)
L4 TF
L4 FT
L4 FF
? must (x lt 9)
51
Tighter Underapproximation
a1
?

• If there exists a1,...,a5 s.t.
• must(a1,a2)
• must(?1)(a2,a3)
• must(?2) (a3,a4)
• must(a4,a5)
• ?1? ?2 ? a3 is satisfiable
• then a5 is weakly-reachable from a1

a2
?
a3
?1
?
?2
a4
?
a5
?
52
Complete Reasoning

• a is reachable by a certain sequence of abstract
  transitions from a
• a is weakly-reachable from a
• Assume-guarantee transitions
• another type of parameterized transitions
• lt?gt must lt?gt

53
Assume-Guarantee Transitions
?
?
lt ? gt MUST lt ? gt
a
?
?c. ?(c) a ? c ? ? ? ?c . ?(c) a
? c ? ? ? c ? c
lt?gtmustlt? gt
a
?
?
?
Which ? and ? predicates do we need?
54
The idea...
?1 a1 ?2 SP(s1, ?1) ? a2 ?3 SP(s2, ?2) ? a3
lt?1gtmust lt?2gt
lt?2gtmust lt?3gt
?3 WP(s3,?4) ? a3 ?4 WP(s4,?5) ? a4 ?5 a5
lt?3gtmust lt ?4gt
?3 ? ?3 is satisfiable
lt?4gtmust lt ?5gt
55
Assume-guarantee transitions

• Complete Reasoning about Weak Reachability
• a is reachable by a certain sequence of
  assume-guarantee transitions from a
• a is weakly-reachable from a
• Finding right parameters computing loop
  invariants

56
Weak Reachability Summary

• Previous work T.Ball FMCO04
• Parameterized transitions
• Assume-guarantee transitions
• complete reasoning

57
Applications

• Falsification of properties in CTL, LTL
• Abstraction-guided test generation
• tighter underapproximation of weakly-reachable
  states improves coverage of the generated tests
• example of QuickSorts partition function

58
Predicate-Complete Testing (PCT)

• T. Ball, FMCO04
• Abstract system defined by predicate abstraction
• Coverage abstract state a is covered when test
  execution reaches some concrete state represented
  by a
• Coverage criteria ?

59
Predicate-Complete Testing (PCT)

• T. Ball, FMCO04
• Abstract system defined by predicate abstraction
• Coverage criterion L / U

initial states
60
Predicate-Complete Testing (PCT)

• T. Ball, FMCO04
• Abstract system defined by predicate abstraction
• Coverage criterion L / U
• Abstraction-guided test-generation strategy
• Tighter underapproximation of weakly-reachable
  states improves coverage of the generated tests

61
Example QuickSorts Partition Function
Predicates (lolthi) (lolthi) (aloltp)
(ahigtp)

• void partition(int a, int n)
• assume(ngt2)
• int p a0
• int lo 1
• int hi n-1
• L0 while (lo lt hi)
• L2 while (alo lt p)
• L3 lo lo 1
• L5 while (ahi gt p)
• L6 hi hi 1
• if (lo lt hi)
• L9 swap(a,lo,hi)
• LC

must(?1)
must(?2)
?1 SP( lolo1,TTTF ) ?2 WP( lolo1,
FFTF) ?1 ? ?2 ? FTTF (lohi) ? (alo?p) ?
(alo-1ltp) ? (alo1ltp)
62
Example QuickSorts Partition Function
Predicates (lolthi) (lolthi) (aloltp)
(ahigtp)

• void partition(int a, int n)
• assume(ngt2)
• int p a0
• int lo 1
• int hi n-1
• L0 while (lo lt hi)
• L2 while (alo lt p)
• L3 lo lo 1
• L5 while (ahi gt p)
• L6 hi hi 1
• if (lo lt hi)
• L9 swap(a,lo,hi)
• LC

L6TTFT
L3TTTF
L3TTTT
L9TTFF
must(?1)
must(?2)
L6FFFT
L3FTTF
L3FFTF
L6FTFT
LCFFFF
p 5
lo
BOF!!
5
3
2
63
Example QuickSorts Partition Function
Predicates (lolthi) (lolthi) (aloltp)
(ahigtp)

• void partition(int a, int n)
• assume(ngt2)
• int p a0
• int lo 1
• int hi n-1
• L0 while (lo lt hi)
• L2 while (alo lt p)
• L3 lo lo 1
• L5 while (ahi gt p)
• L6 hi hi 1
• if (lo lt hi)
• L9 swap(a,lo,hi)
• LC

L6TTFT
L3TTTF
L3TTTT
L9TTFF
must(?3)
L6FFFT
L3FTTF
L3FFTF
L6FTFT
must(?4)
LCFFFF
?3 SP( lolo1,TTTT ) ?4 WP( hihi-1,
FFFT) ?3 ? ?4 ? FTFT is unsatisfiable
The path is infeasible ! must(?3) is lt
TTTT gt must lt ?3 gt must(?4) is lt ?4 gt
must ltFFFT gt
64
Summary

• Ternary Modal Transition System (TMTS)
• onto and total must transitions
• full-PML logical characterizes precision preorder
  on TMTS
• 6-valued semantics of ?-calculus for TMTS
• Tighten underapproximation of weak reachability
  with parameterized transitions
• completeness result using assume-guarantee
  transitions