Verification of RealTime Systems with BDDlike DataStructures

1
Verification of Real-Time Systemswith BDD-like
Data-Structures
ATVA2003 Tutorial

• Farn Wang
• Dept. of Electrical Engineering
• National Taiwan University

2
Real-Time Systems

• Not only the right answer,
• but only at the right time !!

• Aviation
• Battlefield
• Missles
• Sonars

• Nuclear reactors
• Medical monitors
• Chemical plants
• Multimedia

3
Real-Time Systems

• What is a Real-Time Systems?
• Not only the right answer,
• but only at the right time !!
• ExampleSynchronous Machine Guns
• A service that is not at the right time can be
  worse than no service!!

4
Schütte-Lanz SL2 airship bombing Warsaw
Courtesy of Rosebuds WWI and early Aviation
Image Archive
5
Wrights, the beginning of flight era
Courtesy of Rosebuds WWI and early Aviation
Image Archive
6
The beginning of air-combatAn imaginary
solution to air-combatDifficult to aim in 3D
maneuver
Courtesy of Rosebuds WWI and early Aviation
Image Archive
7
Sopwith Camel
Courtesy of Rosebuds WWI and early Aviation
Image Archive
8
SPAD VII S.254
Courtesy of Rosebuds WWI and early Aviation
Image Archive
9
WWI dogfight????
Courtesy of Rosebuds WWI and early Aviation
Image Archive
10
Timed automata

• real-time system behaviors
• In 500ms, update the actuators every 50ms, until
  the target is hit.

x0 z0
monitor xlt500ms z?50ms
x? z are real-number clocks of the same rate.
hit
z50ms
z0
11
Timed Automata

• A?Q, X, I, ?, E, ?, ??
• Q set of control locations
• X set of clocks
• I initial constraints
• ? Q??(P,X)invariance constraint at locations
• E?Q?Q set of transitions
• ? E ??(P, X)triggering conditions
• ? E ?2X clocks to reset at transitions

12
A?Q, X, I, ?, E, ?, ??

• Qmonitor,hit E(monitor,monitor),(mo
  nitor,hit)
• Xx,z ? (monitor ,monitor) z 50
• Ihit ? (monitor, hit) true
• ?(monitor) x ? 500?z ? 50 ? (monitor ,monitor)
  z
• ?(hit) true ? (monitor, hit)

monitor x ? 500ms z ? 50ms
x0 z0
hit
z50ms
z0
13
TCTL (Timed Computation-Tree Logic)

• ? q x?c ?? ?1??2 x.? ??1U?2 ???
• ExampleIt is possible that I will get my salary
  at the 7th day.
• day.??(day7?salary)
• ExampleNo matter what, I will be married in 10
  years.
• year.??(year lt 10?married)

14
TCTL(Timed Computation-Tree Logic)

• ExampleAfter you are married, you can remain
  happy in five days.
• ??(married ?day.? ? (daylt5? happy))
• ExampleAfter you are married, you will remain
  happy in five days.
• ??(married ? day.? ? (daylt5 ?happy))

15
TCTL verification complexities

• TCTL model-checking problem against timed
  automata is PSPACE-complete.
• TCTL satisfiability problem is undecidable.
• Alur, Cocoubetis, Dill. IEEE LICS 1990

16
Zonesbasic objects for manipulation and
representation

• a (convex) state-space bounded by constraints
  like
• x-x lt d, x,x ? X d ? N??
• x-x ? d, x,x ? X d?N
• Features
• convex
• the reachable state-space is a union of zones
• can intersect with one another ?nondisjoint paths
• can contain one another ? redundancy
• non-canonical ? needs normal (or canonical) forms

17
Zones
Two zones (0-x1?-3 ?x1-x2lt5? x2-x1lt4)?(0-x2lt-2?
x2-x1lt4) Can also be viewed as a union of set of
literals, 0-x1?-3, x1-x2lt5, x2-x1lt4?0-x2lt-2,
x2-x1lt4
0-x1lt-3
x1-x2lt5
x2
x2-x1lt4
0-x2lt-2
x1
18
Zones
(0-x1?-3 ?x1-x2lt5? x2-x1lt4)?(0-x2lt-2? x2-x1lt4)

• Normal forms
• closure form all-pair shortest-path form
• always the most number of constraints
• reduced form minimum number of constraints

(0-x1?-3 ?0-x2lt2?x1-x2lt5? x2-x1lt4) ?(0-x2lt-2?
0-x1lt2?x2-x1lt4)
(0-x1?-3 ?x1-x2lt5? x2-x1lt4) ?(0-x2lt-2?x2-x1lt4)
19
Symbolic manipulation of dense-time state-space
Henzinger, et al, IEEE LICS 1992
s0 p ? x? 3 ? y ? x5
s1 q ? x6 ? y8
y? 15
y0

• What is the weakest precondition in s0 that
• can transition from S0 to S1?
• xtion_bcke(?) the weakest preconditoin after
  transition e ?
• after time ? ?0 , remains in s0?
• time_bck(?) the weakest precondition to?through
  time-progress?

20
xtion_bcke(?) the weakest preconditoin after
transition e ?
s0 p ? x? 3 ? y ? x5
s1 q ? x6 ? y8
y? 15
y0
Source invariance
assignment
Destination invariance
trigger

• p ? x? 3 ? y ? x5 ?y? 15 ? ?y(y0 ? x6 ? y8??)
• How to get rid of the ?y ?

21
time_bck(?) the weakest precondition to?through
time-progress?
s0 ? p ? x? 3 ? y ? x5
s0 p ? x? 3 ? y ? x5
?
Destination invariance
Time progress
Source invariance

• p?x?3 ?y?x5 ? ??(0?? ? x? ?3 ? y??x?5???)
• How to get rid of the ?? ?

22
How to get rid of the ?? ?
s0 ? p ? x? 3 ? y ? x5
s0 p ? x? 3 ? y ? x5
?

• Pairwisely,deduce all transitivities through ?
• p ? y ? x 5 ? ?? (0 ? ? ? 2-x ? ? ? ? ? 15-y
  ? ? ? 3-x)
• independent of ? ? at RHS
  ? at LHS
• 0 ? ? ? ? 15-y 0 ? 15-y ,
  0 ? 3-x
• 2-x ? ? ? ? 3-x
  2-x ? 15-y , 2-x ? 3-x
• Thus we get p ? y ? x 5 ? y?15 ? x? 3 ? y-x?13

23
TCTL Model checking procedures

• Basic procedures
• Xtion_bcke(?)
• weakest precondition of discrete transitions
• Time_bck(?)
• backward time-progression
• Reachable-bck(?1,?2) lfpY.(?2?(?1?time_bck(?1?Ve
  ?Txtion_bck(Y,e))))

24
Model checking with Non-Zeno requirement

• Lemma given d?1, A,
• ????iff there is a finite run?
• ?from?
• of duration ?d
• along ? every state satisfies ?and
• ? ends at a state satisfying ???
• ???gfp Y.(ZC.reachable-bck(?,Y?ZC?d))

25
The BDD-like Data-Structures

• Farn Wang
• Dept of Electrical Engineering
• National Taiwan University
• NSC 90-2213-E-002-132.

26
Background

• up to 2000, many proposals for BDD-like
  data-structures for dense-time state-spaces.
• NDD, DDD, CDD, RED
• None of them shows advantage over DBM.
• Motivation for this research
• Find out the reasons and ways for improvements.

27
Related work

• BDD for untimed systems Bryant 86 Burch,
  Clarke, et al, 90
• DBM 2-dimensional matrix for a region. Dill 89
  
• NDD BDD to encode discrete time state-spaces.
  ABKMPR97
• CDD a decision diagram for dense-time
  state-space membership BLPWW99. Like CRD,
  except
• default value is (- ?, ?).
• a value c,d of variable x-x' means c?x-x?d.
• DDD variable like x-y c WME92,Balarin96,MLAH99
  
• Exponential number of BDD atoms
• RED encode the ordering of the fractional parts
  of clock readings with a single-bit varible Wang
  00
• for symmetric systems

28
What we have done in this research ?

• Identication of reasons for ineffiency in
  previous research
• semantics of the variables and
• choice of normal forms
• Such issues have not been paid proper attention
  in the past.
• Proposal of CRD and its efficient manipulations
• Experiments.
• Tools available at http//cc.ee.ntu.edu.tw/val/

29
CRD Clock-Restriction Diagram

• A BDD-like data-structure
• Recording device for (zone) DBM set
• variables like x-x
• Arc values like (lt, d), d ?-CA,CA?? or
• (?, d), d ?-CA,CA
• Default value on arcs (lt, ?)
• No constraint!

30
CRD Example
(0-x1?-3 ?x1-x2lt5? x2-x1lt4)?(0-x2lt-2? x2-x1lt4)
0-x1
0-x1
0-x1
(-3,?)
(-?,-3
lt?
(-?,-3
(-?,?)
?-3
0-x2
0-x2
0-x2
x1-x2
0-x2
x1-x2
(-?,-2)
-2,?)
(-?,5)
lt5
lt-2
(-?,-2)
(-?,-2)
x1-x2
x2-x1
x2-x1
x1-x2
(-4,?)
(-?,4)
lt4
(-4,5)
true
true
true
(c) equivalent CDD with disjoint intervals

• (b) CRD with interval representation

(a) CRD with upperbounds
31
Representation fragmentation of CDD
Two zones (0-x1?-3 ?x1-x2lt5? x2-x1lt4)?(0-x2lt-2?
x2-x1lt4) CDD is forced to represent it in 3
disjoint zones.
0-x1
(-?,-3
0-x1lt-3
0-x2
0-x2
x2
-2,?)
(-?,-2)
x1-x2
x1-x2
x1-x2lt5
(-4,?)
x2-x1lt4
(-4,5)
true
0-x2lt-2
x1
32
Sensitivity to variable semanticsRepresentation
fragmentation of CDD

• Example (0?x1?4?1?x2?5)?(1?x1?5?0?x2?4)

x1-0
x1-0
x1-0
(4,5
0,1)
1,5
1,4
0,4
x2-0
x2-0
x2-0
x2-0
x2-0
1,5
0,5
0,4
1,5
0,4
true
true
true

• CDD representation fragmentation problem!
• Exponential blow-up in sizes
• Destroying the shape of analyzed zones

33
Sensitivity to variable semanticsOn the other
hand, CRD prevails.

• Example (0?x1?4?1?x2?5)?(1?x1?5?0?x2?4)

0-x1
0-x1
0-x1

• Linear to the sizes of components
• Preserve zone structures

?-4
?-1
?-4
?-1
x1-0
x1-0
x1-0
x1-0
?0
?5
?0
?5
0-x2
0-x2
0-x2
0-x2
?-1
?-4
?-1
?-4
x2-0
x2-0
x2-0
x2-0
?0
?5
?5
?0
true
true
true
34
Sensitivity to variables semantics

• A family of state-spaces with n clocks that blows
  up CDD,
• ?1 ? i ? n ?1 ? j ? n ((i j)n) ? xj ? 2n ((i
  j)n)
• Caution such fragmentation problem also may
  nullify previous analysis effort on zones.

35
Sensitivity to choice of normal forms

• In general, reduced CRDs have much fewer nodes
  along each path!
• 0-x2lt-3, x1-x3?1, x2-x1lt-4

0-x2
0-x1
0-x1
lt-3
lt-7
(-?,-7)
x1-x3
0-x2
0-x2
?1
lt-3
(-?,-3)
x2-x1
0-x3
0-x3
lt-4
lt-6
(-?,-6)
true
x1-x3
x1-x3
?1
(-?,1
x2-x1
x1-x2
CRD in reduced form
lt-4
(4, ?)
CRD in closure form
CDD in closure form
x2-x3
x2-x3
lt-3
(-?,-3)
true
true
36
Sensitivity to choice of normal forms

• Reduced CRDs are less likely to interfere
  data-sharing!

(x1-x2lt3 ? x3-x1?6 ? x3-x2?6) ?(x1-x4?5 ?
x3-x1?6 ? x3-x4?11)
(x1-x2lt3? x3-x1?6) ? (x1-x4?5? x3-x1?6)
x1-x2
lt?
x1-x2
lt3
x1-x4
lt?
lt3
?5
x1-x4
x3-x1
x3-x1
?6
?6
x3-x1
closure
reduced
x3-x2
x3-x4
?6
?11
lt-9
true
true
37
Sensitivity to choice of normal forms

• Reduced form makes it difficult to detect
  zone-containment.
• (x1-x3 ? -1 ? x3-x2 ? -2 ? x2-x1 ? 3)
• ? (x1-x3 ? -1 ? x3-x1 ? 1)

x1-x2
lt?
?-3
x2-x1
?3
x2-x1
x1-x3
x1-x3
Note, it is O(n3) to deduce the all-pair
shortest-path relation.
?3
lt?
?-1
?-1
x1-x3
x1-x3
x3-x1
x3-x1
?1
?-1
?-1
?1
x2-x3
x3-x1
x3-x2
?2
?1
?-2
x3-x2
closure
?-2
true
reduced
true
38
Contained zone-path elimination

• A node with single outgoing arc labeled lt? can be
  bypassed.
• Given two arcs, when
• 1d1 more restrictive than 2d2
• D1? D2
• then D1 can be removed.
• The operation MAY or MAY NOT
• lead to smaller CRD sizes.
• Don't know how to do this
• with CDD.

x-x
1d1
2d2
D1
D2
slim
x-x
1d1
2d2
D1-D2
D2
39
Set-oriented manipulations on CRDs

• Given two CRDs D1?1, ?2 and D2?2, ?3,
• D1 ? D2 is the CRD for ?2
  O(D1?D2)
• D1 ? D2 is the CRD for ?1, ?2 , ?3
  O(D1?D2)
• D1 - D2 is the CRD for ?1
  O(D1?D2)
• Space-intersection D1 ? D2
• For every ?1(x,x') (1,d1) and ?2(x,x') (2,d2)
  
• ?1??2(x,x') (1,d1) if d1ltd2 ? (d1d2 ?
  lt'')
• (2,d2) otherwise
• D1?D2 ?1??2 ?1? D1?2? D2
  O(D12?D22)

40
BDDCRD

• Can combine BDD with CRD in the same
  data-structure.
• D1 ? D2 is like D1 ? D2
• D1 ? D2 is like D1 ? D2
• D1 - D2 is like D1 ? ?D2
• with integrated evaluation ordering.

41
Style of CRD manipulating algorithm

• set ? / database of already-processed cases /
• ?(B,D) ? ? return rec?(B,D)
• rec?(B, D) with B(xB-xB',(?i, Bi)1?i? n),D
  (xD-xD',(?j, Dj)1?j? m)
• if Btrue, return D else if D is true,
  return B
• else if ?H,(B,D,H)??, return H
• else if xB-xB precedes xD-xD, H(xB-xB ,
  (?i, rec?(Bi,D))1?i? n
• else if xD-xD precedes xB-xB, H(xD-xD ,
  (?j, rec?(B, Dj)1?j? m
• else
• for (in, j m, Hfalse i?1?j ? 1, do
• if ?i ?j , H H ? (xB-xB, (?i ,
  rec?(Bi,Dj))) i-- j--
• else if ?ilt ?j, H H ? (xB-xB, (?j,
  Dj)) j--
• else if ?igt ?j, H H ? (xB-xB, (?i ,
  Bi)) i--
• if I ? 1, H H ? (xB-xB, (?h , Bh)1? h?
  i)
• if j ? 1, H H ? (xB-xB, (?h , Dh)1? h?
  j)
• ? ? ? (B,D,H) return H

42
Computation of closure CRDs

• closurep(?)
• for x? X,
• for x1, x2 ? X,
• if ?(x1,x) ?(x,x2)? ?(x1,x2),
• ?(x1,x2) ?(x1,x) ?(x,x2)
• return ?
• Does not respect the evaluation-ordering of
  BDD-like data-structures!
• Random-accessing is expensive.

43
Computation of closure CRDs

• Reformulation of closure form computation
• closurep(?)
• for x? X,
• for x1, x2 ? X such that x1-x precedes
  x-x2,
• if ?(x1,x) ?(x,x2)? ?(x1,x2),
• ?(x1,x2) ?(x1,x) ?(x,x2)
• for x1,x2 ? X such that x1-x succeeds x-x2,
  
• if ?(x1,x) ?(x,x2)? ?(x1,x2),
• ?(x1,x2) ?(x1,x) ?(x,x2)
• return ?

44
Computation of closure CRDs

• closurep(D) for x?X, D xtive(D, x) return D
  
• clock L, M, R upperbound ? set ?, F
• xtive(D, x) ? ? Mx return rec_xtive(D)
• rec_xtive(D) with D(x-x', (?i, Di)1? i? m)
• if D is true, return D else if ?H,(D,H) ? ?,
  return H
• Hfalse
• if x is M, for i 1 to n,
• D' rec_xtive(Di) Rx' ? ?i F ?
• H H? (x-x' ?i ?rec_xtive_right(D'))
• else if x' is M, for i 1 to n,
• D' rec_xtive(Di) Lx ? ?i F ?
• H H ? (x-x' ?i ?rec_xtive_left(D'))
• else H?1?i?n (x-x' ?i ?rec_xtive(D'))
• ? ? ? (D,H) return H

45
Computation of closure CRDs

• rec_xtive_left(D) with D(x-x', (?i, Di)1? i? m)
  
• if D is true, return D else if ?H,(D,H)?F,
  return H
• if x is M, H ?1?i?n (x-x'?i ? L-x'(? ?i)
  ?rec_xtive_left(Di)
• else H ?1?i?n (x-x'?i ? rec_xtive_left(Di)
• F F ? (D,H) return H
• rec_xtive_right(D) with D(x-x', (?i, Di)1? i? m)
  
• if D is true, return D else if ?H,(D,H)?F,
  return H
• if x is M, H ?1?i?n (x-x'?i ? x-R(? ?i)
  ?rec_xtive_right(Di)
• else H ?1?i?n (x-x'?i ? rec_xtive_right(Di)
• F F ? (D,H) return H

46
DRC a new normal form(Difference-Reduced
Closure)

• Specifically bound to our variable-ordering
  constraints
• x-0 and 0-x precede
  x-x'
• magnitude constraints
  difference constraints
• Magnitude constraints are used commonly in
  modelling.
• x-x' is magnitude-redundant if it can be derived
  from x-0 and 0-x
• DRC CRD
• Identical to closure form except all
  magnitude-redundant constraints are eliminated.

47
TCTL Inevitability Analysis of Dense-Time Systems

• Farn Wang
• Dept. of Electrical Engineering, National Taiwan
  University
• Geng-Dian Hwang, Fang Yu
• Institute of information science, Academia Sinica

48
Outline

• Why ?
• TCTL inevitability analysis
• Performance solutions
• Experiments
• Strategy for combinations of techniques

49
Why ?

• Motivation
• Efficiency for general TCTL model-checking.
• The state-of-art
• Safety properties ???
• Negation reachability properties ???
• Least fixpoint evaluation
• Heavily researched for efficient evaluation
• nonZeno requirement, not very necessary
• Inevitability properties ???
• Kind of parallel to liveness properties in LTL
• Negation ???
• Greatest fixpoint evaluation
• Not very much researched for efficient evaluation
• nonZeno requirement necessary

50
Our contribution

• a symbolic greatest fixpoint algorithm with
  non-Zeno requirement
• Techniques for performance solution
• Timing distance to measure non-Zenoness
• Early Decision on the Greatest Fixpoint
• Abstraction techniques
• Experiments to evaluate the techniques
• Strategies to combine techniques

51
Outline

• Why ?
• TCTL inevitability analysis
• Performance solutions
• Experiments
• Strategy for combinations of techniques

52
Gfp procedure

• ???gfp Y.(ZC.reachable-bck(?,Y?ZC?d))
• gfp(?)
• Y? Ytrue
• Repeat until YY,
• YY
• YY?clock_eliminate
• ( ZC0?reachable-bck(?,Y?ZC?d), ZC)
• return Y

53
Outline

• Why ?
• TCTL inevitability analysis
• Performance solutions
• Experiments
• Strategy for combinations of techniques

54
Performance solutions

• Goal
• To build an arsenal of on-the-shelf reduction
  techniques

55
EDGF early decision on GFP evaluation

• Observation
• The state space shrinks iteratively
• Basic idea
• Stop at a gfp iteration if already no target
  states are in the gfp.
• Cost
• Small extra computation

56
EDGF early decision on GFP evaluation

• Example
• TargetIdentified???TargetHit
• After negation,
• TargetIdentified ? ?? ? TargetHit
• Can quit evaluation iff
• the intersection is already empty!!!

57
Tolerating Zenoness

• Observation
• Well-designed ystems may satisfy inevitability
  properties even for non-Zeno computations
• Less expensive procedure
• ???gfp Y.(??time-bck(??Ve?Txtion_bck(Y,e)))
• Trade-off for precision
• Over-estimate the greatest fixepoints

58
Abstract model-checking with TCTL?

• A safe abstraction
• Over approximate with subclass TCTL? of TCTL
• Extend procedure of reachable-bck
• Reachable-bckO(?1,?2)
• lfpY.abs(?2?(?1?time_bck(?1?
• Ve?Txtion_bck(Y,e))))
• abs()
• A generic abstraction procedure
• In our tool red, we have implemented
• Game abstraction
• Game discrete abstraction
• Game magnitude abstraction

59
Outline

• Why ?
• TCTL inevitability analysis
• Performance solutions
• Experiments
• Strategy for combinations of techniques

60
Experiments

• red 4.1 Our implementation
• A full TCTL mode-checker/simulator
• CRD (Clock-Restriction Diagram)
• VMCAI2003
• Counter-example
• Coverage estimation for TA symbolic simulation
• FORTE2003

61
Experiments

• Benchmarks
• PATHOS real-time operating system scheduling
  specification
• ??(pending3???running3)
• Leader election specification
• ??(parent1NULL??ii?1,(parenti?NULL?parentilti))
• CSMA/CD protocol
• A ??((transm1?transm2)?x.??(xlt26?bus_idle))
• B ??((transm1?x1?52)???wait1)
• C ??(bus_idle???(bus_collision???bus_idle))
• Pentium 4 / 1.7GHz / 256MB / Linux

62
Performance w.r.t parameter for measuring
time-progress

• Learn how to choose the value of time-progress
  parameter d for verification performance

PATHOS
63
Performance w.r.t parameter for measuring
time-progress

• Learn how to choose the value of time-progress
  parameter d for verification performance

Leader Election
64
Performance w.r.t parameter for measuring
time-progress

• Learn how to choose the value of time-progress
  parameter d for verification performance

CSMA/CD (A)
65
Performance w.r.t parameter for measuring
time-progress

• Learn how to choose the value of time-progress
  parameter d for verification performance

CSMA/CD (B)
66
Performance w.r.t parameter for measuring
time-progress

• Learn how to choose the value of time-progress
  parameter d for verification performance

CSMA/CD (C)
67
Performance w.r.t.non-Zeno and EDGF
68
Performance w.r.t.abstraction techniques
69
Performance w.r.t.Kronos 5.2
70
Semiformal verification

• or Symbolic simulation
• or Symbolic trajectory evaluation
• Use BDD to represent symbolic traces for better
  coverages.

71
Numerical Coverage Estimation for theSymbolic
Simulation of Real-Time Systems

• Farn Wang
• EE, National Taiwan University, Taiwan, ROC
• Geng-Dian Hwang, Fang Yu
• IIS, Academia Sinica, Taiwan, ROC

72
Outline

• Introduction
• Previous work
• Framework of verification
• TA arc coverage metric (ACM)
• Back-and-forth region coverage metric (RCM)
• Triggering-condition coverage metric (TCM)
• Experiments

73
Introduction

• 50 development resource ? Verification
  Integration
• Depends on coverage metrics to measure
• progress and
• Confidence
• Mythical ? / not the issue of this work /
• Still cannot get enough coverage before project
  deadline
• Formal verification new hope ?
• Intrinsic high complexity!
• Functional completeness!
• Simulation Formal verification ? Symbolic
  Simulation
• Using symbolic states for a set of concrete
  states
• Using symbolic traces instead of concrete traces
• Forward simulation strongest postcondtion
  calculation
• Backward simulation weakest precondition
  calculation

74
Why Symbolic Simulation ?
State-Space
75
What is coverage ?

• Partition the verification function into portions
• A portion is visited if one of its elements is
  visited.
• Use the
• visited
  portions
• all
  portions
• to evaluate the progress of verification.
• EXAMPLE
• FSM ARC elements?arcs portions ?
  arcs
• Line elements ?code lines portions
  ? code lines
• State elements?states portions
  ? states
• Region elements?states portions
  ? regions

coverage
76
Why coverage ?

• Too many functional elements (e.g. states) to
  visit!
• To save the development budget,
• Make the portions big enough
• only visit a representative in each portion
• Need a number to tell your boss

77
Why Symbolic Simulation ? (revisited)

• In a dense or infinite-state systems,
• state coverage of concrete traces is always 0.
• Needs
• Symbolic traces with nonzero volume
• New coverage metrics for infinite and dense
  systems

78
Previous work

• VLSI industry
• Statement
• transition
• abstraction of models and semantic control over
  transition coverage
• observability issue
• GOTCHA Generator of Test Cases for Hardware
  Architecture
• a prototype coverage-driven test generator
  implemented as an extension of the Murphi
  model-checker
• Coverage "completeness" w.r.t. CTL Ho, et al
• simulation formal verification Dill
• Coverage estimation for dense and infinite
  systems ????

79
Framework of verification

• System model timed automata (TA)
• A(Q, X, I, ?, E, ?, ?)
• Symbolic Simulate(A, p) /A is a TA p is the
  safety state predicate./
• Compute the estimation f of the whole target
  function F.
• ?the initial condition of A
• do
• Select ? ? ?, T ?E
• ? ? ? ?e?T next(A, ? ,e)
• Compute the estimation v of the verified
  proportion
• Print the numerical coverage estimation v/f
• until (fixpoint is reached or v/f gt threshold
  or ???p)

80
Criteria for good coverage metrics

• Accountability Each portion is accounted for
  once and only once.
• Coverability 100 coverage can be done.
• A tight estimation of the function!
• Efficiency low overhead!
• Discernment risk states and non-risk states fall
  in different portions.
• Interestingly, people seldom talk about what
  makes of good coverage metrics.

81
TA arc coverage metric (ACM)

• Portions arcs of TAs
• (1,7), (1,10), (4,7), (4,10), (2,8), (5,8),
  (3,6,9)

Idle
Idle
Idle
1
4
3
6
7
2
!start x10
5
8
9
!start x20
?collision
?collision
?start
?end
!end x1lt5
!end x2lt5
!collision !collision
busy
busy
busy
collision
10
?start
Sender 1
Sender 2
Bus
82
TA arc coverage metric (ACM)

• ACM can very often be too coarse to discern risk
  states

83
Back-and-forth region coverage metric (RCM)

• How to estimate state-coverage
• for dense-time systems ?
• Concrete state (q, ?), ? ? R
• Symbolic state (q,q, ?),
• ? Boolean formulus of c x-x c
• Three Challenges
• tight estimation of the target ? coverability
• estimation of a zone ? rough estimation of
• volume,
• estimation as a set of zones ? accountability

84
Challenge I tight estimation of the target
function

• How to get rid of as much irrelevant states as
  possible ?
• The target F ? B
• F the untimed quotient structure of the
  state-space of A from initial states
• B the magnitude quotient structure of the
  backward reachable state-space from risk states

85
Challenge II coverage estimation of a zone

• For the efficiency, we intuitively compute
  something like a normalized volume.
• For each constraint, partition the real number
  lines into the following 4Cmax3 basic intervals
• (-8, -Cmax),-Cmax,-Cmax,,-1,-1,(-1,0),0,0,(
  0,1),1,1,,Cmax,Cmax, (Cmax, 8)
• For example, -3x-xlt2 is 10
• The normalized volume of (Q,K) is

86
Challenge II coverage estimation of a zone

• Encoding in implementation -3x-xlt2
  3-(-6)110
• Assume Cmax is 5
• (-8, -5) -5 -4 -3 -2 -1 0 1 2
  3 4 5 (5, 8)
• -10 -8 -6 -4 -2 0 2
  4 6 8 10
• integer
  points
• -11 -9 -7 -5 -3 -1 1
  3 5 7 9 11
• Open intervals between
  integers

87
Challenge III estimation for a set of zones

• Zones may intersect with one another and
  accountability may not be maintained
• DBMs do not work since they may intersect.

x2
(0ltx13 ? 2ltx2lt4 ? x2-x1lt3) ? (2ltx1lt5 ?
1ltx2lt3)
x1
0
88
Challenge III estimation for a set of zones
(0ltx13 ? 2ltx2lt4 ? x2-x1lt3) ? (2ltx1lt5 ?
1ltx2lt3)
x1
x2
(3,5)
(0,2
(2,3
x2
x2
x2
3,4)
(1,2)
2,3)
(2,4)
(1,3)
x1-x2
x1-x2
(0,2)
x1-x2
x1-x2
x1-x2
(-2,0
(-1,1
(-3, 0)
(0,4)
0
x1
true
(4,8)
89
recVolumeCDD(D)

• if D is true, return 1
• else if (D,R)?F, return R
• else if D is a BDD (x,(I1,D1),,(In,Dn)),
• R(S1?i?n Ii recVolumeCDD(Di))/Ux
• else if D is a CDD (x,(I1,D1),,(In,Dn)),
• R(S1?i?n Ii recVolumeCDD(Di))/(4Cmax3)
• F F? (D,R)
• return R

90
Challenge III estimation for a set of zones
(0ltx13 ? 2ltx2lt4 ? x2-x1lt3) ? (2ltx1lt5 ?
1ltx2lt3)
x1
x2
(3,5)
(0,2
(2,3
(3/23)(5/23)
17/529
21/529
x2
x2
x2
3,4)
(1,2)
2,3)
3/23
4/23
(2,4)
(1,3)
x1-x2
x1-x2
4/23
(0,2)
7/23
x1-x2
5/23
x1-x2
x1-x2
(-2,0
(-1,1
(-3, 0)
(0,4)
0
x1
true
(4,8)
91
Back-and-forth region coverage metric (RCM)

• RCM may result in low coverability.
• It is difficult to efficiently and tightly
  estimate the reachable state-space.

92
Triggering-condition coverage metric (TCM)

• A balance between ACM and RCM
• Function triggering conditions of all
  transitions
• Normalized volume of ?
• ?e?T normalized volume of abstracte(???(e))
• abstracte() eliminates constraints irrelevant to
  ?(e).
• ? low overhead
• ? less portions

93
Triggering-condition coverage metric (TCM)
94
Experiments with Bluetooth L2CAP

• The L2CAP defines the actions performed by a
  master and a slave

95
Coverage estimation when there is no fault
96
Coverage estimation when there is a fault

• Six L2CAP faulty models
• Two simulation strategies

97
Testing of real-time systems

• Issues in the design of a good coverage metrics
  for dense systems
• Techniques to estimate dense state-space
• Groundwork for new research direction
• Test simulation coverage of dense systems

(010, 3)
(100, 22)
(000, 18)
(001, 12)
(110, 10)
(input,time)
Input has to be fed in at the right times to
track the suspicious traces.