Guiding Simulation with Increasingly Refined Abstract Traces

1
Guiding Simulation with Increasingly Refined
Abstract Traces

• Kuntal Nanshi
• Fabio Somenzi
• University of Colorado at Boulder

2
Outline

• Background
• Dynamic Verification
• Static Verification
• Abstraction Refinement
• Motivating Example
• Algorithm Overview
• Overall algorithm
• Simulation-based concretization
• Need for Care Set
• Refinement of the abstraction
• Experimental Results
• Conclusions

3
Verification Flows

• Dynamic Verification (Simulation)
• Commonly used in industry
• Scalable
• Probability of simulation finding a bug
  vanishes rapidly with the depth of search
• Static Verification (Model Checking)
• Automatic and thorough check, search conducted
  over entire state space
• Complete and sound
• Limited by capacity constraints
• Large analysis required for complex anddeep
  models

4
Abstraction Refinement

• Approach to mitigate the limits of model checking
• An abstract model is refined until it has enough
  detail to either prove a property true or produce
  a counterexample
• Counterexample is a sequence of states starting
  from initial state to a state where property
  fails
• Need to concretize
• Methods for concretization
• SAT
• Simulation

5
SAT Based Concretization

• Common approach requires that trace found in
  concrete model is one-to-one correspondence with
  abstract counterexamples(s)
• Concrete model is unrolled as many times as the
  length of abstract counterexample
• Simple Satisfiability check with each concrete
  state constrained to match corresponding abstract
  state
• Considerable progress made in the field, has
  extended the reach of model checking
• Concretization checks for long counterexamples
  are too much for even todays sophisticated SAT
  solvers

6
Simulation Based Concretization

• An alternative to SAT based approaches
• Pseudo-random vector based simulation with
  information extracted from abstract models
  (a.k.a. hybrid verification)
• Early attempts focused on improving the test
  coverage of a model
• Attempts were made to prove that certain target
  states are not reachable, i.e. bugsdo not
  exist
• Recent attempts in guided simulation, require
  concrete counterexamples to be of equal length to
  abstract ones

7
Motivating Example B12 (ITC99)

• Counter example for invariant property nloss -gt
  nl30 15

Abstract model
8
Our Approach

• Focus on invariants
• Predicates that hold for all reachable states
• Get initial Abstraction
• Localization the form of abstraction
• Model check property overabstract model
• Property passes in abstract model, passes in
  concrete model as well
• Property fails in abstract model, returns all
  abstract counterexamples (ACEs) in the form of
  Synchronous Onion Rings (SORs) data structure
  introduced in GRAB
• C Wang et al. ICCAD03

9
Our Approach - 2

• Concretization (with guided simulation)
• Concretize abstract counterexamples withguided
  simulation
• Simulation trace is constrained to a potential
  concrete counterexample path
• Refinement of the abstraction
• Adopted refinement procedure from GRAB
• If concrete counterexample is not found, abstract
  counterexamples are declared spurious
• Refine the abstract model by adding invisible
  variables - state variables from the concrete
  model that are not in the current abstraction

10
simulAbs

• Concretize ACEs by sequentially simulating
  concrete model with pseudo-random vectors
  starting from initial state
• Simulation trace is tracked and constrained to a
  potential counterexamples with help of SORs and
  hence guided
• Abstract transition can map to multiple concrete
  transitions

SORs representing abstract counterexamples
Concrete counterexample
11
Care Set

• No path from state g back to any in SORs
• Path from h to c, bad state f can still
  potentially reached from h
• Additionally, path from b to a is also a
  valid as e or f could still be reached
• Care Set is a set of states in the abstract
  model that potentially lie on counterexample path

12
Backtrack

• Guided simulation does not guarantee that
  simulation progresses towards target states
• Backtracking procedure is triggered when
  simulation does not reach the next target within
  the number of allotted vectors

Concrete counterexample
13
Refinement of Abstraction

• Current Abstraction may not have enough
  information to produce a counterexample
• Set of invisible variables are added to the
  abstract model with their associated transition
  relations
• Invisible variables that have a greater chance in
  keeping abstract model minimal are heuristically
  chosen
• Set of refinement variables not minimal
• Refinement Minimization
• Greedy procedure
• Remove each variable from refinement set
• Add back the variable if ACEs reappear

14
Complete Sound

• Complete
• Abstract model is refined in case simulAbs does
  not find a counterexample
• Abstract model eventually reaches the concrete
  model, for a failing property simulAbs returns a
  counterexample path from the SORs
• Sound
• Abstract model retains all possible behaviors of
  the concrete model over visible variables
• Failure is declared only if a concrete
  counterexample is found

15
Experiments

• We compared our approach to
• CI BDD based invariant checking algorithm
• GRAB abstraction refinement algorithm with SAT
  based concretization
• Our experiments were implemented in VISURL
  http//vlsi.colorado.edu/vis
• Circuits for comparison are from industry and VIS
  verification benchmarks with a mix of failing and
  passing properties
• 19 properties 12 failing and 7 passing
• Limit run times to 5 hours
• Number of attempts for simulAbs limited to 4,
  number of vectors increased after each attempt

16
CI vs. simulAbs
17
GRAB vs. simulAbs
18
Conclusions

• A new approach to checking invariant properties
  whose shortest counterexamples are very long
  traces
• Efficient guided simulation method
• Includes backtrack in case simulation gets stuck
  in a dead end state
• As examples get more complex CI is hit by
  capacity constraints, while for longer
  counterexamples SAT check in GRAB runs out of
  steam
• Experimental results show substantially better
  performance for long counterexamples, holds up
  pretty well for short counterexamples and passing
  properties

19

• Thank You

20
Overall Pseudo Code

• abstractionRefinement(M, F)
• Ma getInitialAbstraction(M, F)
• while(TRUE)
• SORs modelCheckAbstractModel(M, F)
• if(SORs empty) return TRUE
• if(Ma M)
• return(FALSE, extractOnePath(SORs))
• CCE simulAbs(M, SORs)
• if(CCE not empty) return (FALSE, CCE)
• Ma refineAbstraction(M, Ma, SORs)

21
Simulation with Check Pseudo Code

• simulateWithCheck(vectors, s, R, careSet)
• foreach vector
• d simulate(vectori, s)
• if(d ? R)
• add d to CES
• return CES
• else if (d ? careSet)
• add d to CES
• s d
• else discard vectori
• return FALSE

22
simulAbs Pseudo Code

• simulAbs(SORs, M)
• si getInitialState(M, S0)
• while(attempt Allowed)
• vectors getPseudoRandomVectors(num)
• CES simulateWithCheck(vectors, si, Ri1,
  careSet)
• if(CES is not empty) // success
• incr i
• CCEi CES
• if (i L) return (TRUE, CCE)
• si si1
• else
• incr attempt
• si backtrackSim(CCE, attempt)
• decr i
• return FALSE

23
Motivating Example - 2

• Concrete counterexample length greater than
  30,000
• An abstract model with nloss, nl and gamma
  provides the summary of concrete counter-example
• Abstract counterexample length 14
• Plausible to use values of gamma as guide points
  for concrete counterexample

24
Care Set - 2

• Set of abstract states that are on potential
  counterexample path to bad states
• States in SORs and few states that did not get
  included in SORs because of abstraction
• Set of states reached by series of pre-image
  computation starting from bad states
• Backward reachable analysis restricted to
  reachable states

25
Backtrack - 2

• Simulation trace reaches milestone state D mapped
  to abstract state d ? R2
• Backtrack makes one more attempt from D
• If unsuccessful, back track to B mapped to state
  b ? R1
• Discard counterexample segment (B, D)