Putting It All Together: Using Formal Verification In Real Life

1
Putting It All Together Using Formal
Verification In Real Life

• Erik Seligman
• CS 510, Lecture 19, March 2009

2
Agenda

• Motivation
• FEV Advice
• Assertion-Based Verification
• FPV Advice
• CDC TOV
• Conclusions

3
Agenda

• Motivation
• FEV Advice
• Assertion-Based Verification
• FPV Advice
• CDC TOV
• Conclusions

4
Motivation

• Class has examined formal tools
• But if you adopt a tool, will it be used?
• Common mistake toss over wall to team
• Sometimes hard for naïve user
• Sometimes easy to misuse tools
• Need to think about plans methods
• Plan from beginning for FV success
• Be aware of changes of mindset
• Be aware of dangers of tool misuse

5
Review From Lecture 1
Simulation spot coverage of design space
6
Review From Lecture 1
Formal Verification (ideal case) full coverage
of design space
Simulation spot coverage of design space
7
Review From Lecture 1
Formal Verification (ideal case) full coverage
of design space
Formal Verification (real life) full coverage
in some areas
Simulation spot coverage of design space
8
Agenda

• Motivation
• FEV Advice
• Assertion-Based Verification
• FPV Advice
• CDC TOV
• Conclusions

9
Is FEV Automatic?

• Well-accepted on design teams
• Key to ensuring correct validation overall
• Main validation target RTL
• But silicon better represented by netlist
• So need to make sure RTLnetlist
• Opportunities in other areas
• RTL-RTL, netlist-netlist incremental checks
• But are there dangers?

10
Remember FEV Challenges

• Not-quite-state-matching elements
• Scan, minor retiming, clock gating,
• Opportunities for Bloopers
• Unchecked constraints
• Conflicting constraints
• Blackbox and library confusion
• Unreachable points
• Obscure tool behaviors (multi-drive handling)

11
FEV Advice

• Plan for FEV from beginning
• Have overall view of chip FEV find holes
• Teams should have central FEV expert
• Know how to handle state-matching variation
• Understand tool quirks
• Understand complexity techniques
• FEV runs should be saved auditable
• Always audit logs before tapeout

12
Agenda

• Motivation
• FEV Advice
• Assertion-Based Verification
• FPV Advice
• CDC TOV
• Conclusions

13
Assertions Part of the Design Process

• Define standard assertion note
• // Assertion a123 Check for legal grants
• Designer adds spec, testplans, RTL
• Assertion idea ! interrupt thought flow
• Assertion expert role
• Scripts to collect assertion notes
• Help designer implement/focus
• Assertions casual easy
• Pitfall Avoid requirements seen as penalty
• Must eventually prove X formally

14
Free Assertions

• Reuse blocks ? standard assertions
• Intel chipset group form for std FSM
• Tool creates RTL template assertions, covers
• FSM never stuck
• All states reachable
• Assertion packages with IP building blocks
• AMBA, PCI asserts from vendors?
• Develop asserts for internal standards

15
Encouraging Useful Assertions

• Focus on high-level intent
• Assertions executable comments
• Add insight to design
• Micro-assert on a couple of RTL lines less useful
• Dont be afraid of some modeling code
• Auxiliary calculations / wires are fine
• Provide ifdef to exclude from synthesis
• Full reference models in areas of concern
• Smaller shadow models often very useful

16
Shadow Model Example (PCIE)
DWORD 1 DWORD 2 DWORD 3
Link/Transaction Layer Interface (RTL under test)
Shadow Model transaction end?
TRANSACTION

• Not full ref model, but enables good asserts
• End of transaction ? commit within n cycles
• FV found serious chipset bugs missed in sims!

17
Challenges Of SVA

• Assertion languages like SVA often too general
• Designed by PhDs to cover all cases
• Provide building blocks, not designer tools
• Average designer has hands full
• Examples of SVA gotchas
• Glitch dangers
• always _at_(a or b) assert (a ! b)
• Sequences vs triggering
• assert property (a 1 b)
• assert property (a gt b)
• Operator definitions
• assert property (s1 and s2)
• assert property (s1 intersect s2)

18
Solution Assertion Libraries

• Dont require use of raw SVA
• Provide library encapsulating common situations
• Improves usability readability

19
Popular Library Solution OVL

• assert_always
• (OVL_ERROR,OVL_ASSERT,Bogus grant")
• a1(clk, reset_n, req!grant)
• Each assert is a cleanly defined module
• Implementation uses SVA at core
• Good, but still room for improvement
• Modules cant be used in SV procedures
• All arguments need to be typed
• Need to explicitly specify clk/rst
• Combo asserts can still glitch if misused

20
Need For Assertion Locality

• always_ff _at_(posedge clk) begin
• ltmany linesgt
• if (foo) begin
• ltmany linesgt
• if (bar) begin
• ltmany linesgt
• v1 lt messy_function(b,c,d,e,f)
• // want assertion on v1 here
• ltmany linesgt
• end
• ltmany linesgt
• end
• ltmany linesgt
• end
• // OVL assertion on v1 has to go here!

21
Improved SVA Libraries

• ASSERT_VERIFY(req!gnt,clk,rst)
• ERR_MSG(Bogus grant)
• Macro version improvements over OVL
• Macro can be placed in procedural code
• Arguments can be any type (even property)
• ? Most Intel projects moved to macro library
• IEEE p1800-2009 providing SVA extensions
• Checkers, inferred clks, LTL, etc.
• Will enable fully parsed library with same
  benefits

22
Agenda

• Motivation
• FEV Advice
• Assertion-Based Verification
• FPV Advice
• CDC TOV
• Conclusions

23
Effective Formal Property Verification
Prerequisites

• Include FV in testplans
• Part of validation, NOT out-of-band extra
• Pick good FV battles
• Size OK, many corner cases, reasonable design
• Decide on goals bug hunting or full proofs?
• FV by model expert
• FV reveals obscure corner cases
• Need deep understanding
• Cover points need to be in place
• Describe typical interesting situations
• Help judge FV env effectiveness

24
Preparing The FV Infrastructure

• FV is different for a simulation team
• Plug Play? sort of, but watch details
• Choose highly usable tools
• Main FV debug activity analyze counterexample
• Counterexample waveform violating assertions
• Fancy debug Wave views, what-if, multi-cex?
• Same assertions in FV and simulation
• Project-specific glue
• Wrapper scripts based on simulation env
• Set up default clock/reset config

25
Starting FV On A Design

• Initial runs are a feasibility test
• Complexity of FV hard to predict
• Not proportional to transistor count
• Small arithmetic can be very hard!
• If time/memory explodes, reconsider
• Lower hierarchy?
• Blackbox/prune parts of model?
• Also, look early at cover point proofs
• FV tool hits each cover point?
• Cover point not reachable ? overconstrained!
• Important quality check
• Repeat after new assumptions added

26
Assumption Creation Loop Majority of FV Time
Run FV
Analyze Failures
Add Assumes

• Mindset change from sim prepare team!
• Early runs have many false negatives
• More assumptions more interesting CEX
• Interesting bugs not found on first run
• Several rounds of assumes ? deep traces
• Be sure to check assumptions too, in simulation
  or FV

27
Assumption Count Exploding? Dont Give Up Yet

• Possible bad choice of boundary
• Effectively reimplementing neighbor block?
• Consider increasing hierarchy level
• Add upper level many blackboxes?
• Also consider simplifying problem
• Only cover certain modes
• Example PCIE prove for x16, not x4, x8?
• Restrict data
• Will one nonzero bus bit test most major logic?
• Are fully general packet payloads needed?

28
Example FV too hard?
MPE0
MRA0
MRA1
MPE1
MPE Memory Protocol Engine MRA Memory Read
Arbiter
29
Correct Hierarchy Makes FV Easy
MPE0
MRA0
MRA1
MPE1
MSB
30
Complexity Issues (Time,Memory)

• As mentioned before, hard to predict
• Issue may appear after adding assumptions
• Various strategies available
• Try running at lower hierarchy
• Consider blackboxing or freeing logic
• Restrict modes or data
• Reduce size of parameterized elements
• Intelligent pruning
• Eventually may need to waive
• Again, dont give up too soon
• FV of partial design space still useful!

31
Bounded vs Full Proofs

• Which of these do you need?
• Assertion can NEVER be violated
• Assertion can never be violated by any possible
  simulation of length up to ltngt
• Bounded proof usually easier for tools
• Use cover point proofs to judge good bound
• Bound lengths of interesting scenarios
• Some coverage lost vs full proofs
• But often at point of diminishing ROI
• Consider modifying starting state
• Fill queue at start of proof?
• Targeted cover point proof as starting state for
  next FPV run

32
Consider FPV At Various Project Phases

• Early design
• Excellent tool to understand RTL modules
• Allows what-if experiments at any hier level
• Validation
• Include in validation plans
• Can greatly improve coverage of design
• Complete coverage of some subspace!
• Late bugs / Post-silicon
• Use FPV (guided by cover points) to replicate
  unexpected errors
• Heroic FPV effort may be justified
• If late error found that was missed by other
  methods

33
Agenda

• Motivation
• FEV Advice
• Assertion-Based Verification
• FPV Advice
• CDC TOV
• Conclusions

34
CDC TOV

• Niche techniques using formal methods
• Small, specialized problems
• But huge impact from avoiding bugs!
• Failures create intermittent silicon issues
• Understand opportunities apply!
• Know your CDC tools, use effectively
• Set parameters to minimize false positives
• Have plan for TOV verification
• Even without CAD tool, can create assertions
• Check assertions in sim if FPV too hard

35
Agenda

• Motivation
• FEV Advice
• Assertion-Based Verification
• FPV Advice
• CDC TOV
• Conclusions

36
Conclusions

• Formal Verification techniques have reached
  maturity
• Usable by non-PhDs
• Usable by design engineers
• Significant opportunities for ROI
• Complete coverage of part of design space
• Like running exponential number of tests
• Be ready for the opportunities!
• Enhance maturity of FEV usage
• Understand Assertions, FPV, CDC, TOV
• Introduce to design team when applicable
• Dont be scared off by usage models
• Different from current methods, but not really
  that hard

37
References / Further Reading

• http//www.soccentral.com/results.asp?CategoryID4
  88EntryID19314
• http//oskitech.com/papers/foster-ftptalk-dvcon06.
  pdf
• http//videos.dac.com/42nd/papers/41_3.pdf
• http//www.erikseligman.com/docs/DAC2008_FPV_Selig
  man.pdf
• http//www.erikseligman.com/docs/dvcon08_zep.pdf