Teaching%20Functional%20Formal%20Verification

1
DAC 2002

• Teaching Functional Formal Verification

Dr. Yaron Wolfsthal Manager, Formal Methods IBM
Haifa Research Laboratories
2

• Overview

• Functional Formal Verification - Motivation and
  Taxonomy
• Property Verification
• Model Checking
• The RuleBase tool, the Sugar language
• The pragmatic view
• Experience using RuleBase
• IBM Server Group
• IBM's industrial partners
• Example The Buffer
• Summary, final remarks, and answers to some
  questions which students often ask

3
Taxonomy of Verification

• what you specified is what you envisioned
• functional verification - specification level
• what you designed (RTL) is what you specified
• functional verification - implementation level
• what you taped-out is what you designed in RTL
• equivalence checking
• what was manufactured is what you taped-out
• production testing

4
The Functional Verification Problem

• Goal eliminate H/W design defects before a
  product is shipped

• Major Challenges
• Micro-architecture complexities grow
• Market requirements get tougher
• Silicon technologies improve
• Verification Exposure Quality risks, soaring
  costs

5
Motivation for Functional Formal
Verification Philosophy Behind Formal Methods

• Engineers build mathematical models of systems so
  they can predict their properties through the
  power of calculation
• It's the power of mechanized calculations (eg.
  fluid dynamics, finite-element analysis) that
  makes the construction of complex physical
  systems possible
• Formal Methods apply these ideas to the complex
  logical design of computer systems
• Build a formal mathematical model of some aspect
  of the system
• Calculate whether or not it has some desired
  property

6
Motivation for Functional Formal
Verification Historical Perspective of
Verification Solutions

• In the beginning there was Simulation
• event-based
• cycle-based
• clever test generation
• But... simulation alone has inherent limitations

Simulasaurus
7

• Example A 256-Entry Arbiter

... will require 1060 years on a simulator
executing 1012 c/s.
8
Motivation for Functional Formal
Verification Need for Methodology Shift Observed
in Industry

• COMPLEXITY

9
Motivation for Functional Formal
Verification Formal Verification vs. Simulation
(Unit View)

• Covering the Design Space -
• with
• Formal Verification (2)

10
The Property Verification Paradigm a.k.a. ABV,
Assertion-Based Verification
Conceptually, consists of two major elements

• Property Specification
• Using a language for formally specify functional
  requirements and behaviors of a chip
• interface protocols
• performance constraints
• Analysis
• Using a procedure for establishing that the
  requirements (properties) hold
• Note
• Property Verification in general is not limited
  to Formal Verification!
• In the context of this talk, however, we focus on
  FV, and in particular on an FV technique called
  "Model Checking"

11
Model Checking - Formal Property Verification
Doing it the Automatic Way

• Represent design as a finite state machine
• Automatically calculate truth or falsity of
  specification by traversing the state space
• "if a request is received, it will be processed
  within 3 clocks"

12
Sample Properties

• Temporal relationships between signal values
• External and internal protocols
• Cache coherency/consistency policies
• Mutual exclusion, absence of contentions
• Correctness of arithmetic computations

13
Key Formal Verification Method "Model Checking"

• A method for mathematically proving functional
  properties ('"specifications") on the design. No
  tests required
• Proving a property is showing that it holds for
  all possible input combinations, across all
  execution paths
• Improved verification quality, reduced cost
• less tapeouts
• less bugs in the field
• ideally, investment in simFV should remain flat

14
Model Checking Flow

• Answer questions like
• does my design satisfy these rules... e.g.
• Request will be always Processed within 3
  cycles
• Inputs
• The design
• The environment (irritator for its inputs)
• The rules (specification or properties to verify)
• Output
• A documented pass/fail answer (with error trace)

Request will be always processed within 3 cycles
Design (VHDL)
Environment (EDL)
Rules (Sugar)
Model Checker (RuleBase)
Pass/Fail
15

• RuleBase
• The IBM Model Checker

• Collection of powerful verification algorithms
  (BDD-based, SAT) under same front-end
• Will verify a design, typically control logic,
  against a set of properties, producing documented
  fail/pass answer
• Deals with industrial-size designs (a real issue)
• GA July 1996
• Version 1.4 released June 2002
• Used throughout IBM, by external licensees, and
  in academia
• Friendly University Program

16
User Experience Model Checking the IBM eServer
p690

• "We applied FFV to some extent on approximately
  40 design components throughout the processor and
  found more than 200 design flaws at various
  stages and of varying complexity. At least one
  bug was found by almost every application of FFV.
  In most cases, FFV began significantly later than
  verification. It is estimated that 15 of these
  bugs were of extreme complexity and would have
  been difficult for traditional verification. In
  some cases, a late bug found in verification or
  in the laboratory was recreated and its
  correction verified efficiently with FFV. "

17

• Formal Verification in Practice

• Tool Perspective (capacity)
• deal with state-space explosion
• User Perspective
• how to define "functional properties" (the
  specification)
• how to define "assumptions on input"
• usability

18

• Formal Verification in Practice

• Tool Perspective (capacity)
• deal with state-space explosion
• User Perspective
• how to define "functional properties
• (the specification)
• how to define "assumptions on input"
• usability

19

• RuleBase - Under the Hood

• Automated Reductions/abstractions
• cone-of-influence
• flip-flop equivalence
• Advanced Model-Checking Algorithms
• classic CMU algorithm
• on-the-fly checking of safety formulas
• bounded model checking
• guided search

20

• Formal Verification Challenges - Putting it to
  Work
• The Specification Perspective

• Tool Perspective (capacity)
• deal with state-space explosion
• User Perspective
• how to define "functional properties"
• (the specification)
• how to define "assumptions on input"
• usability

21
Sugar Goes Public

• April 22, 2002
• Sugar selected by Accellera as basis for IEEE
  standard specification language

Sugar-based tools already announced by Cadence,
Verplex, TransEDA, IBM
22
Specifying Properties - the Sugar 2.0 Language

• Natural, concise way for formulating properties
• English
• Whenever we see an assertion of signal req,
  followed by an assertion of signal ack, which is
  not followed by an assertion of signal cancel, we
  should see - starting at the next cycle - an
  assertion of signal start_trans, followed by
  between 1 and 8, not necessarily consecutive
  assertions of signal data_valid, followed by an
  assertion of signal write_end
• Sugar
• always req ack !cancel gt start_trans
  data_valid 1..8 write_end

23

• Formal Verification Challenges - Putting it to
  Work
• The "Environment Assumptions" Perspective

• Tool Perspective (capacity)
• deal with state-space explosion
• User Perspective
• how to define "functional properties" (the
  specification)
• how to define "assumptions on input"
• usability

24

• Why "Environmental Assumptions"

env read only
25

• How to Write "Environmental Assumptions"

• Same syntax as "properties"
• restrictions on input behavior
• State-Machine syntax (w/non-determinism)
• Verilog
• VHDL
• EDL

Universities Your installation of RuleBase
currently supports only Sugar 1.0, which has only
EDL flavor
26

• Example The Buffer

BUF is a design block that buffers a word of data
(32 bits) sent by a sender to a receiver. It has
two control inputs, two control outputs, and a
data bus on each side
27

• Example The Buffer

Communications (on both sides) takes place by
means of a 4-phase handshaking as follows When
the sender has data to send to the receiver, it
initiates a transfer by putting the data on the
data bus and asserting StoB_REQ (Server to Buffer
REQuest). If BUF is free, it reads the data and
asserts BtoS_ACK (Buffer to Server ACKnowledge).
Otherwise the sender waits. After seeing
BtoS_ACK, the sender may release the data bus and
deassert StoB_REQ. To conclude the transaction,
BUF deasserts BtoS_ACK. When BUF has data, it
initiates a transfer to the receiver by putting
the data on the data bus and asserting BtoR_REQ
(Buffer to Receiver REQuest). If the receiver is
ready, it reads the data and asserts RtoB_ACK
(Receiver to Buffer ACKnowledge). Otherwise, BUF
waits. After seeing RtoB_ACK, BUF may release
the data bus and deassert BtoR_REQ. To conclude
the transaction, the receiver deasserts RtoB_ACK.
28

• Property Specification
• Note This is not About Sugar Syntax!

Yet I will make an attempt to expose you to Sugar
2.0

• Basic functional properties
• Protocol properties
• Data-path properties

29
Property-Based Verification of the Buffer Basic
Functional Properties

• First Property
• Overflow (two reads without a write in between)
  cannot occur!
• Second Property
• Underflow (two writes without a read in between)
  cannot occur!

30
Property-Based Verification of the Buffer
Sugar Formulation of the
Overflow/Underflow Properties

• rule ack_interleaving
• formula
• "No overflow RtoB_ACK is asserted between
  any two BtoS_ACK assertions "
• always
• !RST rose(BtoS_ACK) true (
  rose(RtoB_ACK) before rose(BtoS_ACK) )
• formula
• "No underflow BtoS_ACK is asserted between any
  two RtoB_ACK assertions"
• always
• !RST rose(RtoB_ACK) true (
  rose(BtoS_ACK) before rose(RtoB_ACK) )

31
Property-Based Verification of the
Buffer Handshaking Properties

• The protocol sequence must always be honored
• Focus on Request signals first
• Request signals will not be prematurely asserted
• Once asserted, Request signals will stay high
  until acknowledged
• Next, focus on Acknowledge signals
• Acknowledge signals cannot spontaneously be
  asserted
• Once asserted, an Acknowledge signal will stay
  high until the triggering Request signal is
  turned off

32
Property-Based Verification of the Buffer Sugar
Formulation of the Handshaking Properties - LHS

• rule four_phase_handshake_left
• formula "A request can not be raised when
  ack is high "
• always !StoB_REQ BtoS_ACK
  StoB_REQ (false)
• formula "A request can not be lowered when
  ack is low"
• always StoB_REQ !BtoS_ACK
  !StoB_REQ (false)
• formula "An acknowledge can not be raised
  when req is low"
• always !BtoS_ACK !StoB_REQ
  BtoS_ACK (false)
• formula "An acknowledge can not be lowered
  when req is high"
• always BtoS_ACK StoB_REQ
  !BtoS_ACK (false)

33

• Property-Based Verification of the Buffer
• Sugar Formulation of the Handshaking properties -
  RHS

rule four_phase_handshake_right formula
"A request can not be raised when ack is high"
always !BtoR_REQ RtoB_ACK
BtoR_REQ (false) formula "A request
can not be lowered when ack is low" always
BtoR_REQ !RtoB_ACK !BtoR_REQ
(false) formula "An acknowledge can not
be raised when req is low" always
!RtoB_ACK !BtoR_REQ RtoB_ACK (false)
formula "An acknowledge can not be lowered
when req is high" always
RtoB_ACK BtoR_REQ !RtoB_ACK (false)
34
Property-Based Verification of the
Buffer Data-Path Property

• Basic transfer property
• Data sent to receiver (RHS) is the same as
  received from sender (LHS)
• To specify in Sugar, we can use next_event (b)(f)
  
• The next time that b holds, f must hold
• A conceptual extension of the next operator

35

• Property-Based Verification of the Buffer
• Data-Path Sugar Property

rule checking_data "Data sent to receiver
is the same data received from the sender
in the last write" formula always
((!RST rose(BtoS_ACK) DI(0)) d
next_event(rose(RtoB_ACK))( DO(0)) )
formula always ((!RST
rose(BtoS_ACK) !DI(0)) d
next_event(rose(RtoB_ACK))( !DO(0)) )

36
Property-Based Verification of the
Buffer Data-Path Property - Another Way

• Basic transfer property
• Data sent to receiver (RHS) is the same as
  received from sender (LHS)
• Question How do you remember what was sent?
• Answer You use an auxiliary state-machine for
  bookkeeping

37
The Bookkeeping FSM ("Satellite") Stores last
input driven by sender
sticky_bit if rises, it stays active forever.

• .

var tmp(0..31) Boolean assign init(tmp(0..31))
0 assign next(tmp(0..31))
IF rose(BTOS_ACK) THEN
DI(0..31) ELSE tmp(0..31) ENDIF
38
Property-Based Verification of the Buffer Sugar
Formulation of the Data Transfer Property

• var tmp(0..31)boolean
• assign init(tmp(0..31)) 0
• assign next(tmp(0..31))
• IF rose(BTOS_ACK) THEN DI(0..31)
• ELSE tmp(0..31)
• ENDIF
• rule checking_data
• "Data sent to receiver is the same data
  received
• from the sender in the last write"
• formula
• Always !RST rose(RTOB_ACK) (
  DO(0..31) tmp(0..31) )

39
This Syntax is Really the "Modeling Layer of
Sugar"
a.k.a. EDL - Environment Description Language
sticky_bit if rises, it stays active forever.
//sticky_bit //once asserted, stays active
forever.

• .

var sticky_bit Boolean assign init(sticky_bit)
0,1 assign next(sticky_bit)
case sticky_bit 1 else 0,1 esac
40

• F.A.Q.

• What tutorials are available?
• What else can be done to address large state
  spaces?
• What is the typical size of design units that can
  be addressed?
• What kind of bugs are we after?

41

• Tutorials

• Sugar Tutorial
• http//www.haifa.il.ibm.com/projects/verification/
  sugar/sugar_tutorial.pdf
• Generalized Buffer
• http//www.haifa.il.ibm.com/projects/verification/
  RB_Homepage/tutorials.html
• New! Fresh Arbiter from the Server Group
• contact Emmanuel Zarpas, zarpas_at_il.ibm.com

42

• Design for Formal Verification
  
• Proper Methodology is Useful for Overcoming Size
  Limitations

• Verify control logic alone
• leave datapath for simulation
• Abstract out internal parts
• Degenerate design parts by limiting environment
  behavior
• Partitioning
• Modes

43

• Partitioning for Model Checking (1)

44

• Partitioning for Model Checking (2)

45
Capacity (in terms of State Variables)

• Practical for block-level verification
  ("designer-level verification")
• Practical also - with proper training - for
• unit-level verification of complicated control
  logic (up to 1000's of FFs)
• system-level verification of abstract
  architectural models

46
Falsification, Bug Hunting A Pragmatic Approach

• Cannot provide absolute guarantee of success
• Done w.r.t. user-provided specification
• On abstract model which may not fully capture
  reality
• Focus on adding value to existing processes
• Finding bugs ASAP!
• Debugging, explore requirements

47
Sample Bug Found with RuleBase
The Rule If Cmd is pending in the L2 Queue,
eventually it will be routed to the L2
48

• Summary Teaching (Functional (Formal
  Verification))

• Model Checking is accessible to engineering
  students
• some will say essential
• prerequisites Digital Systems / Logic Design
• Property Verification via model checking is a
  powerful tool
• Start from the beginning
• design for formal verification
• pragmatically approach your verification problem
• thorough verification plan
• Friendly RuleBase University Program!

49

Back-Up Foils


50
User Difficulties in Model Checking

• User Survey, 2001

51
Formal Verification Challenges - Putting it to
Work The Usability Perspective

• Tool Perspective (capacity)
• deal with state-space explosion
• User Perspective
• how to define "functional properties"
  (the specification)
• how to define "assumptions on input"
• usability

52
Formal Verification Challenges - Putting it to
Work The Usability Perspective

• Counter-examples as waveforms/tests
• Witness Production
• Explanation of
• counter-examples
• witnesses
• vacuity (which precondition is always false)
• Reduction analyzer

53
Model Checking Classic Algorithm

• Searching for error states (e.g. where properties
  are violated) in expanding circles
• The wisdom is in the search algorithm!