A Proposed Methodology of Verification for Field Programmable Gate Arrays within the Aerospace Industry

1
A Proposed Methodology of Verification for Field
Programmable Gate Arrays within the Aerospace
Industry

• Melanie Berg

2
Agenda

• Verification Methodologies Past, Present, and
  Future
• Verification Process Development
• Creating a Test Bench
• Re-usability (package implementation)
• Self-Checking

3
Verification Methodologies - Objectives

• Make sure the FPGA is doing the right thing
• Make sure the FPGA is not doing the wrong thing
  Break it!!!!
• Make sure Bugs found are fixed
• Parallel development of test benches and RTL
• Goal first-pass success minimize time to market

4
Why Bother

• Common Responses
• I can do all of this in the Lab
• Lab is more efficient testing than simulation
• However
• Lab is more expensive hardware must be built
• Corner case can be difficult to invoke
• Debugging can be very hard to do in the lab
• Bugs can be more difficult to fix when found late
  in the cycle

5
The Importance of Verification

• Time is money!!!!!!!!!!!!!!!
• Finding Bugs earlier in the design cycle saves
  time
• Less debug effort
• Less lab time (man power)
• Less FPGA burning
• Easier Fix

6
The Importance of a Thorough Verification Process

• Has this ever happened to you?
• Not sure if the malfunction is the board
  electronics or the FPGA?
• Why does this work most of the time but
  sometimes
• Based off of a phone call you included
  functionality that wasnt quite what was desired.
• System level change has occurred (assumed not to
  affect the FPGA)no one contacted you but guess
  what!
• Corner cases were not pointed out, not designed
  for and not thoroughly tested
• Corner cases developed because the designers
  technique added extra complexity
• Power-UP nothing works flip flops and outputs
  come up in undefined states
• Power Down board components are affected due to
  uncontrolled circuitry

7
Early Verification Methods
Design RTL
Output Vectors
Simulator
Input Vectors

• Batch Mode Simulation
• Designer supplies RTL code
• Designer or verification engineer supplies Input
  test vectors (Vector Based Test-cases)
• Expected results are pre-calculated and used to
  compare to the simulation output
• or Waveforms are used to verify the simulation

8
Current Verification Techniques
Design RTL
Simulator
Test bench Monitors
Test bench Generators

• Must use a Separate team of Verification
  Engineers vs. Design Engineers
• Test bench using peripheral models (BFMs)
  Real-Time Output Responses
• Abstraction by loops, procedures, functions
  (behavioral modeling)
• Automated verification of output values
  (self-checking)
• Use of design techniques that will allow for most
  of the verification to be preformed in the RTL
  domain vs. the Gate-level domain.
• Objective is to cover the highest percentage of
  design state-space within an allotted amount of
  time.

9
Verification Strategies that Increase Coverage

• Simulations must run faster more tests are
  performed
• Tests should cover real cases stay inside of
  specifications and dont waste time
• Corner cases must be carefully thought out and
  invoked
• Real time Stimuli must be supplied (Bus
  Functional Modeling)
• Verification Metrics must be established (code
  coverage, functional coverage)
• Self Checking must be utilized
• Timing Analysis must be performed
• Static Synchronous portion of circuit
• Special Case Asynchronous Domain Crossings

10
Verification Strategies Continued

• Design/Verification Process must be well defined
• System Level Architectural Planning
• Unambiguous Documentation
• Reviews
• Version Control (code and documentation)
• Designs must be discrete and as simple as
  possible (Synchronous design with no spaghetti
  code)

11
Simulation Speed-Up Techniques

• Move most of the Verification to the RTL Domain
  vs. the Gate-Level Domain can reduce simulation
  time for just one test from days to minutes.
• How?
• Synchronous Design Techniques
• plus thorough Static Timing Analysis
• and Formal Verification (gate level matches RTL
  checking)
• Replace Gate-Level Cores with sound Behavioral
  RTL models

12
Simulation Speed-Up Techniques Synchronous
Design

• A design with a single clock domain can speed up
  simulation immensely.
• The internal design has moved away from having
  asynchronous continuous domains to a synchronous
  discrete domain less test cases are necessary.
• However, this does not fully remove the necessity
  for gate level testing
• The Input interface is considered as a separate
  clock domain and must be tested for corner case
  conditions
• Outputs must be checked for glitches

13
Example Multiple Clock Domains -Common Corner
Case Error Condition
14
How do we Verify Asynchronous Behavior?

• An efficient Verification Test Bench must cover
  corner cases due to clock domain crossings
• We need to verify what occurs within the clock
  domain that will extract data? (for our example
  clock x domain)
• If data transitions right before the clock edge
• If the data transitions right After the clock
  edge
• If the data transitions During a clock edge
• If the data transitions far enough from the clock
  edge
• Common Misperception Random test cases will
  cover the corner cases

15
Synchronous Verification vs. Asynchronous

• A synchronous design that meets timing will not
  have all of the corner cases listed on the last
  slide
• For just that data signal crossing clock domains
  we have increased the verification test cases
  by 4x.
• Simulation time will also be increased. In order
  to invoke the corner case conditions Gate level
  Verification will have to be performed
• Although there is a trick to get around this
• Most Efficient Design Technique is to stay away
  from multiple clock domains when possible

16
Simulation Speed-Up Techniques

• So far we have pointed out how Synchronous design
  techniques can speed up the Verification Effort
• Another method that will induce speed-up is
  Behavioral Modeling Replacement
• Replace Gate Level cores with Behavioral Models
• Write the Test Bench at the Behavioral Level and
  KISS
• Replace corner case HDL modules with special
  Behavioral models that can invoke corner case
  conditions at the RTL level alleviates the
  necessity for gate level testing

17
RTL Code for A Metastability Filter
process(sysclk,reset) begin if (reset
active_reset) then SigIn_d1 lt 0 SigOut lt
0 elsif rising_edge(sysclk) then SigIn_d1
lt SigIn SigOut lt SigIn_d1 end if end
process
18
Behavioral Code for Metastability Filter

• process (reset, sysclk, SigIn)
• begin
• if (reset '0') then
• SigIn_d1 lt '0'
• elsif (rising_edge(sysclk) and
  (sysclk'last_value '0') and (not
  SigIn'stable(Tsu) or SigIn'event)) then
• if flip then
• SigIn_d1 lt not SigIn
• else
• SigIn_d1 lt SigIn
• end if
• flip lt not flip
• elsif rising_edge(sysclk) then
• SigIn_d1 lt SigIn
• end if
• if reset '0' then
• SigOut lt SigIn_d1
• elsif rising_edge(sysclk) then
• SigOut lt SigIn_d1

19
Simulation Comparison of Behavioral Metastability
Filter vs. RTLRTL will always catch a slightly
early signal although Tsu is violated Thus the
RTL output will always be the same shape as the
input 2 cycles delayed.Behavioral replacement
can randomly (or controllably catch the incoming
signal upon Tsu violations thus can take on a
different shape than the input pulse
20
Behavioral Replacement

• Can be accomplished by using generate and use
  clauses in order to differentiate between
  simulation and synthesis.
• There should always be simulations performed that
  do not include the behavioral model must test
  what is being synthesized
• Behavioral model replacement expands the design
  state-space coverage while avoiding gate-level
  simulations
• MAJOR SIMULATION TIME SPEED-UP WITH EXPANDED TEST
  COVERAGE!!!!!!

21
Design Techniques KISS

• Use Synchronous Design Methodologies tools and
  verification techniques are set up to handle it
• Why?
• Complexity can create hidden corner cases
• Complexity (usually) incurs more logic gates and
  thus increases your verification state space
• Design Reviews become extremely tedious
• Bottom Line Complexity can get to the point
  where no one knows what your designing (even you).

22
Verification Plan

• Part of a defined Process
• Creation of Specifications for the verification
  effort
• Defines steps to be taken to reach goal
• Reviewed by all stakeholders (upstream and
  downstream)
• Starting point for the verification
• Should not start coding test harness until Plan
  is complete
• Used as the golden reference
• Think in terms of Reusability

23
Verification Metric

• Problem When are we done?
• Must define completion up front
• Identify what features must work
• Describe how these features are invoked and
  checked
• Describe what response particular input states
  produce
• Define how correctness is determined
• Cannot be accomplished without a Specification
  Document
• Can use vendor supplied tools Code Coverage, Bug
  Tracking (analyzes the frequency of errors during
  the lifetime of verification

24
Verifying the Response

• Plan how to check the response
• Self-Checking Calculate and compare on the fly
  (great for regression testing)
• Waveform viewing only good for initial portion of
  verification
• Creation of efficient stimuli in order to cover
  a large state space of responses (BFM)
• Detect, report, and analyze Errors as early as
  possible

25
Create a Test Bench Library that Contains the
Following Routines for Reusability

• Stimuli Generation
• Clock oscillator
• Change of inputs/stimulus (at a clock edge at a
  random time or at defined corner case times)
• For-loops to generate all possible input
  combinations (data generators)
• Random data generation
• Random wait times
• File output routines (error reporting)
• Interface commands (I.e. CPCI_WRITE, VME_READ)

26
Create a Test Bench Library that Contains the
Following Routines for Reusability (Cont)

• Self-Checking of FPGA Outputs
• Check frequency of a signal
• Check distances of signal spacing
• Verify that a signal occurs n cycles after a
  particular input state/sequence
• Verify all sequences of data have occurred (very
  easy if data is sequential however may need to
  invoke more difficult data patterns)
• Check that this will never occur when this
  occurs or this will always occur when this
  occurs remember never and always are truly
  formal verification directives (used as an
  example within simulation limitations)

27
Test Bench Routine Examples Clock Oscillator
procedure clock ( constant clock_period in time
signal clock_out out std_logic constant
phase in time 0 ns) is begin wait for
phase loop clock_out lt '1' wait for
clock_period/2 clock_out lt '0' wait for
clock_period/2 end loop end clock
28
Test Routine Examples Distance between Signals
Self-Check
procedure check_distanceRF(signal ARG in
std_logic ARG2 string signal ARG3 in
std_logic ARG4 string period in time
File TestFile text constant eps in time 0
ns) is variable first_time time variable
second_time time constant error_head string
"ERROR during distance Length Check " ARG2 "
to " ARG4 begin wait until rising_edge(ARG) fi
rst_time now wait until falling_edge(ARG3)
second_time now if (((second_time -
first_time) gt (period eps)) or ((second_time
-first_time) lt (period - eps)))
then write_error(error_head,to_string(second_tim
e-first_time),to_string(period),TestFile) end
if end check_distanceRF
29
Test Bench Behavioral Models

• Use them for
• Modeling FPGAs (or some subset of the FPGA)
• Modeling Board Components that surround the FPGA
  (BFM Bus Functional Models)
• Test bench verification
• Much faster to write than RTL
• Simulates faster
• Great for reusability

30
Summary

• We Covered Several Verification Methodologies
• Design for Verification
• Simulation Speed-Up Techniques
• Utilizing Common Routines
• Behavioral Modeling
• Self Checking Output Verification
• It is suggested to follow a well defined Design
  and Verification Process
• Plan Ahead Create a Verification Plan
• Usage of the aforementioned techniques is a must
• Documentation, Documentation, Documentation!!!
• Must have Reviews of design and Verification

31
Summary End Result

• By Following a strict Verification scheme, we
  will be able to save Time, Money and gain
  confidence within each project