ICS 216 Embedded Systems Validation and Test

1
ICS 216Embedded Systems Validation and Test
Instructor Professor Ian G. Harris Department
of Computer Science University of California
Irvine
2
Reading Material

• No single book.
• Some Verilog book is useful (some are on reserve
  at the library)
• Selected papers
• Useful book - Writing Testbenches Functional
  Verification of HDL Models, Janick Bergeron,
  Second Edition
• Useful book -Introduction to Formal Hardware
  Verification, Thomas Kropf

3
Course Structure

• Three problem sets two on simulation-based
  validation and one on formal verification
• Final Project - individual or pairs
• - Final Project Proposal - 2 pages
• - Final Project Proposal Presentation - 20 min.
• - Final Document - Complete description any
  code

4
Embedded Systems

• A computer that doesnt look like a computer

• Complex computations hidden behind a simple
  interface
• Ex. Cell phone, digital camera, automobile, etc.
• Design requirements are varied
• Power, performance, cost, life- critical
• Design components are varied
• Digital/analog hardware
• Systems/application software
• Mechanical sensors/actuators

5
Importance of Embedded System Validation
Validation is a bottleneck in the design process
High cost of design debug (designers,
time-to-market)
High cost of faulty designs (loss of life,
product recall)
Hardware/Software covalidation problem is more
acute
Hardware and software are often used together
Hardware and software are designed separately
Covalidation is performed late in the process,
necessitating long
redesign loops
6
Validation and Test

• Validation/Verification
• Ensuring that the design matches the designers
  intent/specification
• Detection of design errors accidentally made by
  the designer(s)
• Usually includes the debugging task as well
• (Manufacturing) Test
• Detection of physical defects in the
  implementation
• Defects appear as a result of manufacturing or of
  wear mechanisms
• Tests the implementation, not the design

7
Validation vs. Verification

• Formal verification - Use of proof-based
  techniques
• Model checking, equivalence checking
• Time complexity is high
• Confidence is high for specified properties
• Simulation based validation - Use of simulation
• Full-chip/full-design validation
• Confidence is not well quantified
• Pentium 4 bugs found by FV (492) vs. Validation
  (5809) 1

1 B. Bentley, Validating the Intel Pentium 4
Microprocessor, DAC01.
8
Embedded System Design Flow
Designers Intent Vague idea of behavior known
only to designer.
Intent
Natural Language Specification Complete
behavioral description Written in English (or
other).
Nat. Lang. Spec.
Executable Behavior A simulatable description of
the behavior. Written in a procedural language
(SystemVerilog, SystemC, )
Exec. Behavior
SW Design
Pre-Designed
HW Design
Fabrication
Embedded System
9
Simulation-Based Validation

• Use of simulation to compare two design
  representations

Intent
Not Simulatable
Nat. Lang. Spec.
Simulatable

• Comparison to natural language spec (or intent)
  requires manual interaction
• Comparison to executable behavior requires
  cosimulation of HW, SW, and pre-designed

10
Formal Verification, Model Checking

• Evaluating design to determine if properties
  always hold

Intent
Properties
Nat. Lang. Spec.

• Properties capture some aspect of the designers
  intent
• - A traffic light cannot stay RED forever
  AF(color!RED)
• Strictly limited in design complexity allowable

11
Formal Verification, Equivalence Checking

• Proving that two different structural design
  representations are equivalent

F1 a b F2 (ab)

• Works well for combinational logic. Requires
  manual interaction for sequential logic.
• Limited to structural hardware. Infeasible for
  behavioral descriptions.

12
Elements of Validation

• Writing a good Test Bench
• Evaluating a Test Bench - Coverage Metrics

• Using the simulator (vcs)

• Manual evaluation using the debugger (virsim,
  CLI)
• Viewing waveforms
• Insertion of breakpoints
• Automatic evaluation
• Comparison with known-good results
• Assertions
• Self-checking code