Verification Through the Principle of Least Astonishment

1
Verification Through the Principle of Least
Astonishment

• Beth Isaksen and Valeria Bertacco
• Advanced Computer Architecture Lab
• University of Michigan, Ann Arbor, MI

2
The Challenges of Verification Today
As hardware designs continue to grow, hardware
designers and verification engineers must respond
with ever-higher standards of productivity and
quality.
Figure adapted from www.intel.com
3
Traditional Approach RTL and Waveforms
RTL descriptions are usually long and difficult
to understand.
Millions of cycles of waveforms are even worse.
4
Formal/Semi-Formal Tools

• Formal verification can help, but..

• The verification engineer must still understand
  the design.
• And write properties

5
Inferno Making Verification Easier
Goal Interpret the activity on a simulation
trace and present it in the most intuitive
way. Inferno

• Helps the engineers understand the design
• Automatically generates properties
• Highlights new and interesting behaviors

6
Verification with Inferno
Trace from Directed Tests
Automatically generated Verilog assertions
describing the behavior of the design
Diagrams to help understand the design
initial begin clock 0 end
Valid Behavior Update Assertions
Design Under Test
Random Tests
Visual Inspection
Assertion Failed
Assertions Satisfied
Invalid Behavior Fix Design
7
Coverage with Inferno

• Coverage metrics
• The distinct transactions observed
• The number of observations at a given interface
• A possible terminating condition
• A fixed number of simulation cycles with no new
  transactions detected
• Once the terminating condition is reached
• The checkers generated by Inferno can be used in
  a formal verification context
• This can prove that no other transactions can be
  generated at the interface under analysis

8
Interpreting Activity through Transactions

• A vertex in a transaction diagram is an observed
  assignment of values to the signals under
  analysis.
• Transaction
• A defined sequence of events with a single
  purpose (e.g. a read trasaction).
• We can represent this as a graph.
• Transaction boundary
• Transactions are initially split at the first
  repeated vertex.
• Frequently, the control bits are the same during
  reset and between transactions, so is often the
  first repeated vertex.
• If any transaction is a subset of another, a new
  boundary is inferred.

9
Algorithm Trace Analysis
Example trace using the Wishbone protocol
Vertex Sequence
g,
b,
e,
d,
c,
b,
f,
a,
, g
f
b
10
Algorithm Trace Analysis
Final vertices and vertex sequence
11
Algorithm Protocol Diagrams
Take the final set of vertices. Draw each
vertex Add connections. Label with signal values.
12
Algorithm Transaction Diagrams
a
b
c
d
e
f
b
g
h
i
j
k
g
h
i
j
k
c
d
e
f
b
c
d
e
i
i
f
d
e
i
e
i
f
b
c
d
e
f
d
e
f
b
Take the final vertex sequence. Draw each
occurrence of every vertex, and connect them in
order. Look for the first repeated vertex. When
you find one, locate all occurrences. Break the
sequence after each one.
13
Algorithm Finding Loops
a
b
c
d
e
f
b
k
g
h
i
j
g
h
i
j
f
k
c
d
e
b
c
d
e
i
e
i
f
d
e
i
e
i
f
b
c
d
e
f
d
e
f
b
Look for repeated vertices. When we find one,
its a loop remove it from the main transaction.
14
Algorithm Matching Transactions
Ignore the loops. See if any transactions
match. Update number of loops. Update number of
occurrences.
Occurred 1 time
1 time
i
g
h
j
k
c
d
e
f
Occurred 1 time
b
Occurred 1 time
1 time
1 time
Occurred 2 times
0-1 times
1 time
15
Algorithm Boundary Refinement
See if any transaction occurs inside
another. Make the last vertex before that another
boundary.
Occurred 1 time
1 time
i
g
h
j
k
c
d
e
f
Occurred 1 time
b
Occurred 1 time
1 time
1 time
Occurred 2 times
0-1 times
16
Algorithm Final Transactions
Occurred 1 time
Occurred 2 times
Reset
Write
Occurred 1 time
Read-Modify-Write
1 time
1 time
b
Occurred 3 times
Burst Read
0-1 times
17
Experimental Results
Altogether, these 8 transactions occurred 93,000
times
18
Previous Work
19
Conclusion and Future Work

• We have presented Inferno a new verification
  tool
• Inferno supports verification engineers by
  automatically
• Providing intuitive diagrams to aid understanding
• Generating Verilog assertions to characterize the
  design and highlight deviations from the expected
  behavior
• Experimental results on a range of designs are
  promising
• In future work, we plan to refine our algorithm
  for detecting transactions, and extend the basic
  approach to other situations, e.g.,
  microprocessors

20
(No Transcript)
21
Algorithm Boundary Refinement
a
b
c
d
e
f
b
g
h
i
j
k
g
h
i
j
k
c
d
e
f
b
c
d
e
i
i
f
d
e
i
e
i
f
b
c
d
e
f
d
e
f
b
Return to the original sequence. Cut the
transactions at all boundary vertices.
22
Algorithm Finding Loops
With the new transactions, repeat the
loop-finding algorithm.
c
d
e
i
e
i
f
d
e
i
e
i
f
b
c
d
e
f
d
e
f
b
23
Algorithm Matching Transactions
Occurred 1 time
Then repeat the matching algorithm.
Occurred 2 times
Occurred 1 time
1 time
1 time
b
Occurred 3 times
1 time
0-1 times
24
Verification with Inferno
Trace from Directed Tests
Verilog assertions describing the behavior of the
design
Diagrams to help understand the design
initial begin clock 0 end
Valid Behavior Update Assertions
Design Under Test
Random Tests
Visual Inspection
Assertion Failed
Assertions Satisfied
Invalid Behavior Fix Design