Dynamic Verification of Cache Coherence Protocols

1
Dynamic Verification of Cache Coherence Protocols

• Jason F. Cantin
• Mikko H. Lipasti
• James E. Smith

2
Introduction

• Multiprocessors are used for a variety commercial
  and mission-critical tasks
• Reliability is a growing concern
• Coherence is a fundamental feature of
  shared-memory MPs
• High design complexity
• Relatively low interconnect reliability

3
IntroductionCache Coherence Protocols

• Notoriously difficult to design and verify
• Often conceptually simple, but with complex
  implementations for efficiency and handling
  special cases
• Multiple finite state machines operating
  concurrently

4
IntroductionA Simple Example

• MSI Protocol Architected State
• Invalid / Not Present
• Shared (readable)
• Modified (read/write)

Bus_RdX, Replace
I
S
Read
Bus_Rd
Write
Bus_RdX, Replace
Write
M
5
IntroductionSimple Example with a Bus

• MSI Protocol Implementation State
• Transient states for pending operations
• Arcs to satisfy requests while operations pending

S
I
Bus_RdX, Replace
Read
Bus_Av
Bus_Av
Pend Rd I
Write
Pend WB M
Bus_RdX
Write
Pend RdX I
Bus_Av
Bus_RdX
Replace
M
Pend RdX S
Bus_Av
Bus_Rd
6
Problem

• In practice, implementations can have dozens of
  states
• Atomic memory operations
• Split transaction buses
• Protocol optimizations
• Complexity grows exponentially with added states
• Random testing Low Coverage
• Exhaustive testing Too time consuming

7
Dynamic Verification

• Check the implementation at runtime
• It is easier to check a computation than to do
  the actual computation, provided there is a delay
  between the computation and the check (Rotenberg,
  AR-SMT)
• Simplified version of a processor implementation
  can be used for online verification (Austin,
  DIVA)

8
Dynamic Verification of Cache Coherence

• A distributed form of dynamic verification for
  multiprocessor memory systems
• Simplified version of protocol added to each node
• Maintains architected state
• Check completed transitions and actions against
  simple protocol
• Additional messages (assertions) sent between
  nodes to ensure coherence

9
Conceptual View for Superscalar Processors (DIVA)

• Single, centralized check processor
• Receives instructions serially in program order
  from implementation

Physical registers
Arch. registers
Complex Execution Processor
Prediction Tables
R.O.B.
Committed results
Arch. registers
Check Processor
10
Conceptual View for Coherence
Shared Logical Bus

• Distributed checking hardware
• Transitions received in parallel, in completion
  order

Implementation Protocol
Completed Transitions
Simple Protocol
Shared Validation Bus
11
High Level Organization
Memory
Cache
P
DV-CC Checker
Controller
Validation bus (assertions to be checked)
Shared logical bus (addresses, data, control)
12
Benefits

• Detects hardware faults
• Redundant computation
• Including intermittent network failures
• Detects design mistakes
• Checker is simple and easy to verify

13
Drawbacks

• Time is required for checking, but
• May be overlapped with other activities
• Simple protocol requires fewer transitions
• Assertions consume bandwidth
• May need second bus / network
• Additional hardware
• But not much

14
DV for coherence in an SMP

• Architected state stored in a second tag array
• Transactions sent to the checker when architected
  state changes
• Address
• Initial State and Final States
• Input (Request, Snoop Responses, etc)
• Action (Send Data, Respond Shared, etc)

15
DV for coherence in an SMP (2)

• Checker compares the initial state of a
  transition against the architected state
• Final state and action recomputed and compared to
  implementations result
• Assertions broadcast to other nodes to check
  coherence and confirm completion of transactions
• Watchdog timer detects deadlock, livelock, and
  other omission failures

16
Checking a State Transition
Transition From Implementation Protocol
Init. state
Final state
Input
Action
Address
Next State Logic
Arch. Tag State
Action Logic
?
?
?
Update Tags
Error Detection / Diagnosis
Asst Send Buffer
Watchdog timer
Validation Bus
17
Checking an Assertion
Address
Arch. Tag State
Remote State
OK
Error Detection / Diagnosis
Watchdog timer
Assert Recv Buffer
Validation Bus
18
When to Broadcast Assertions

• For MSI
• 1. I ?S (readable copy loaded)
• 2. I ?M (writeable copy loaded)
• 3. S ? M (upgrade)
• 4. M ? I (writeback)
• Note The M?S transition results from remote
  reads, and doesnt require an extra assertion.
  Replacements (S?I) are not considered here.

19
Preliminary Data (4-way SMP)

• Most memory references do not change cache state
  (checker need not have high bandwidth)

20
Preliminary Data (4-way SMP)
21
Future Work

• Performance impact for a real SMP protocol
  implementation
• In progress
• Directory-based protocols
• Dynamically verifying memory models
• Recovery
• Can stall to avoid error propagation
• Can write checkpoints periodically

22
In Summary

• Dynamic verification can be applied to
  multiprocessor systems (in a distributed manner)
• Improves fault-tolerance, and design verification
  may be relaxed
• More to come