Token Coherence for CMPs

1
Token Coherence for CMPs

• Mike Marty
• CS 838
• 12/09/2003

2
Outline

• Motivation for Token Coherence
• Token Coherence 101
• Issues with CMPs
• Maintaining correctness
• When to retry?
• Persistent requests

3
Motivation 1 Performance Tradeoffs

• Snooping
• Low latency (w/o network contention)
• Bandwidth doesnt scale
• Requires ordered interconnect
• Directory
• No broadcast scalability
• Enables unordered interconnects
• Indirections increase latency
• Can we get best of both worlds?

4
Motivation 2 Complexity

• Even basic coherence protocols are hard to get
  right
• Future sources of additional complexity
• Prediction
• e.g. Destination-Set Prediction
• Hierarchy
• CMPs
• DNUCA

5
Complexity Example
Dir/Mem
2. GETX
2. GETX
3. Fwd
3. Inv
S
L2
L2
1. GETX
4. Inv
2. WB
1. GETX
5. DATA
4. Fwd
4. Inv
O
S
S
S
6
Token Coherence

• Global Invariant
• For each block, allow either one writer or
  multiple readers
• Enforce Locally
• Each (logical) has T tokens (initially at memory)
• Tokens never created nor destroyed
• Components exchange tokens data
• All tokens to write ltgt one writer invariant
• At least one token to read ltgt multiple readers
• Invariants enforced explicitly, rather than
  grab-bag of invalidations, acks, nacks

7
Starvation Avoidance

• Purpose handle pathological cases
• Detect possible starvation
• E.g. gt 4 retries issued
• Invoke Persistent Request
• Starving processor issues
• Fair arbitration scheme activates request
• Request persists at each processor until
  satisfied
• Deactivate upon completion
• Persistent Requests should be rare

8
Token Coherence Generalizes

• Correctness Substrate
• Safety with token counting
• Starvation avoidance with persistent requests
• Performance Protocol
• Make the common case fast
• Alternatives
• TokenB broadcast
• TokenM multicast with prediction
• TokenD soft-state directory for a large system

9
Token Coherence for CMPs

• Ensuring safety with tokens
• Tokens to individual caches
• Exploiting CMP locality
• Extra tokens to chip on initial requests
• Scaling Persistent Requests

10
TokenCMP
logically
11
(No Transcript)
12
TokenCMP Reissues

• Transient requests can fail
• Coherence race
• Network contention
• When to reissue?
• Reissue transient or persistent request?

13
TokenCMP Persistent Requests

• Recall
• Anything that holds/sends/receives tokens must
  remember all outstanding persistent requests
• Scale persistent requests via hierarchy
• One outstanding persistent request per chip
• Achieve performance via locality
• Satisfy all local persistent requests first

14
Conclusion

• CMPs move the complexity from the processors to
  the memory system
• Token Coherence reduces complexity by separating
  performance from correctness
• CMP performance largely depends on coherence
• Token Coherence gives designers more performance
  tradeoffs

15
Token Coherence Example
16
T1(R)
T15(R)
1
2
4
T1
3

• P2 responds with data to P1

17
T1(R)
T15(R)
1
2
4
3

• P0s delayed request arrives at P2

18
6
T15
T0
T1(R)
T15(R)
1
T16 (R/W)
T15(R)
5
P2
P0
2
7
4
3

• P2 responds to P0

19
6
T0
T1(R)
T15(R)
1
T16 (R/W)
T15(R)
5
P2
P0
2
7
4
3
20
Now what? (P0 wants all tokens)
21
Timeout!
22

• P0s request completed