Efficient Representations and Abstractions for Quantifying and Exploiting Data Reference Locality

1
Efficient Representations and Abstractions for
Quantifying and Exploiting Data Reference Locality

• Trishul Chilimbi
• Performance Monitoring and Analysis Group
• Microsoft Research
• http//research.microsoft.com/trishulc/Daedalus.h
  tm

2
The Big Ideas

• Collect trace of dynamic data references
• Kind of Ginormous
• Perform compression using SEQUITUR
• Produces analyzable compressed form
• Extract locality information
• Hot streams
• Stream flow graph
• Perform Optimizations
• Prefetching
• Clustering

3
Processor-Memory Imbalance
µProc 60/yr.
10000
1000
Processor-Memory Performance Gap53/yr.
Performance
100
DRAM 7/yr.
10
1
Year
Graph from Dave Patterson
4
Performance Impact
5
Thesis 90/10 Rule Applies to Data
6
Silently Falling off a Performance Cliff
Cache size
Memory Access Time
7
Silently Falling off a Performance Cliff
Cache size
Memory Access Time
8
Software Assistance
Working set size
9
Improving Memory Performance

• Making caches more efficient
• Data layout transformations (e.g., clustering)
• Beyond caches
• Prefetching

prefetch a
Memory
Cache
load a
10
Barriers to Memory Optimization

• Detailed data reference sequence information
• Program path analogue?
• Compact data reference representation
• Control flow graph analogue?
• Consequences
• Poor quantitative understanding
• Ad-hoc, extensive use of heuristics
• Imprecise frequency profiles
• Inadequate for cache-level optimizations

11
Overview

• Background and motivation
• Detailed data reference sequence information
• Compact data reference representation
• Improving memory system performance

8 23 improvements in execution time
12
Overview

• Background and motivation
• Detailed data reference sequence information
• Exploitable locality abstraction
• Efficient computation
• Quantitative analysis
• Compact data reference representation
• Improving memory system performance

13
Exploitable Locality
Exploitable Locality Reference Skew Regularity
Reference Skew
a b c a c b d b a e c f b b b c g a a f a d c c
14
Exploitable Locality Abstraction

• Hot Data Streams
  Data object sequences that frequently
  repeat
• Hotgtreference skew Repititiongtregularity
• Analogous to hot program paths

ABAACEFABADCEFABAACEFDDCEFBABA
ABA
CEF
Hot Data Streams
15
SEQUITUR (Nevill-Manning Witten)
aaabac aaabac aaabac aaabac aaabad aaabad aaabad
aaabad aaabad aa
SEQUITUR
S -gt BBDDCaa A -gt aaabac B -gt AA C -gt aaabad D -gt
CC
S
B
D
A
DAG representation of grammar
C
SEQUITUR Grammar
a
b
c
d
16
Computing Exploitable Locality
Address Abstraction
Increases repetitions
Abstracted Trace
Data Reference Trace
SEQUITUR
S
regularity detector
A
C
B
a
b
c
d
Whole Program Streams (WPS)
Hot Data Streams
d
a
b
a
b
a
b
17
Quantitative Analysis
A
B
C
D
C
B
D
A
Hot data stream size? (1025 elements) Hot data
stream repetition interval? (505000 references)
18
Program Exploitable Locality
eon
vortex
parser
sqlserver
perlbmk
twolf
boxsim
gcc
19
Overview

• Background and motivation
• Detailed data reference sequence information
• Compact data reference representation
• Eliminating noise
• Inter-stream analysis
• Improving memory system performance

20
Eliminating Noise
a b c i k l f a b c j k l f a b c m k l f
a b c i k l f a b c j k l f a b c m k l f
21
Eliminating Noise
a b c k l f a b c k l f a b c k l f
S
Hot data stream analyses
SEQUITUR
A
C
WPS1
B
a
b
c
..
A
a
l
b
c
k
f
Hot data streams1
22
Inter-stream Analysis
S
Whole program streams (WPS)
d
A
b
a
A
B
a
b
B
Hot Data Streams
a
b
70
A
Graph summarization
75
5
C
70
B
Stream Flow Graph (SFG)
23
Data Reference Representation Size
24
Overview

• Background and motivation
• Detailed data reference sequence information
• Compact data reference representation
• Improving memory system performance
• Opportunity
• Tools

25
What Is the Opportunity?

• Making caches more efficient

2 cache blocks
Ideal hot data stream layout
a
f
g
d
c
h
cache block packing efficiency ( cache
blocks with ideal layout)/( cache blocks used)
0.5
26
(No Transcript)
27
DrillData Reference Locality Locator

• Associates hot data streams with code data
  structures responsible
• Displays hot data streams properties
• Heat
• Spatial regularity (size)
• Temporal regularity
• Cache block packing efficiency

28
(No Transcript)
29
Structure Layout Transformations
823 improvements in execution time
30
Conclusions

• Hot data streams useful locality abstraction
• Capture sufficient detail for cache optimizations
• Computed efficiently (seconds to minutes)
• Small number account for 90 of data references
• Stream flow graph compact representation
• 10 GB Trace gt 100 KB SFG
• Analyze hot data stream interactions
• Stream-based optimizations appear promising
• 823 execution time improvements

31
Current and Future Work

• Hot data stream profile stability (PACT 2001)
• Custom memory allocators
• Automatic prefetching
• Low-overhead on-line optimization framework
• GC-based data layout transformations
• Scalability optimizations

http//research.microsoft.com/trishulc/Daedalus.h
tm