Title: Efficient Representations and Abstractions for Quantifying and Exploiting Data Reference Locality
1Efficient Representations and Abstractions for
Quantifying and Exploiting Data Reference Locality
- Trishul Chilimbi
- Performance Monitoring and Analysis Group
- Microsoft Research
- http//research.microsoft.com/trishulc/Daedalus.h
tm
2The Big Ideas
- Collect trace of dynamic data references
- Kind of Ginormous
- Perform compression using SEQUITUR
- Produces analyzable compressed form
- Extract locality information
- Hot streams
- Stream flow graph
- Perform Optimizations
- Prefetching
- Clustering
3Processor-Memory Imbalance
µProc 60/yr.
10000
1000
Processor-Memory Performance Gap53/yr.
Performance
100
DRAM 7/yr.
10
1
Year
Graph from Dave Patterson
4Performance Impact
5Thesis 90/10 Rule Applies to Data
6Silently Falling off a Performance Cliff
Cache size
Memory Access Time
7Silently Falling off a Performance Cliff
Cache size
Memory Access Time
8Software Assistance
Working set size
9Improving Memory Performance
- Making caches more efficient
- Data layout transformations (e.g., clustering)
- Beyond caches
- Prefetching
prefetch a
Memory
Cache
load a
10Barriers to Memory Optimization
- Detailed data reference sequence information
- Program path analogue?
- Compact data reference representation
- Control flow graph analogue?
- Consequences
- Poor quantitative understanding
- Ad-hoc, extensive use of heuristics
- Imprecise frequency profiles
- Inadequate for cache-level optimizations
11Overview
- Background and motivation
- Detailed data reference sequence information
- Compact data reference representation
- Improving memory system performance
8 23 improvements in execution time
12Overview
- Background and motivation
- Detailed data reference sequence information
- Exploitable locality abstraction
- Efficient computation
- Quantitative analysis
- Compact data reference representation
- Improving memory system performance
13Exploitable Locality
Exploitable Locality Reference Skew Regularity
Reference Skew
a b c a c b d b a e c f b b b c g a a f a d c c
14Exploitable Locality Abstraction
- Hot Data Streams
Data object sequences that frequently
repeat - Hotgtreference skew Repititiongtregularity
- Analogous to hot program paths
ABAACEFABADCEFABAACEFDDCEFBABA
ABA
CEF
Hot Data Streams
15SEQUITUR (Nevill-Manning Witten)
aaabac aaabac aaabac aaabac aaabad aaabad aaabad
aaabad aaabad aa
SEQUITUR
S -gt BBDDCaa A -gt aaabac B -gt AA C -gt aaabad D -gt
CC
S
B
D
A
DAG representation of grammar
C
SEQUITUR Grammar
a
b
c
d
16Computing Exploitable Locality
Address Abstraction
Increases repetitions
Abstracted Trace
Data Reference Trace
SEQUITUR
S
regularity detector
A
C
B
a
b
c
d
Whole Program Streams (WPS)
Hot Data Streams
d
a
b
a
b
a
b
17Quantitative Analysis
A
B
C
D
C
B
D
A
Hot data stream size? (1025 elements) Hot data
stream repetition interval? (505000 references)
18Program Exploitable Locality
eon
vortex
parser
sqlserver
perlbmk
twolf
boxsim
gcc
19Overview
- Background and motivation
- Detailed data reference sequence information
- Compact data reference representation
- Eliminating noise
- Inter-stream analysis
- Improving memory system performance
20Eliminating Noise
a b c i k l f a b c j k l f a b c m k l f
a b c i k l f a b c j k l f a b c m k l f
21Eliminating Noise
a b c k l f a b c k l f a b c k l f
S
Hot data stream analyses
SEQUITUR
A
C
WPS1
B
a
b
c
..
A
a
l
b
c
k
f
Hot data streams1
22Inter-stream Analysis
S
Whole program streams (WPS)
d
A
b
a
A
B
a
b
B
Hot Data Streams
a
b
70
A
Graph summarization
75
5
C
70
B
Stream Flow Graph (SFG)
23Data Reference Representation Size
24Overview
- Background and motivation
- Detailed data reference sequence information
- Compact data reference representation
- Improving memory system performance
- Opportunity
- Tools
25What Is the Opportunity?
- Making caches more efficient
2 cache blocks
Ideal hot data stream layout
a
f
g
d
c
h
cache block packing efficiency ( cache
blocks with ideal layout)/( cache blocks used)
0.5
26(No Transcript)
27DrillData Reference Locality Locator
- Associates hot data streams with code data
structures responsible - Displays hot data streams properties
- Heat
- Spatial regularity (size)
- Temporal regularity
- Cache block packing efficiency
28(No Transcript)
29Structure Layout Transformations
823 improvements in execution time
30Conclusions
- Hot data streams useful locality abstraction
- Capture sufficient detail for cache optimizations
- Computed efficiently (seconds to minutes)
- Small number account for 90 of data references
- Stream flow graph compact representation
- 10 GB Trace gt 100 KB SFG
- Analyze hot data stream interactions
- Stream-based optimizations appear promising
- 823 execution time improvements
31Current and Future Work
- Hot data stream profile stability (PACT 2001)
- Custom memory allocators
- Automatic prefetching
- Low-overhead on-line optimization framework
- GC-based data layout transformations
- Scalability optimizations
http//research.microsoft.com/trishulc/Daedalus.h
tm