Using Historic Patterns in Both Cache Prefetching and Cache Replacement

1
Using Historic Patterns in BothCache Prefetching
and Cache Replacement

• Jim Gast Laura Spencer
• CS703, Spring 2000

2
Motivation

• Disk I/O is slow and not getting faster
• 10 milliseconds for an unexpected open
• Memory is cheap and getting cheaper
• Cache more, read ahead, stay ahead
• LRU is a lousy algorithm
• Program modification is hard
• History is (comparatively) easy

3
New metadata per file

• Uses (actual opens from actual programs)
• Cache-behind successes
• Prefetch successes, Prefetch tries
• Prefetch failures

4
Store history in triads

• Historically, file E is typically followed by
• file F after interval IE-gtF with Bias BE-gtF
• And E, F is often followed by
• file G after interval IEF-gtG with Bias BEF-gtG
• Bias is the conditional probability
• P(GE) P(FE)P(GE,F)
• BE-gtF BEF-gtG

5
Simplifying Assumptions

• Every file is just one page
• Unique file name implies unique disk address
• Ignore writes
• Any file open takes 10 milliseconds
• No disk contention

6
Prediction Algorithms

• NONE
• Kroger Long
• 3rd order Trie
• We added expected request time
• Ours
• 5th order Trie simulated using recursion on 3rd
• Including expected request time

7
Typical Trie - Frequencies
FileList.o
31
9
22
Welem.o
Path.o
9
22
Welem.o
VolID.o
When FileList.o is opened, the bias to prefetch
VolID.o is 22 / 31
8
Typical Trie - Intervals
FileList.o
Welem.o
Path.o
0.012
0.011
Welem.o
VolID.o
0.007
0.009
Predict that VolID.o will be needed in 0.021
seconds
9
Using 3rd order Trie as 5th order
100
30
60
a.o
b.o
c.o
30
15
20
b.o
c.o
d.o
20
20
15
c.o
d.o
e.o
If a.o is opened, the bias to predict c.o is 50,
e.o is 33
10
Bias to predict a future open
11
Replacement Algorithms

• LRU
• One long queue
• Re-queue on fresh open
• ENU
• Speculated Queue for files not yet actually
  opened
• Popular Queue for files worth keeping
• Idle Queue for files not yet evicted

12
Compare 3 items to find victim
Each time a file is hinted or opened, it is
re-queued
Incoming
Speculated
Popular
Idle
New
Most
Choose Most Willing
Old
Least
Overdue
of the three tails
13
Unexpected Open
OnDisk
Open (Miss)
Evicted
Incoming
Popular or
Arrived
Idle
Open (NearMiss)
Open (Hit)
14
Hinted Open
OnDisk
Open (Prefetch NearMiss)
Hint (Prefetch Try)
Evicted
Incoming
Popular or
Evicted (Prefetch Failure)
Idle
Speculated
Arrived
Open (Prefetch Hit)
15
Trace-driven Simulation

• Seer traces from UCLA
• Nine LINUX servers
• Software development environment
• Every file open for several weeks
• We only used one week from one server

16
Simulation

• 21Cache sizes
• 10 pages much smaller than workingset
• 1000 pages much larger than workingset
• Various levels of aggression
• 90 bias (conservative)
• 10 bias (aggressive)
• Compared LRU, KL, Pop, ENU

17
Measurements

• Total Time Waiting
• And Time Fully Busy
• Hits prefetch Hits
• Near Misses prefetch Near Misses
• Misses
• Prefetch Failures
• Memory Pressure

18
Total Time Waiting
Constant Bias Threshold 90
19
All Opens - LRU
20
All Opens using KL Algorithm
21
All Opens by Algorithm
22
All Opens ENU
23
Prefetch Mistakes by Bias Threshold
24
Memory Pressure
25

CurrentTime
Speculated
Willingness
Idle
Popular
Idle ENU
Popular ENU
Time
26
Light Memory Pressure
CurrentTime
Idle
Popular
Willingness
Speculated
Time
27
Prefetch Failure
Willingness for Speculated Files
CurrentTime
Speculated
Willingness
Idle
Popular
Time
28
Idle Queue close to empty, w/hint
CurrentTime
Speculated
Willingness
Popular
Idle, ENU soon
Time
29
Future Work

• Block Level
• Intelligent disks
• Model dirty pages
• Better Simulation Traces
• Interleaved requests, limited disks
• Real Kernel Mods
• Study long strings without trace times anomoly
• Refinements to measurements
• Willingness, Popularity