Caching in the Sprite Network File System

1
Caching in the Sprite Network File System

• Ref Nelson, Welch, Ousterhout paper

2
Overview

• Caching in distributed file systems can be done
  via
• Memory in file server
• Memory at client machine
• Local disk at client machine
• Sprite Cache in memory at both client and server
• Disk caching benefits
• Faster access (save disk access and/or network
  latency)
• Reduces contention for disk
• Two main ideas in Sprite
• Cache consistency (no stale data)
• Caches vary in size dynamically
• provide as large a cache as possible, consistent
  with needs of virtual memory system
• file and virtual memory system negotiate for
  physical memory

3
Cache Design Issues

• Where should the client cache be located memory
  or local disk? Sprite rationale (for using
  memory)
• Allows diskless workstations
• Faster than disk
• Large physical memory cheap (this was 20 years
  ago!)
• Works well with memory-based server cache
• How should caches be structured and addressed?
• Sprite based on disk blocks to simplify design
• Virtual addresses used (server-provided id
  block number within file) reduces communication
  w/ server since physical address not needed to
  allocate new blocks
• Clients cache only data blocks server caches
  file maps and other disk management information
• What policy should be used in writing blocks back
  to disk?
• Write through - poor write performance, but
  minimizes data loss if client/server crashes
• NFS write on close policy

4
Motivation File Usage Study

• Sprite design motivated by file usage study (note
  times are circa 1985)
• One third of all file accesses are writes
• a cache using write through only helps on at most
  2/3 accesses
• Data written to file often deleted soon (writing
  to disk is wasted effort in this case)
• 20-30 of new data deleted within 30 seconds
• 50 deleted within 5 minutes
• Files not open very long
• 75 of files open
• 90 of files open
• Suggests write on close will not improve
  performance much
• Above two points suggest files often closed, then
  data deleted soon after that (so no need to write
  data back to disk!) - argues for delaying write
  back until well after the file is closed
• File sharing rare

5
Sprite Write Policy

• Sprite Write Policy Every 30 seconds, scan dirty
  blocks those not modified in last 30 seconds
  written back (first to server memory, then to
  disk)
• Block written to cache will update servers cache
  in 30-60 seconds, will update disk in 60-120
  seconds
• Note this may be too infrequent if files are
  used for communication / synchronization

6
Cache Consistency

• Sequential write-sharing consistency
• Two clients not allowed to simultaneously open a
  file for writing
• Each client sees most up-to-date version in this
  case
• Typically implemented in write-on-close systems,
  e.g., NFS (NFS checks if a newer version exists
  when opening a file in its cache)
• Concurrent write-sharing
• When a client reads a file, it gets the most
  recently written data, even if there are one or
  more concurrent writers
• Sprite implements both
• Note application must implement synchronization
  (e.g., locks) to make sure interleaving
  reads/writes makes sense

7
Concurrent Write-Sharing

• A file is concurrent write-shared if
• Open by multiple clients
• One or more clients has it open for writing
• Main idea
• Disable client caching for concurrent write
  shared files
• Write operations go through to the server
  serialize operations there

8
Concurrent Write Sharing Details

• Server detects a file about to become concurrent
  write shared
• A new open operation on file occurs, and
• the file is already open by another client, and
• and there is at least one writer
• Notify client that has already opened the file
  for writing (if there is one there can be at
  most one) that it needs to flush dirty blocks to
  the server
• Notify all clients that have the file open that
  it is no longer cacheable
• All future references to file go to server
• Server serializes all accesses to the file -
  yields semantics the same as if on a single
  machine

9
Concurrent Write Sharing

• A client may cache files it does not have open
  consistency checks required when the client opens
  the file
• Files open by a single client can be cached (even
  if client is writing to file)
• Files open by multiple readers can be cached
• Clear this protocol is not very efficient, but
  concurrent write sharing should be rare
• Not so if file used for communication/synchronizat
  ion

10
Sequential Write-Sharing

• This occurs when a file opened by one client,
  modified, closed, then opened by another client
• Problem 1 Second client may have stale blocks in
  its client cache
• Solution Check with server if file has since
  been modified (same as NFS)
• Problem 2 recall server updated 30-60 seconds
  after last write new client could get stale data
  (current data in another clients cache)
• Solution Server remembers last writer, i.e.,
  client that (1) cached the file and (2) was a
  writer (at most one such client) notify that
  client to flush any dirty blocks before opening
  file for a new client

11
Virtual Memory and the File System

• RAM used both as file cache and to hold pages of
  virtual memory pages applications
• Would like to dynamically adjust file system
  cache to accommodate changing application memory
  requirements
• Sprite
• Virtual memory uses approximate LRU replacement
  (physical memory caches VM pages)
• File system cache uses exact LRU replacement
• Each tracks oldest page in its cache
• Idea when loading a new block/page (file cache
  miss or page fault), replace the oldest page
  this could come from the others cache, in which
  case, the memory shifts from VM to file cache (or
  vice versa)

12
VM and File Cache Interactions

• Issue 1 Double-caching
• Backing store implemented as ordinary files (swap
  files)
• When page loaded into memory, copies reside in
  both file cache and virtual memory page pool
  inefficient
• Solution swap files bypass file cache
• Issue 2 Multi-block pages
• VM page may hold several file blocks
• How is age of page determined (which block)?
• Sprite 8K byte pages, 4K byte blocks, age is
  youngest block in page

13
Summary

• Measurement-driven design methodology -
  delay-write policy
• Write Policy
• Cache consistency mechanism (concurrent write
  sharing)
• Dynamic memory allocation between VM and file
  cache