Chapter 12: File System Implementation

1
Chapter 12 File System Implementation

• File System Structure
• File System Implementation
• Directory Implementation
• Allocation Methods
• Free-Space Management
• Efficiency and Performance
• Recovery
• Log-Structured File Systems - omit for now
• NFS - omit for now Annotated for use with
  Silberschatzs book used during CS350, Fall
  03Instructor's annotation in blue Last updated
  12/4/03

2
File-System Structure

• File structure
• Logical storage unit
• Collection of related information
• File system resides on secondary storage (disks).
• Design problem
• How should the file system look to the user
  attributes, operations, directory structure
• Creating algorithms and data structures to map
  the logical file system to physical secondary
  storage device.
• File system organized into layers.
• File control block (FCB) storage structure
  consisting of information about a file stored
  on disk, copied to memory for use.

3
Layered File System
Manages file attribute/control info Manages
directory structure symbolic file names to
logical block, Uses File Control Block (FCB)
Logical block in ? physical block out, (knows
location of file and allocation scheme used)
Generic cmds to dev drivers using disk
addresses cylinder, track, sector
Device drivers interfaces to hardware/controllers
, gets generic commands.
Hardware controllers
4
A Typical File Control Block
?Location on disk
5
Overview

• On-disk structures
• Boot control block 1st block of partition -
  used for booting if this is a bootable partition.
  May also have information on another partition
  which may be where booting may start (the Master
  Boot Record, MBR).
• Partition control block- attributes of partition,
  FCB pointers, number blocks in partition, free
  block count, free block pointers, etc.
• Directory structure
• The FCBs - (inode in UNIX)
• In-Memory structures
• In memory copy of partition table
• Cache of recently accesses directories
• System-wide open file table copy of FCBs for all
  open files more
• Per-process open file table (pointers to
  system-wide open file table) accessed via file
  descriptor returned from open call reduces
  search time

6
In-Memory File System Structures
The following example (fig. 12.3) illustrates the
necessary file system structures provided by the
operating systems See Sect. 12.2.1 for scenario
description
opening a file
reading a file
7
Scenario for file operationssee p. 415 of text

• Creating a file
• Allocate a new FCB
• Read appropriate directory into memory, and add
  the new file name and FCB to it
• Write it back to disk
• Opening a file
• Directory structure searched parts cached un
  memory
• FCB copied to System-wide-open file table
• File descriptor returned which indexes into the
  per-process-open file table.
• Reading a file
• Using file descriptor, access the per processes
  open file table
• Which in turn gets the FCB from system wide open
  file table
• FCB has information needed to access the data in
  the file

8
Directory Implementation

• Must search director for a required file
• Linear list of file names with pointer to the
  data blocks.
• simple to program
• time-consuming to execute
• Hash Table linear list with hash data
  structure.
• decreases directory search time
• collisions situations where two file names hash
  to the same location
• fixed size

9
File system allocation(new slide)

• Many files of variable size stored on the same
  disk
• Main problem is how to allocate disk blocks for
  these files so that disk space is utilized
  effectively and files can be accessed quickly.
• Three major allocations methods
• contiguous
• Linked
• indexed

10
Contiguous Allocation

• Each file occupies a set of contiguous blocks on
  the disk.
• Minimal head movement high performance(minimal
  head movement) - used by IBM VM system
• Simple only starting location (block ) and
  length (number of blocks) are required.
• Random access for file stating at block b, a
  block in that file at i blocks into the file
  would be at block bi
• Good for sequential access also
• Wasteful of space external fragmentation same
  problem as in dynamic contiguous memory
  management (ch 9) with same solutions.
• Files cannot grow if we over-allocate then
  internal fragmentation.
• Difficult to estimate how much space is needed
  for file
• Modified contiguous allocation give extents
  to make file grow just postpones the original
  problem

11
Contiguous Allocation of Disk Space
Start pointes to 1st block In file Length is
the number of blocks in file
12
Extent-Based Systems

• Many newer file systems (I.e. Veritas File
  System) use a modified contiguous allocation
  scheme.
• Extent-based file systems allocate disk blocks in
  extents.
• An extent is a contiguous block of disks. Extents
  are allocated for file allocation. A file
  consists of one or more extents.
• solves growth problem, but just postpones the
  original problem- what should be the size of the
  extent too large internal fragmentation, too
  small external fragmentation.

13
Linked Allocation

• Each file is a linked list of disk blocks blocks
  may be scattered anywhere on the disk.
• Pointer is to next block file is treated like
  a linked list
• Sequentially follow pointers to access a given
  block

14
Linked Allocation (Cont.)

• Simple need only starting address
• Free-space management system no waste of space
• No random access

File-allocation table (FAT) disk-space
allocation used by MS-DOS and OS/2. Combines
linked with indexed (see later)
Incorrect block location calculations removed
15
Linked Allocation (Cont.)

• Solves external, internal (minimal) fragmentation
  problem, and growth problem only limit is free
  space pool(as paging did in memory management,
  but no random access)
• Only sequential access direct access support
  poor (must be simulated )
• Access is slow and variable because of having to
  read all pointers in blocks before the one
  desired.
• Each block has wasted space for pointer
  minimized by clustering blocks but this may
  result in increased internal fragmentation
• Reliability a problem lost or damaged pointers

16
Linked Allocation
17
Indexed Allocation

• Brings all pointers together into the index
  block.
• One logical index block per file in general
  may have multiple linked index blocks see
  later.
• Logical view.

index table
Solves internal, external fragmentation, and
growth problem ? In addition allows direct/random
access In simplest form, size of index block
determines number of blocks in file If index
block too large, then wasted space for small
files. If index block not large enough then need
to link index blocks together see later.
18
Example of Indexed Allocation
19
Indexed Allocation (Cont.)

• Need index table (ie., block)
• Random access
• Dynamic access without external fragmentation,
  but have overhead of index block.
• Index block permanently stored on disk, but may
  be cached in memory to further improve disk
  performce. If cached, we have the old problem of
  keeping the disk updated especially if we have
  a crash.

Incorrect block location calculations removed
20
Indexed Allocation Mapping (Cont.)(new slide)

• Problem is when the file so large that there are
  not enough entries in a single index block.
• Want to map from logical to physical in a file of
  unbounded length
• Use a scheme in which the index blocks are
  themselves linked Link blocks of index table (no
  limit on size).
• The last entry of a linked block of an index
  table is a pointer to the next index table
• Now no logical limit on the size of a file
• Multilevel index block is another approach to
  this problem see below
• Can combine linked index block idea with
  multi-level UNIX I-node scheme see below

21
Indexed Allocation Mapping (Cont.) Multi-level
Index blocks - See text
?
outer-index
file
index table
22
File-Allocation Table

• Used for windows/OS/2 PC systems
• Combines linked and indexed concepts
• gt See Instructors notes
• Comments on File System Implementation For
  detailed description available on CS350
  website

23
Combined Scheme UNIX (4K bytes per block)See
text
The UNIX Inode Performance tradeoff favors small
files
24
Free-Space Management

• See instructors notes Comments on File System
  Implementation from the web site.
• Bit vector (n blocks)

0
1
2
n-1

0 ? blocki free 1 ? blocki occupied
biti
???
A bit for every block on disk offset into map
is block number Block number calculation (moving
from left to right)
(number of bits per word) (number of 0-value
words) offset of first 1 bit in 1st nonzero
word Note number of 0-value words of
leading all zero words
25
Free-Space Management (Cont.)

• Bit map requires extra space. Example
• block size 212 bytes - 4K page (common)
• disk size 230 bytes (1 gigabyte) - on the
  small side
• n 230/212 218 bits (or 32K bytes)a 1.3 GB
  with 512byte page has bit map of 332KB - stresses
  memory if cached
• But easy to get contiguous files
• Linked list (free list)
• Link together all free disk blocks. Keep pointer
  to 1st free block in a special location cache
  it
• Cannot get contiguous space easily
• No waste of space
• Grouping modification of free-list approach
  an indexed approach
• 1st block in partition (an index block) contains
  pointers to n free data blocks
• Last block in the above n free blocks is another
  index block pointing to another n free blocks
  with the last of these being another pointer
  block, etc.
• Similar the the linked index block approach for
  file allocation.
• Advantage the addresses of a large number of
  free blocks can be found quickly
• Counting
• Keep track of groups of contiguous blocks.
• Just need address of first block in group
  number of free contiguous blocks following
  similar to extents
• Shorter index list

26
Free-Space Management (Cont.)

• Need to protect
• Pointer to free list
• Bit map
• Must be kept on disk
• Copy in memory and disk may differ.
• Cannot allow for blocki to have a situation
  where biti 1 in memory and biti 0 on
  disk.
• Solution - keep memory copy disk copy in synch
• Set biti 1 in disk.
• Allocate blocki
• Set biti 1 in memory

27
Linked Free Space List on Disk
Free (unused) blocks physically Linked
together. Pointer to 1st block may be cachedin
Memory The FAT method has free space Management
built into the access scheme. The text
unjustifiably criticizes this method as
inefficient due to the need for transverse the
list. We should never have to transverse
thelist. As long as we have a pointer tothe
first block in the list, we need only to access
the first block Takethe first block if you need
a new block, or add a free block the head of
the list.
28
Efficiency and Performance

• Efficiency dependent on
• disk allocation and directory algorithms
• types of data kept in files directory entry -
  maintaining statistics in the directory - see
  p. 390?
• UNIX vary cluster size as file grows (minimizes
  internal fragmentation)
• UNIX Pre-allocates inodes on a disk to
  distribute this structure, wastes some space but
  improves performance - see below
• Performance
• disk cache separate section of main memory for
  frequently used blocks
• free-behind and read-ahead techniques to
  optimize sequential access
• improve PC performance by dedicating section of
  memory as virtual disk, or RAM disk. History was
  frequently done on PCs when all you had was a
  floppy.
• UNIX distribute inodes across partition - keep
  data blocks near files inode - minimize seeks

29
Various Disk-Caching Locations

• Note the difference between a RAM disk and disk
  cache
• Contents of RAM disk under user control - same
  as a real disk
• Contents of disk cache under control of OS
• cache disk referenced blocks on assumption that
  they will be needed again (temporal locality)
• Use I/O protocol for disk cache

Disk Controller
Track buffer page 434
30
Page Cache

• A page cache caches pages rather than disk blocks
  using virtual memory techniques.
• More efficient that caching disk blocks (disk
  cache) - higher performance
• Use memory management protocol for page cache
  uses memory calls or instructions.
• Memory-mapped I/O uses a page cache.
• Routine I/O (I/O reads and writes) through the
  file system uses the buffer (disk) cache.
• Problem what if you want to do memory mapped
  paged caching if the underlying architecture
  uses ordinary I/O?
• This leads to the following figures.

31
I/O Without a Unified Buffer Cache
Page caching mapped onto buffer caching
Because the virtual Memory cannot Interface with
the Buffer cache, the Data in buffer cache Is
copied to page Cache double Caching -
inefficient
Virtual Memory Interface use memory references
Fig. 12.11 p. 435
pages
Disk blocks
File I/O interface Use read/write calls
32
I/O Using a Unified Buffer Cache
A unified buffer cache uses the same page cache
to cache both memory-mapped pages and ordinary
file system I/O
Only a single Buffer needed More efficient.
page cache?
33
Recovery

• lazy disk update may be used - can leave the
  disk in an inconsistent state if a crash ocurrs
  before a disk update is made.
• Consistency checking compares data in directory
  structure with data blocks on disk, and tries to
  fix inconsistencies - a scandisk function?
• Use system programs to back up data from disk to
  another storage device (floppy disk, magnetic
  tape).
• Recover lost file or disk by restoring data from
  backup
• Ex. The Master Boot Record is backed up - better
  to resore an obsolete version than none at all.

34
Log Structured File Systems ltltomitgtgt

• Log structured (or journaling) file systems
  record each update to the file system as a
  transaction.
• All transactions are written to a log. A
  transaction is considered committed once it is
  written to the log. However, the file system may
  not yet be updated.
• The transactions in the log are asynchronously
  written to the file system. When the file system
  is modified, the transaction is removed from the
  log.
• If the file system crashes, all remaining
  transactions in the log must still be performed.

35
Network File System - NFS Overview

• NFS views a set of interconnected machines as
  having independent file system.
• Goal allow sharing of file systems omong
  machines on the network in a transparent manner.
  Sharing is not mandatory it is upon request of
  a machine.
• Sharing is based on a client/server relationship.
• Sharing is allowed between any pair of machines
  raher than only with a dedicated server.
• The file mounting protocol is used to make a
  remote directory accessible in a transparent
  manner.

36
The Sun Network File System (NFS)Omit NFS slides
for now

• An implementation and a specification of a
  software system for accessing remote files across
  LANs (or WANs).
• The implementation is part of the Solaris and
  SunOS operating systems running on Sun
  workstations using an unreliable datagram
  protocol (UDP/IP protocol and Ethernet.

37
NFS (Cont.) Omit NFS slides for now

• Interconnected workstations viewed as a set of
  independent machines with independent file
  systems, which allows sharing among these file
  systems in a transparent manner.
• A remote directory is mounted over a local file
  system directory. The mounted directory looks
  like an integral subtree of the local file
  system, replacing the subtree descending from the
  local directory.
• Specification of the remote directory for the
  mount operation is nontransparent the host name
  of the remote directory has to be provided.
  Files in the remote directory can then be
  accessed in a transparent manner.
• Subject to access-rights accreditation,
  potentially any file system (or directory within
  a file system), can be mounted remotely on top of
  any local directory.

38
NFS (Cont.) Omit NFS slides for now

• NFS is designed to operate in a heterogeneous
  environment of different machines, operating
  systems, and network architectures the NFS
  specifications independent of these media.
• This independence is achieved through the use of
  RPC primitives built on top of an External Data
  Representation (XDR) protocol used between two
  implementation-independent interfaces.
• The NFS specification distinguishes between the
  services provided by a mount mechanism and the
  actual remote-file-access services.

39
Three Independent File Systems Omit NFS slides
for now
40
Mounting in NFS Omit NFS slides for now
Mounts
Cascading mounts
41
NFS Mount Protocol Omit NFS slides for now

• Establishes initial logical connection between
  server and client.
• Mount operation includes name of remote directory
  to be mounted and name of server machine storing
  it.
• Mount request is mapped to corresponding RPC and
  forwarded to mount server running on server
  machine.
• Export list specifies local file systems that
  server exports for mounting, along with names of
  machines that are permitted to mount them.
• Following a mount request that conforms to its
  export list, the server returns a file handlea
  key for further accesses.
• File handle a file-system identifier, and an
  inode number to identify the mounted directory
  within the exported file system.
• The mount operation changes only the users view
  and does not affect the server side.

42
NFS Protocol Omit NFS slides for now

• Provides a set of remote procedure calls for
  remote file operations. The procedures support
  the following operations
• searching for a file within a directory
• reading a set of directory entries
• manipulating links and directories
• accessing file attributes
• reading and writing files
• NFS servers are stateless each request has to
  provide a full set of arguments.
• Modified data must be committed to the servers
  disk before results are returned to the client
  (lose advantages of caching).
• The NFS protocol does not provide
  concurrency-control mechanisms.

43
Three Major Layers of NFS Architecture Omit NFS
slides for now

• UNIX file-system interface (based on the open,
  read, write, and close calls, and file
  descriptors).
• Virtual File System (VFS) layer distinguishes
  local files from remote ones, and local files are
  further distinguished according to their
  file-system types.
• The VFS activates file-system-specific operations
  to handle local requests according to their
  file-system types.
• Calls the NFS protocol procedures for remote
  requests.
• NFS service layer bottom layer of the
  architecture implements the NFS protocol.

44
Schematic View of NFS Architecture Omit NFS
slides for now
45
NFS Path-Name Translation Omit NFS slides for now

• Performed by breaking the path into component
  names and performing a separate NFS lookup call
  for every pair of component name and directory
  vnode.
• To make lookup faster, a directory name lookup
  cache on the clients side holds the vnodes for
  remote directory names.

46
NFS Remote Operations Omit NFS slides for now

• Nearly one-to-one correspondence between regular
  UNIX system calls and the NFS protocol RPCs
  (except opening and closing files).
• NFS adheres to the remote-service paradigm, but
  employs buffering and caching techniques for the
  sake of performance.
• File-blocks cache when a file is opened, the
  kernel checks with the remote server whether to
  fetch or revalidate the cached attributes.
  Cached file blocks are used only if the
  corresponding cached attributes are up to date.
• File-attribute cache the attribute cache is
  updated whenever new attributes arrive from the
  server.
• Clients do not free delayed-write blocks until
  the server confirms that the data have been
  written to disk.