Operating Systems Lecture 11

1
Operating SystemsLecture 11

• File System Management

phones off (please)
2
Overview

• Disk space management
• block allocation
• keeping track of free blocks
• File system reliability
• file system consistency
• fragmentation and defragmentation
• File system performance
• disk caching concepts
• logical block v. virtual block caching
• Windows NT cache manager

3
Disk Space Management
4
Block Allocation

• We have seen how the hard disk is usually split
  up into blocks of a fixed size
• we have seen some methods for block allocation
• contiguous
• linked-list / FAT
• indexed
• we have seen how allocation is implemented at a
  physical level in MS-DOS, Win UNIX
• Two questions we havent yet addressed
• how do we determine the best block size?
• how do we maintain the list of free blocks?

5
Block Size

• In some cases, e.g. MS-DOS / Win 16-bit FAT, the
  block size is determined by the size of disk
• the maximum number of blocks is fixed (65536), so
  large disks require large block sizes
• Or we can make a more considered choice
• larger blocks give greater speed
  (pseudo-contiguous)
• small blocks will require many disk seeks as well
  as reads
• smaller blocks cause less wasted space
• studies have shown that UNIX median file size is
  around 1K
• a 32K block wastes 31K space for a 1K file
• Typical block sizes are around 1K to 4K
• linux (ext2fs file system) 1K default (can be 2K
  or 4K)
• WinNT 512 up to 512Mb, 1K up to 1Gb, 2K up to
  2Gb, or 4K

6
Free Blocks

• Two methods of keeping track of free blocks are
  commonly used
• these are very similar to the methods of keeping
  track of free main memory discussed in lecture
  4.1
• linked-list of disk blocks
• a linked-list is kept on the disk of blocks that
  are used to list block numbers of free blocks
• the last entry is a link to the next free-list
  block
• e.g. a 1K block holds 255 32-bit free-block
  numbers 1 link pointer
• bit map
• each disk block is represented by a single bit in
  a bit map
• this is the same as a bit-map used for memory
  allocation

7
Free Block Systems

• MS-DOS / Win uses the FATto store the free
  blocks list
• 0 cannot be a valid disk block
• so it is used to signify free

• UNIX uses the linked-list method to store free
  blocks
• the start of the linked-list is heldin a fixed
  position on the hard diskin a block called the
  superblock
• usually block 1 on the hard disk(block 0 is the
  boot block)

8
File System Reliability
9
Bad Blocks

• Hard disks often have blocks that are faulty
• sometimes on manufacture, sometimes after use
• There are two solutions to such bad blocks
• The hardware solution
• a sector on the disk contains a bad block list
  which is initialised on a low-level format
• the controller maps a good block to replace bad
  one
• The software solution
• the operating system creates a list of bad blocks
  when creating the file system structures
  (high-level format)
• bad blocks may be recorded in a similar way to
  free blocks, or
• may be assigned to a special file so that they
  are not free

10
Backups

• However reliable the file system, a hard disk
  fault can cause a loss of data (at worse, all
  data!)
• in MS-DOS a fault in the FATs can be catastrophic
• in UNIX a fault in the superblock can lose all
  data
• There are two main backup strategies
• keep copies of all or specific files on multiple
  secondary storage devices
• manual copying files onto separate devices,
  network drives
• automatic disk mirroring or RAID
• backup all files on a regular basis
• most operating systems include facilities for
  system backups

11
Incremental Backups

• When performing regular backups there are ways
  to avoid having to backup every file at each
  backup
• weekly e.g. overnight on Sunday evening
• backup every file
• daily e.g. every weekday evening
• backup only the files that have changed since
  yesterday
• OR backup files that have changed since last
  weekly backup
• The schedule period can be changed to suit user
• e.g. use monthly and weekly periods (not
  weekly/daily)
• May use multiple backup sets
• if latest set is faulty, then you still have
  previous set(s)

12
File System Consistency

• Another aspect of reliability is consistency
• imagine that the first block is being written to
  a newly created file the following actions take
  place
• the free list is scanned to find a suitable free
  block
• the data is written to this block on the hard
  disk
• the allocation list for the file is updated to
  add the block
• the allocated block is removed from the free list
• suppose there is a system crash between these
  steps
• There are two types of consistency
• block consistency
• every block should be allocated (once), bad or
  free
• file system consistency
• for example file size attributes match number of
  blocks

13
Consistency Algorithm

• Create a table with two counters per disk block
• one is to keep track of how many times the block
  is in a file
• one is to keep track of how often it is present
  in the free list
• initialise all counters to zero
• Descend the entire directory tree to visit every
  file
• for each block allocated to a file, increment 1st
  counter
• Examine the free list
• for each block, increment the blocks second
  counter
• If consistent, every block will have a one in
  either the first counter or the second counter
• any other state signifies a type of inconsistency

14
Consistency Example
consistent
missing block easy to deal withadd block to free
list
duplicate free block easy to deal withrebuild
the free list
duplicate data block more difficult to deal
with copy one to free block
15
Fragmentation

• The same fragmentation problem found in memory
  allocation is found in file systems
• as blocks are allocated, freed and re-allocated
  the blocks for each file are distributed over the
  disk
• the free blocks are fragmented, not contiguous
• There are, in fact, two categories of
  fragmentation
• external fragmentation
• free blocks are separated and distributed over
  the disk
• internal fragmentation
• free space is wasted inside blocks when files
  dont fill the last block in the allocation chain

16
Defragmentation

• In a single-user system like MS-DOS / Win, a disk
  can be defragmented
• a special program is runto swap blocks around
  on the disk by copyinginto memory out again
• this process is very time consuming!

• This cannot be done in a multi-user system as
  files could be modified during defragmentation
• one solution is to reinitialise and restore the
  entire file system
• a better solution is to make the allocation
  strategies much more sophisticated, to keep
  fragmentation under control

17
File System Performance
18
Disk Caching

• Access to disk is much slower than to memory
• an operating system should try to minimise disk
  I/O
• A disk cache is a collection of blocks that
  logically belong on the disk, but are stored in
  memory
• read caching
• when a block is read, it is stored in memory and
  marked
• future reads for the same block are satisfied
  from the cache
• write caching
• when a block is written, it is simply marked as
  dirty
• blocks are physically written to disk only on
  certain conditions
• Managing the cache is similar to memory paging
• page replacement algorithms can be used for blocks

19
Write-Through Cache

• Although disk write caching improves performance
  greatly, it is likely to exacerbate problems of
  maintaining file system consistency
• any system crash is likely to have many dirty
  disk blocks in memory which have not been written
• particularly bad if these are important blocks
  (e.g. inodes)
• Most systems solve this by writing blocks that
  are critical to file system integrity directly to
  disk
• Some systems (e.g. MS-DOS) write all blocks
  straight to disk, the cache is only used for
  reads
• this is called a write-through cache

20
UNIX Disk Syncing

• UNIX has a straight forward cache using the
  least-recently-used algorithm for block
  replacement
• critical blocks are written immediately
• The problem of non-critical blocks remaining too
  long is solved by having a special system call
• the sync system call forces all dirty blocks to
  be written to the disk immediately
• it does not return until this has been done
• A system program (usually called update) runs
• it performs an endless loop sync sleep(30)
• no more than 30 seconds of work is lost due to a
  crash

21
Logical Block v. Virtual Block

• There are two methods for implementing caching

• logical block caching
• the operating system keeps track of physical
  sectors read from disk and caches the sectors by
  their addressing
• original UNIX (BSD4.3), MS-DOS / Windows 3.1 /
  Win 9x

• virtual block caching
• the operating system keeps track of blocks at
  the file level (e.g. the 2nd block of file
  FRED.DOC)
• newer UNIX (Linux, Solaris, System V), Windows NT

22
Windows NT Cache Manager

• A fixed amount of virtual address space is
  allocated to the cache manager
• 128Mb 64Mb per 4Mb real memory above 16Mb
• Only a small amount of the virtual cache is in
  real physical memory at any given time
• file I/O cache is closely integrated with VM
  manager

non-cached I/O
23
Read-Ahead Lazy Writing

• Because WinNT works by virtual block caching, it
  can monitor the usage pattern of file access
• if a file is being read sequentially, then future
  requests will be anticipated
• a user process can hint the cache manager that
  this is likely
• the next blocks in the file are read-in during
  idle time
• this is called read-ahead caching
• The cache manager accumulates dirty blocks
• every second a system thread wakes up and a
  quarter of the dirty data is flushed to disk
• this is called lazy-writing
• write-through can also be selected by user
  processes

24
Summary

• Disk space management
• block allocation
• keeping track of free blocks
• File system reliability
• file system consistency
• fragmentation and defragmentation
• File system performance
• disk caching concepts
• logical block v. virtual block caching
• Windows NT cache manager