CSE 8343 - Team A1

1
CSE 8343 - Team A1

• Presentation 1
• September 17, 2001
• Modern File Systems

2
EXT2/EXT3

• Alex MacFarlane

3
Outline

• Introduction
• History
• ext2fs Structure
• Advanced Features
• Performance Optimizations
• Software Support
• The future ext3fs
• Bibliography

4
Introduction

• Most widely used filesystem on Linux
• Supports 4TB filesystems
• Supports 2GB filesize
• Supports filenames up to 255 chars
• Variable block size
• Extensible for future growth
• Developed by Rémy Card, Theodore Ts'o, and
  Stephen Tweedie

5
Brief History

• Minixfs was the original filesystem for Linux
  but too limited
• VFS added to the Linux kernel (ca. 1991)
• VFS allowed for integration of Extended
  Filesystem, extfs (1992)
• extfs was too slow and had some limitations
• ext2fs is born in Jan 1993, based upon extfs code
• Over time stability has been improved and
  features added.

6
ext2fs Structure

• Filesystem Toplevel

Block Group
7
Inodes
Each inode contains information for one file

• Type of file (char/block/link/etc)
• uid of owner
• gid of file
• Size in bytes
• Last access time
• Last inode modification time
• Last content modification time
• Time when file was deleted
• Number of links pointing to file
• Number of blocks allocated to this file
• Fragment information
• Flags

8
Inode Diagram
9
Advanced Features

• Reserved blocks for superuser
• Synchronous updates
• Secure Deletion
• Undelete Information
• Immutable Files
• Filesystem State Tracking
• Clean / Not Clean / Erroneous
• Maximal mount count / interval

10
Performance Optimizations

• Fast symbolic links
• Readaheads on sequential or directory reads
• Block groups keeps inodes and data close
• Preallocation leads to contiguous allocation (75
  hit rate on full filesystems)

11
Software Support

• ext2fs utilities (e2fsprogs)
• e2fsck
• tune2fs
• mke2fs
• dumpe2fs, debugfs
• ext2fs library
• Easy maintenance of code
• Programs need not be recompiled to use new code

12
ext3fs

• An extension of ext2fs to provide journaling
  support.
• Increases availability and reliability.
• Completely backward compatible with ext2fs.
• Uses jfs generic journaling layers to provide
  transaction support.
• Ships with upcoming Redhat Linux 7.2

13
Bibliography

• Analysis of the Ext2fs structure
• Louis-Dominique Dubeau
• http//step.polymtl.ca/ldd/ext2fs/ext2fs_toc.html
  
• Design and Implementation of the Second Extended
  Filesystem
• Rémy Card, Theodore Ts'o, Stephen Tweedie
• http//khg.redhat.com/HyperNews/get/fs/ext2intro.h
  tml
• Johns Spec of the Second Extended Filesystem
• John Newbigin
• http//uranus.it.swin.edu.au/jn/explore2fs/es2fs.
  htm
• ext2fs home page
• http//web.mit.edu/tytso/www/linux/ext2.html
• Linux ext2fs Undeletion mini-HOWTO
• http//www.linuxdoc.org/HOWTO/mini/Ext2fs-Undeleti
  on.html
• A Tour of the Linux VFS
• http//khg.redhat.com/HyperNews/get/fs/vfstour.htm
  l

14
Solaris File Systems

• Garrick Williamson

15
The UNIX File System (UFS)

• The UXIX File System (UFS) was derived from the
  Berkeley UNIX Fast File System developed during
  the 1980s.
• Supports 1TB file systems
• Supports 2 GB file size
• Variable block size

16
UFS Structure

• 4 types of blocks boot block, super block, Inode
  and Storage/Data block.

17
Inode Structure

• Each Inode contains information for one file
• File Length(bytes)/File Type/File Mode(r,w,etc)
• Link Count
• Owner and Group Ids
• Access Privilege
• Time of Last Access
• Time of Last Modification
• Etc.

18
Inode Diagram
19
UFS Error Checking/Recovery

• Due to UFS storing of large amounts of data in
  caches in main memory, the potential of losing
  data is substantial when the system crashes.
• A file-system consistency check must be performed
  at reboot in order to ensure reliable operation
  after the next mount of the file system.
• As file systems increase in their size, the time
  performance of the consistency check has become
  unacceptable in its length.
• In order to improve this newer file systems use
  logging techniques to facilitate faster recovery
  times.

20
UFS Comments

• UFS is the file system that is shipped with
  Solaris.
• UFS uses block based allocation schemes which
  provide adequate random access and latency for
  small files, but has limited through put for
  large files.
• Not suitable for continuous media applications.
• Not suitable for real-time access.
• As stated, UFS is not appropriate in the area of
  error recovery as file system size increases.

21
Veritas File System (VxFS)

• VxFS is geared toward UNIX environments that
  require high performance and availability and
  deal with large amounts of data. 1
• Supports 1TB file systems
• Supports 2 TB file size
• Variable block size (1024, 2048, 4096 and 8192
  bytes)
• Extent (one or more adjacent blocks) based
  represented as an address-length pair.
• Fast File System Recovery through logging
  (Journaling)

22
Inode Structure

• Each Inode (256 bytes) contains information for
  one file
• File Length
• Link Count
• Owner and group Ids
• Access privileges
• Time of last access
• Time of last modification
• Pointed to the extents that contain the files
  data

23
VxFS Comments

• Extents makes it possible for disk I/O to take
  place in units of multiple blocks since the
  storage is allocated in consecutive blocks.
• Multiple block operations are considerably faster
  than single block operations for sequential I/O.
• Uses Journaling, logging of disk operations, to
  facilitate faster recovery. Instead of checking
  the entire file system during a crash recovery,
  only the blocks listed in the log need to be
  checked. This substantially decreases the
  recovery time.

24
Bibliography

• Lee W., D. Su, J. Srivastava, QoS-based
  evaluation of file systems and distributed system
  services for continuous media provisioning,
  Information and Software Technology, Elsevier
  Science, December 2000, pp. 1021-1035.
• Kotz, David and Nils Nieuwajaar, Flexibility and
  Performance of Parallel File Systems, ACM
  Operating Systems Review 30(2), ACM Press, April
  1996, pp. 63-73.
• Peacock, J., A. Kamaraju, S. Agrawal, Fast
  Consistency Checking for the Solaris File System,
  Proceedings of the USENIX Annual Technical
  Conference, June 1998.
• Veritas File System 3.4, Admin. Guide
• Veritas Software Corporation
• http//www.sun.com/products-n-solutions/hardware/d
  ocs/Software/Storage_Software/VERITAS_File_System/
  index.html

25
XFS File System

• Brad Crabtree

26
XFS Overview

• 64-bit Database Journaling File System
• Developed by SGI in min 1990s
• Available for Linux, May 2001
• Guaranteed Rate I/O (GRIO)
• Individual Contiguous Extents lt 1TB
• PB of data and millions of files supported
  without performance degradation
• Dump while in use

27
XFS Overview (cont.)

• Supported by XLV Volume Manager
• striping (128 max), concatenation, and disk
  plexing (4 max)
• including root partition mirroring
• dynamic modification of mounted file systems
• remove/add/replace mirror, grow file system
• journal (can be) stored on separate partition for
  performance

28
XFS Architecture
29
Space Overview
SuperBlock
Alloc. Group Header
0
1
Inodes...
Data Block
Data Block
Data Block
Data Block
Extents
Allocation Groups
File System
30
Directory Design
31
B Tree Structure
32
B Tree Allocation

• Two Complimentary B Trees maintained for free
  space
• sorted by length, sorted by starting block
• allows fast allocation for large files as well as
  directory of many small files

Avoids multiple indirection and linear search of
directory files
33
Delayed Block Allocation

• As files are written
• Space is reserved but blocks are not allocated
• Data held in buffer cache
• Allows XFS to allocate largest number of blocks
  to an extent (contiguous space) and allocate
  fewest extents as possible

34
Superblock

• Superblock contains count of inodes, free inodes
  and free blocks
• Bottleneck Avoidance
• Move from common buffer cache to private
• Use special counter modify routines which only
  lock superblock until just before transaction
  occurs

35
Misc. Features

• Small File Handling
• Very small files are stored in the inodes
• Buffer cache before write for contig. alloc.
• Attribute Management
• User defined attributes stored outside of file
• Supports DMAPI for HMS File Systems
• Files identified by inode (magic cookie) and
  unique file ID

36
XFS Sub-volumes

• Data Sub-volume
• Variable Contiguous Extent allocations instead of
  blocks
• Allows more data to be accesses in one disk
  action
• Journal Sub-volume
• Separate circular serial log partition for each
  volume
• Real-Time Sub-volume (see GRIO)

37
Guaranteed Rate I/O (GRIO)

• Block sizes of 512 to 1G bytes
• Larger better for streaming media
• Guarantees are expressed as a file descriptor,
  data rate, duration, and start time
• Hard and Soft Rate Guarantees
• Hard requires disabling HD self-diagnostics and
  error correction, single SCSI bus

38
GRIO (cont.)

• Tunable Large extents are statically allocated at
  file system make
• Deterministic Bitmap Allocation

39
Bibliography

• XFS A Next Generation Journalled 64-Bit
  Filesystem With Guaranteed Rate I/O, Mike
  Holton, Raj Das, Silicon Graphics, Inc
• Modern File Systems and Storage,Rodney R.
  Ramdas, Competa IT b.v
• Open Source Systems - XFS Design Documents (all),
  Silicon Graphics, Inc.