The Linux file system

1
The Linux file system

• Recitation 10

2
The Linux FS

• Linux supports many different file system types
• ext2 (second Extended File System")
• ext3 ( third Extended File System")
• Minix
• MSDOS (for PC compatibility, read floppies)
• /proc ()
• FAT
• NFS
• AFS
• Etc.
• All this file systems can be mounted,
  transparently in view of user.
• This support is achieved through the concept of
  VFS virtual file system

3
VFS - virtual file system

• VFS is a software layer acting as a "front end"
  to all these FS and more
• The key idea introducing a common file model
  capable of representing all supported FS
• Its main strength is to provide a standard
  interface for various file systems.
• Hide FS differences from applications

4
VFS - virtual file system

• passes calls to the actual file system
• Each specific FS implementation must translate
  its physical organization into the VFS's common
  file model.
• For example in common file model each directory
  entry is a file.containing list of directories
  and files.
• In FAT FS directories are not files.
• construct on the fly, when needed, the files
  corresponding to the directories. Such files
  exist only as objects in kernel memory..
• data stored on block-oriented devices
• hard disk, floppy disk, CD-ROM etc.
• kernel holds information on many file-system.
• In a way of FS modules.
• defines unified interface to them.
• This allow to mount different FS to Linux/UNIX.

5
VFS simple copy example

• VFS is an abstraction layer between applications
  and FS implementation.
• VFS type of target and source are not needed to
  be known.
• Interaction with VFS is made by means of generic
  system calls

6
VFS
7
FS types supported in Linux

• Disk FS
• Linux FS
• Ext2,ext3, ReiserFS
• Created for Linux but ported to other OS as well.
• Unix FS.
• SYSV, UFS, MINIX, VERITAS VxFS.
• Dos/Windows FS
• FAT, NTFS
• ISO9660 CD-ROM FS.
• And others
• Remote FS, using network
• NFS, AFS, SMB
• FS to manage kernel data structures

8
Linux special FS

• Allowing system processes, and administrators to
  control and view kernel operation in a simple
  way.
• Allow implementation of special OS facilities
  like pipes and sockets.
• Some special FS
• devfs used for work with devices as a file
  interface.
• pipefs used for system pipes
• proc need I say more

9
VFS structure

• VFS defines a set of functions that every FS has
  to implement.
• VFS knows about FS types supported in kernel.
• Holds a table where each entry is a file system
  type.
• Entry holds file system type name and a pointer
  to function called at mounting time.
• This function reads super-block from disk and
  return mounted FS descriptor to VFS.
• After FS is mounted, VFS functions can use this
  descriptor to access the physical FS routines.

10
Mounted FS descriptor

• contains several kinds of data
• information that are common to every FS types.
• pointers to functions provided by the physical FS
  kernel code.
• private data maintained by the physical FS code
• The function pointers contained in the FS
  descriptors allow the VFS to access the FS
  internal routines.

11
VFS data structure - superblock

• VFS uses objects (conceptually) to manage FS,
  directories and files.
• Superblock object general information on a
  specific file system
• object usually corresponds to a FS control block
  stored on disk.
• Some fields of object
• s_dev Identifier to device
• s_type Type of FS
• s_op Superblock methods
• s_inodes_count no of I-nodes,
• s_blocks_count FS size in blocks,
• s_free_blocks_count no free blocks,
• s_blocksize block size in bytes

12
VFS data structure - superblock

• Examples for superblock methods
• read_inode(inode)
• Read I-node info from disk.
• dirty_inode(inode)
• Invoked when the inode is marked as modified
  (dirty). Used by FS like Ext3 to update the FS
  journal on disk.
• delete_inode(inode)
• Deletes the data blocks containing the file,
  disk inode, and VFS inode.
• put_super(super)
• Releases the superblock object (corresponding FS
  is unmounted).
• FS that does not support a method will set its
  pointer to NULL.
• No mount command superblock object doesnt
  exist before mount.

13
VFS data structure Ext2 superblock methods
14
VFS data structure I-node

• I-node object information on specific file.
• All information needed by FS to handle a file
• I-node number correspond to exactly one file.
• this object usually corresponds to a file control
  block stored on disk
• Some fields of object
• i_ino I-node number identifier
• i_count Usage counter.
• i_mode File type and access rights.
• i_uid Owner identifier.
• i_size File length.
• i_op I-nodes methods
• i_fop Default ile methods

15
I-node object

• Each file in the regular FS has an I-node on the
  disk with file information.
• I-node identify file.
• File name might change and can not be used to
  identify file.
• I-node object is a memory object which
  initialized from disk I-node.
• If I-node object changes, disk I-node is needed
  to be updated as well.
• I-node object contain information on file such
  as
• Ownership, size, time stamp, pointers to data
  blocks etc.

16
I-node block pointers
17
I-node and file size

• File sizes that can be held with I-node with ext2
  FS

18
I-node assignment

• The file system contain a list of I-nodes, when a
  process need a new I-node, the kernel can search
  the I-nodes list for a free one.
• To improve performance, the super block contains
  the number of free I-nodes in the file system.
• If the super block list of free I-nodes is empty,
  the kernel searches the disk and places as many
  free I-node numbers as possible into the super
  block.
• Freeing an I-node - The kernel places the I-node
  number in the super block free I-nodes list.

19
VFS data structure I-node

• Examples for I-node methods
• create(dir, dentry, mode)
• Creates a new disk inode for a regular file
  associated with a dentry object in some
  directory.
• mkdir(dir, dentry, mode)
• Creates a new inode for a directory associated
  with a dentry object in some directory.
• rmdir(dir, dentry)
• Removes from a directory the subdirectory whose
  name is included in a dentry object.
• permission(inode, mask)
• Checks if specified access mode is allowed for
  file associated to inode.

20
VFS data structure File object

• File object contains data on opened file
• A file object describes how a process interacts
  with a file it has opened.
• File descriptor in descriptor table is a pointer
  to file object.
• This information exists only in kernel memory
  during the period when each process accesses a
  file.
• File object is initialized form file I-node.
• Contains pointer to dentry pointing to file, file
  flags (read/write) etc.

21
VFS data structure File object

• Some fields of object
• f_dentry Directory object who points to file.
• f_vfsmnt Mounted FS containing the file
• f_op Pointer to file operation table
• f_pos Current file offset (file pointer)
• f_ralen Number of read-ahead bytes
• f_uid User's UID

22
VFS data structure File object

• Main information stored in a file object is the
  file pointer
• the current position in the file from which the
  next operation will take place.
• several processes may access the same file
  concurrently, the file pointer cannot be kept in
  the I-node object.
• Examples for file methods
• open(inode, file)
• Opens a file by creating a new file object and
  linking it to the corresponding inode object
• llseek(file, offset, origin)
• Updates the file pointer.
• read(file, buf, count, offset)
• write(file, buf, count, offset)

23
VFS data structure denty object

• dentry object
• Stores information about the linking of a
  directory entry with the corresponding file.
• Each disk-based FS stores this information in its
  own particular way on disk.
• Kept in cache for better performance.
• The kernel creates a dentry object for every
  component of a pathname that a process looks up
• dentry object associates the component to its
  corresponding inode
• For example, when looking up the /tmp/test
  pathname,
• kernel creates a dentry object for the / root
  directory,
• a second dentry object for the tmp entry of the
  root directory,
• and a third dentry object for the test entry of
  the /tmp directory.

24
dentry object

• Kernel create a dentry for each directory entry.
• denrty contain a pointer to I-node of the file or
  directory.
• Creating dentry object from disk data is
  expensive operation.
• dentries are cached in FS.

25
VFS data structure denty object

• Some fields of object
• d_parent Dentry object of parent directory
• d_child Pointers for the list of dentry objects
  included in parent directory
• d_mounted Flag set to 1 if dentry is the mount
  point for a FS
• d_name Filename
• d_op Dentry methods
• Examples for denrty methods
• d_revalidate(dentry, flag)
• Determines whether the dentry object is still
  valid before using it
• d_delete(dentry)
• Called when the last reference to a dentry object
  is deleted

26
UNIX organization into blocks

• boot block
• reserved for code info to boot the OS
• not used if we don't boot from that device
• super block
• contains control management info for the whole
  file system of inodes, of (free) blocks,
  etc.
• inode blocks contain inodes
• data blocks contain file data, directories, and
  indirect blocks

27
Mount/Un-mount

• New FS can be added to VFS.
• Operation is called mounting.
• New FS is mounted to existing directory.
• This point is called mounting point.
• The main FS is mounted into the root of VFS.
• Mounting is performed using mount().
• FS can be mounted in few different directories.

28
Mount/Un-mount

• Removing FS is done using the unmount() call.
• Will succeed only if user has needed permissions.
• If unmounted FS has mounted FS in its
  directories, theses FS will be unmounted as well.
• Regardless no of mounts, FS has only one
  superblock.

29
Mount/Un-mount

• To keep track of mount FSs, VFS has a table of
  mounted filesystem descriptors
• each descriptor is a data structure that has type
  vfsmount,
• Example fields of descriptor
• mnt_parent Points to the parent FS on which
  this FS is mounted on
• mnt_mountpoint Points to the dentry of the mount
  directory of this FS
• mnt_sb Points to the superblock object of this
  FS
• mnt_count Usage counter

30
mount/unmount from shell.

• From shell
• mount
• Call with no parameters will display all mounted
  fs in system (the mount table).
• To mount a new device
• mount t type device dir
• Example mount t vfat /dev/sda2 /mnt/WinFAT
• unmount dir, will unmount it
• Example unmount /mnt/WinFAT

31
Unix/Linux system callsopen()/create().

• include ltsys/types.hgt
• include ltsys/stat.hgt
• include ltfcntl.hgt
• int open(const char pathname, int flags)
• int open(const char pathname, int flags, mode_t
  mode)
• int creat(const char pathname, mode_t mode)
• Flags
• O_RDONLY, O_WRONLY, O_RDWR, O_CREAT, O_APPEND,
  O_DIRECTORY
• Mode (for file permissions)
• S_IRWXU 00700 owner has read, write and execute
  permission
• S_IRUSR 00400 owner has read permission
• S_IWUSR 00200 user has write permission
• (see man 2 open for full documentation)

32
close()

• include ltunistd.hgt
• int close(int fd)
• Close file descriptor.

33
lseek()

• reposition read/write file offset.
• include ltsys/types.hgt
• include ltunistd.hgt
• off_t lseek(int fildes, off_t offset, int
  whence)
• int whence
• SEEK_SET The offset is set to offset bytes.
• SEEK_CUR The offset is set to its current
• location plus offset bytes.
• SEEK_END The offset is set to the size of the
  file plus
• offset bytes. (in data is written gap is
• filled with zeros)

34
lseek() examples.

• Move to byte 16
• newpos lseek( fd, 16, SEEK_SET )
• Move forward 4 bytes
• newpos lseek( fd, 4, SEEK_CUR )
• Move to 8 bytes from the end
• newpos lseek( fd, -8, SEEK_END )
• Move bckward 3 bytes
• lseek(fd, -3, SEEK_CUR)

35
read()

• read from a file descriptor
• include ltunistd.hgt
• ssize_t read(int fd, void buf, size_t count)
• Try read count bytes from file descriptor fd into
  buf buffer.
• Return value The number of bytes read is
  returned
• (zero indicates end of file), and the file
• position is advanced by this number.

36
write()

• write to a file descriptor
• include ltunistd.hgt
• ssize_t write(int fd, const void buf, size_t
• count)
• Writes up to count bytes to the file referenced
  by
• the file descriptor fd from the buffer starting
  at
• buf.
• Return value the number of bytes written are
  returned
• (zero indicates nothing was written). On error,
  -1 is
• returned.

37
Link() soft and hard

• include ltunistd.hgt
• int link(const char existingpath,
• const char newpath)
• int symlink(const char actualpath,
• const char newpath)
• int unlink(const char pathname)
• int remove(const char pathname)
• int rename(const char oldname,
• const char newname)

38
chmod()

• include ltsys/types.hgt
• include ltsys/stat.hgt
• int chmod(const char pathname, mode_t mode)
• Sets the file permission bits of the file
  specified by the pathname pathname to mode.
• Must be owner to change mode

39
chown()

• include ltsys/types.hgt
• include ltunistd.hgt
• int chown( const char pathname,
• uid_t owner,
• gid_t group)
• The owner of the file specified by path or by fd
  is changed. Only the
• super-user may change the owner of a file. The
  owner of a file may
• Change the group of the file to any group of
  which that owner is a
• member. The super-user may change the group
  arbitrarily.