Project 3: File System Design

1
Project 3 File System Design

• COS318
• Fall 2002

2
Last Time

• Web Server
• Extensive use of a file system on server machine
  without actually worrying about how it was
  implemented.

server
client
u/
request
vivek/
response
public_html/
cos318_02/
index.html
3
Outline

• Introduction
• Design requirement
• Helper functions
• Implementation of file system calls
• Hints
• Reference

4
Introduction

• Why do we need a file system?
• Abstraction
• Protected access
• Security
• Simple Unix-like file system
• i-node
• directory tree
• links(symbolic/physical)

5
What does my file server need to accomplish

• A New Client Program with
• New Methods
• GET ltdir name file namegt
• Fetch a file or dir listing and show them
• CREATE ltdir namegt
• Create a dir in server disk
• DELETE ltfile name dir namegt
• Delete a file or dir in server disk
• populated directories should not be deleted
• PUT ltfile namegt ltlocal file namegt ltstartposgt
  ltendposgt
• Create a new file in server disk, or append to an
  exsiting file with contents from local file
• Notice name uses absolute
  path

6
More on file server functionality

• Use our client program to create a populated
  directory tree inside of server disk, and then
  use your web server to serve this content.
• More test programs
• Randomly requests
• Maybe not just be PUT or GET after some file
  deletions, things will become more interesting.
• Coming soon

7
So what do you have

• A Virtual Disk a data structure
• Disk_name dev_318
• Disk_pointer used to access the disk(in fact a
  file descriptor)
• Sector_size 512 bytes
• Capacity M, G
• Rather than write to a real disk, you'll be
  building a disk inside of a large file
• All development and debugging are in user space

8
Helper Functions not tuxedo

• GetExistingDisk(char pathname)
• CreateNewDisk(void)
• CloseDisk(void)
• ReadSector(int sectorNum, char buf)
• WriteSector(int sectorNum, char buf)
• Notice
• You are allowed to access disk only with those
  function calls

9
Helper Functions(cont.)

• GetExistingDisk(char pathname)
• returns a pointer to previously created disk in
  directory pathname, or exits if no such disk
• CreateNewDisk(void)
• returns a pointer to a new disk
• CloseDisk(void)
• closes the virtual disk(dev_318)

10
Helper Functions(cont.)

• ReadSector(int sectorNum, char buf)
• reads 512 bytes from disk
• WriteSector(int sectorNum, char buf)
• writes 512 bytes from disk
• Notice
• If any other user space function calls can
  facilitate your work, feel free to use them.
• Using file system functions to build disk is
  relatively easy compared to using really disk to
  build file system

11
What do I need to do

• You should build on the code you wrote for the
  previous assignments
• Good thing we have a reference server?
• The first step is to understand the default file
  system provided by the base code(helper
  functions)
• You'll have to understand at least one new header
  with one of the new methods.

12
What do I need to do(cont.)

• You'll have to write versions of various system
  calls that you used in Project 1
• mkfs, open, close, read, and write
• Additionally, you'll also have to implement
  versions of stat, mkdir, and a few others, also
  function calls to getting file metadata and
  directory information in order to properly build
  directory listings
• No need to handle all of the various flags and
  parameters, but should implement enough to get
  the project working correctly

13
One example-open()

• For example, the open() system call can be used
  to try to test the existence of a file in
  addition to creating it for writing
• O_CREATE the file creation flag
• To properly handle PUT" method
• Not always true need other flags
• GET a non-existent file

14
mkfs

• File system creation
• Dos format
• Linux mkfs/mke2fs/mkfs.msdos
• Solaris newfs
• Structure of a typical file system(unix-like)
• Superblock
• Describes layout of i-Nodes, bitmaps, blocks,
  root directory, etc.
• i-Nodes
• Describes a file/dir Link counter, list of
  blocks, size, etc. (owner, permission, )
• Block Allocation Bitmap
• Data blocks

15
mkfs(cont.)

• Creates the file system
• Set up the Superblock
• Flag all i-Nodes as type free (f)
• Zero out the Alloc Bitmap
• Create the root directory
• Flag an i-Node as type directory (D)
• Store its number in Superblock
• Alloc one block in the bitmap
• Initialize the . and .. directory entries
  and set the others as being INVALID

16
SuperBlock

• Describes the layout of the file system
• signature A magic number
• size Total size of fs in blocks
• root_inode The root directory
• inode_start 1st block for i-Nodes
• inode_blocks Total i-Nodes blocks
• bitmap_start 1st block of bitmap
• bitmap_blocks Total bitmap blocks
• alloc_start 1st block for data
• num_blocks Total data blocks
• free_blocks Count of unallocated blocks

17
Bitmap

• Bitmap
• Indicates which data blocks are free, and which
  are used
• Maps the data blocks only (not i-Nodes)

18
I-node

• Describes a file/directory
• type File, directory
• opens Count of open file handles
• links Count of links to this i-Node
• size Total byte count
• direct10 List of data blocks
• indirect3 List of indirect blocks
• Advantages
• Fast access to small files
• Supports very large files
• Support sparse files

19
Hints

• Break this project into two components -
  expansion of the main server itself, and
  implementation of the underlying disk
• Within the file system
• Start with the creation of files and try to get
  that working properly
• Then build directories on  top of that mechanism
• Finally work on directory listing support.
• For files direct blocks first

20
More Hints

• Keep the high level interface as much as possible
• By having wrappers that can either call the real
  call or your replacement, you'll be able to do
  testing quickly and efficiently
• Standard interface
• Stat(const char pathname, struct stat buf)

21
stat

• Returns statistics for given a filename
• inodeNo The files i-Node number
• Type File or directory
• Opens The open count
• Size The byte count to EOF
• numBlocks Number of allocated blocks

22
Reference

• Check the manual page of the function calls
• What they do
• Whats their interface
• Whats the most useful parameters

23
Wrap Up

• Fun again? Probably.
• Not too difficult.
• But a little bit more coding.
• Questions?