Uppsala University Operating Systems

1
Uppsala UniversityOperating Systems
File System Brahim Hnichhttp//www.csd.uu.se/b
rahim/os1.html
2
Overview I

• Files
• file mapping
• file structure
• file types
• file access
• file attributes
• file operations
• memory-mapped files
• Directories
• hierarchical directory system
• path names
• directory operations
• File system implementation
• implementing files
• implementing directories
• shared files
• disk space management
• file system reliability
• file system performance

3
Overview II

• Security
• environment
• famous security flaws
• internet worm
• generic security attacks (viruses)
• design principles
• user authentication
• Protection mechanisms
• protection domains
• access control lists
• capabilities
• protection models
• Summary

4
Files

• All applications need to store and retrieve
  information
• amount of information?
• duration / persistence?
• Sharing of information?
• 3 requirements for long-term information storage
• it must be possible to store a very large amount
  of information
• the information must survive the termination of
  the process using it
• multiple processes must be able to access the
  information concurrently
• Store information on disks and/or other secondary
  media in files
• File must be persistent
• OS manages files through the file system
• user interface
• implementation

5
File Naming

• Character strings, (digits, special characters)
• two-part
• files names
• file extension
• example
• file.bak Backup file
• file.bin executable binary program
• file.doc documentation file
• file.html an HTML file

6
File Structure

• Byte sequence
• Record sequence
• Tree
• tree of records
• keys

Tree
1 Record
1 Byte
Ant
Fox
Pig
Cat
Cow
Dog
Pony
Rat
Worm
Goat
Lion
Owl
Hen
Ibis
Lamb
7
File Types I

• Regular files contain user information
• ASCII files lines of text
• binary file control information (internal
  structure)
• five sections header (magic number), text, data,
  relocation bits, symbol table

Magic Number
Header
Text size
Module name
Data size
BSS size
Object module
Date
Symbol table size
Entry point
Owner
Flags
Text
Header
Protection
Data
Object module
Size
Relocation bits
Header
Symbol table
Object module
16 bits
Archive
Executable file
8
File Types II

• Directories maintain structure of file system
  (system files)
• Character special files input/output handling
• Block special files disk handling

9
File Access

• Sequential access
• Random access
• READ operation
• SEEK operation

10
File attributes

• Name, data, date, and time of creation
• Protection who can access the file and in what
  way
• Password password needed to access the file
• Creator Id of person who created the file
• Owner Current owner
• Read-only flag 0 for read/write, 1 for read only
• Hidden flag 0 for normal, 1 do not display the
  listing
• System flag 0 for normal, 1 for system file
• Archive flag 0 has been backed up, 1 for needs
  to be backed up
• ASCII/binary flag 0 for ASCII, 1 for binary file
• Random access flag 0 for sequential access only,
  1 for random access
• Temporary flag 0 for normal, 1 for deleted on
  process exit
• Lock flag o for unlocked, 1 for locked
• Record length number of bytes in a record
• Key position offset of key within each record
• key length number of bytes in key field
• Creation time date and time file was created
• Time of last access date and time file was last
  accessed
• Time of last change date and time file was last
  changed

11
File Operations

• Create
• Delete
• Open
• Close
• Read
• Write
• Append
• Seek
• Get attributes
• Set attributes
• Rename

12
Unix Example
//cp src dest include ltsys/types.hgt include
ltfcntl.hgt include ltstdlib.hgt include
ltunistd.hgt define BUF_SIZE 4096 define MODE
0666 void main (int argc, char argv) int
src, dst, in, out char bufBUF_SIZE if
(argc ! 3) exit(1) src open(argv1,
O_RDONLY) if (src lt 0) exit(3) while (1)
in read( arc, buf, BUF_SIZE) if
(in lt 0) break out write(dst,
buf, in) if (out lt 0) break
close(src) close(dst) exit(0)
File descriptor
13
Memory-mapped Files

• MAP UNMAP map file(s) into address space of a
  running process
• File mapping segmentation

abc
Data
xyz
14
Directories
games
attributes
games
mail
attributes
mail
news
attributes
news
work
attributes
work
Attributes in the directory entry
Attributes elsewhere

• Directories keep track of files. A directory
  typically contains a number of entries, one per
  file.
• Directories are often themselves files
• hierarchical directory systems
• path names
• directory operations

15
Hierarchical Directory Systems
Single directory shared by all users
Root directory
Root directory
Root directory
A
B
C
C
A
B
User directory
A
A
B
A
C
B
B
B
A
C
B
C
C
A
B
B
B
Directory name
File owner
User sub-directories
B
B
B
B
Directory
File
16
Path Names
A UNIX directory tree
/
Root directory
bin
/usr/jim
etc
lib
usr
bin
etc
lib
usr
tmp
tmp
ast
jim
jim
ast

• Absolute path names /usr/ast/mailbox
• Relative path names mailbox
• Working / current directory /usr/ast

17
Directory Operations

• CREATE
• DELETE
• OPENDIR
• CLOSEDIR
• READDIR
• RENAME
• LINK
• UNLINK

18
File System Implementation

• Implementing files
• contiguous allocation
• linked list allocation (using an index)
• I-nodes
• Implementing directories
• CP/M
• MS-DOS
• UNIX
• Shared files
• Disk space management
• block size
• free block management
• disk quotas
• File system reliability
• bad block management
• backups
• file system consistency
• File system performance

19
Implementing Files I

• Key issue in implementing file storage is keeping
  track of which disk blocks go with which files
• Contiguous allocation
• simplest allocation scheme
• store each file as a contiguous block of data on
  the disk
• () simple to implement
• () good performance
• (-) not feasible unless the maximum file size is
  known at file creation time
• (-) fragmentation of the disk

20
Implementing Files II

• Linked list allocation
• one linked list of disk blocks per file
• () no fragmentation, every block can be used
• () enough to store disk address of first block
• (-) random access is slow
• (-) amount of data in block not a power of 2

File A
0
File Block 0
File Block 1
File Block 2
File Block 3
File Block 4
4 7 2
10 12
Physical block
21
Implementing Files III

• Linked list allocation using an index (MS-DOS)
• both disadvantages of linked list allocation can
  be eliminated by taking the pointer word from
  each block and putting it in a table or index in
  memory

Physical block
0
1
10
2
() entire block is available for data () random
access is easier (-) entire table must be in
memory all the time to make it work
3
4
7
File A starts
5
6
7
2
8
9
10
12
11
0
12
22
Implementing Files IV

• I-nodes (UNIX)
• associated with each file is a little table
  called I-node (index-node)

Addresses of data blocks
I-node
Single indirect block
Addresses of data blocks
Attributes
Single indirect block
Disk addresses
Double indirect block
Double indirect block
Single indirect block
Tripple indirect block
23
Implementing Directories I

• Before a file can be read, it must be opened
• OS uses the path name supplied by the user to
  locate the directory entry
• Directory entry provides the information needed
  to find the disk blocks
• Main function of the directory system is to map
  the ASCII file name onto the information needed
  to locate the data
• Where should the attributes be stored?

24
Implementing Directories II

• CP/M
• simple example
• only one directory containing the disk block
  numbers for each file

16
Bytes 1 8
3 1 2
File name
Disk block numbers
User code
File type (extension)
Block count
Extent
Directory entry
25
Implementing Directories III

• MS-DOS
• hierarchical directory trees

Bytes 8
3 1 10
2 2 2 4

File name
Size
Time
File type (extension)
Date
Reserved
First block number
Attributes
Directory entry
26
Implementing Directories IV

• UNIX
• simple directory structure
• all information about type, size, time ownership,
  and disk blocks contains the I-node

Bytes 2
14
File name
I-node number
Directory entry
Block 406 is /usr/ast directory
Block 132 is /usr directory
I-node 26 is for /usr/ast
Root Directory
I-node 6 is for /usr
26
.
1
.
6
.
Mode size times
Mode size times
6
..
1
..
1
..
64
grants
4
bin
19
dick
92
books
7
dev
30
erik
60
mbox
14
lib
51
jim
132
406
81
minix
9
etc
26
ast
17
src
6
usr
45
bal
8
tmp
27
Shared Files I

• File system becomes a directed acyclic graph
  (DAC)
• Symbolic linking create a file of type link
  containing path name of the file to be linked

Root directory
C
A
B
A
C
B
B
B
link
C
C
B
?
C
C
Shared file
28
Shared Files II

• Removal of linked files

Before Linking
After link is created
Owner has removed file
Cs dir.
Cs dir.
Bs dir.
Bs dir.
Owner C Count 1
Owner C Count 2
Owner B Count 1
29
Disk Space Management I

• Block size
• median size in UNIX is 1K

Data rate (KB/sec)
Disk space utilization ()
Disk space utilization
100 75 50 25 0
200 150 100 50 0
Data rate
128 256 512 1K 2K
4K 8K
30
Disk Space Management II

• Keeping track of free blocks
• linked list of free blocks
• each block holding as many free disk block
  numbers as will fit
• often free blocks are used to hold the free list
• bit maps
• a disk with n blocks requires a bit map with n
  bits
• free blocks are represented by 1
• allocated blocks are represented by a 0
• requires less space than the linked list

31
Disk Space Management III

• The purpose of the disk quota is to prevent
  people from hogging too much disk space
• Disk quotas are kept in a quota table, where
  every user has an entry

Quota table
Open file table
Soft block limit
Attributes Disk addreses User8 Quota pointer
Hard block limit
Quota record for user 8
Current of blocks
blocks warnings left
Soft file limit
Hard file limit
Current of files
file warnings left
32
File System Reliability I

• Bad block management
• disks and floppy disk have often bad blocks
• 2 solutions to bad block management
• Hardware solution
• dedicate a sector on the disk to the bad block
  list
• when the controller is first initialized, it
  reads the bad block list
• picks a spare block (or track)
• Software solution
• user or file system construct a file containing
  all the bad blocks
• they are removed from the free list
• bad block file must never be read or written

33
File System Reliability II

• Backups (whole disk can crash)
• incremental and complete dumps
• archive bit (MS-DOS)
• example (storage wasted)

Disk 0
Disk 1
Backup of data 1
Backup of data 0
Data 0
Data 1
CPU
34
File System Reliability III

• File system consistency
• system crashes during modify operations
• file system check operations

0 1 2 3 4 5 6 7 8 9 10 11
12 13 14 15
0 1 2 3 4 5 6 7 8 9 10 11
12 13 14 15
Blocks in use
1
1
0
1
0
1
1
1
1
0
0
1
1
1
0
0
1
1
0
1
0
1
1
1
1
0
0
1
1
1
0
0
0
0
1
0
1
0
0
0
0
1
1
0
0
0
1
1
0
0
0
0
1
0
0
0
0
1
1
0
0
0
1
1
Free blocks
consistent
Missing block
0 1 2 3 4 5 6 7 8 9 10 11
12 13 14 15
0 1 2 3 4 5 6 7 8 9 10 11
12 13 14 15
Blocks in use
1
1
0
1
0
1
1
1
1
0
0
1
1
1
0
0
1
1
0
1
0
2
1
1
1
0
0
1
1
1
0
0
0
0
1
0
2
0
0
0
0
1
1
0
0
0
1
1
0
0
1
0
1
0
0
0
0
1
1
0
0
0
1
1
Free blocks
Duplicate data block
Duplicate block in free list
Files check directory system (uses also table of
counters per file) -gt list indexed by I-node
number
35
File System Performance

• Block / buffer cache
• is the block likely to be needed soon?
• Is the block essential to the consistency of the
  file system?
• Blocks can be divided into categories
• I-node
• indirect
• directory
• full data
• partly-full data
• Write-through caches
• Data placement
• clustering (group of consecutive blocks)
• partitioning (distribution and I/O parallelism)

36
Security and Protection

• Security overall problem
• Protection mechanisms specific OS mechanisms
  used to safeguard information in the computer
• 2 facets data loss and intruders
• Causes for data loss
• Fires, floods, earthquakes, wars, rats gnawing
  tapes or floppy disks
• Hardware or software errors
• CPU malfunctions
• unreadable disks or tapes
• telecommunication errors
• program bugs
• Human errors
• incorrect data entry
• wrong tape or disk mounted
• wrong program run
• lost disk or tape
• Solution maintain adequate backups

37
Security Environment

• Intruders
• passive just want to read files they are not
  authorized to read
• active want to make an unauthorized changes to
  data
• Categories of active intruders
• casual prying by no-technical users
• snooping by insiders
• determined attempt to make money
• commercial or military espionage
• Privacy protecting individuals from misuse of
  information about them -gt legal and moral issues
• Security holes
• errors in OS
• Trojan horse attack
• password problem
• internal worm self replicating program making
  use of bugs in OS (UNIX)

38
Generic Security Attacks

• System should withstand the following attacks
• request memory pages, disk space, or tape and
  just read them
• try illegal system calls, or legal system calls
  with illegal parameters, or even legal but
  unreasonable parameters
• start logging in and out and then hit DEL,
  REBOOT, or BREAK halfway through the login
  sequence
• try modifying complex OS structures kept in user
  space
• Mislead the user by writing a program that types
  login on the screen and go away
• look for manuals that say do not do this
• convince a system programmer to change the system
  to skip certain vital security checks for any
  user with your login name (trapdoor)
• problems caused by personnel tricking or
  bribing/corruption
• Virus program fragment attached to a legitimate
  program with the intention of infecting other
  programs

39
Design Principles for Security

• The system design should be public
• The default should be no access
• check for current authority
• Give each process the least privilege possible
• The protection mechanism should be simple,
  uniform, and built in to the lowest layers of the
  system
• The scheme chosen must be psychologically
  acceptable

40
User Authentication

• Passwords
• choice of password
• en/decryption, salting
• one-time password
• answer personal questions instead of password
• challenge-response (compute unknown and may be
  changing algorithm)
• Physical identification
• identification card password
• fingerprint
• voiceprint
• signature analysis
• finger length analysis
• Counter measures
• make system access harder
• monitoring of all logins and critical operations
• lay baited traps to crash intruders

41
Protection Mechanisms I

• Protection domain set of (object, right) pairs
• each pair specifies an object and some subset of
  the operations that can be performed on it
• at every instant of time, each process runs in
  some protection domain
• collection of objects it can access
• and for each object it has some set of rights
• Objects that need to be protected
• hardware CPU, memory segment, terminal, disk
  drive, printer,
• software process, file, data-base, semaphore,
• each object has a unique name by which it is
  referenced, and a set of operations that can be
  carried out on it
• Rights permission to perform operations
• R read access allowed
• W write access allowed
• X execution allowed

42
Protection Rings and Matrices
Rings
0 operating system 1 critical utilities 2
grading procedures 3 student procedures
0
1
2
3
Matrices
File 1File2File3File4 File5File 6
Printer1Plotter2 Dom. 1 Dom. 2 Dom. 3
R
RW
Enter
Domain1
R
RWX
RW
W
Domain2
RWX
W
W
Domain3
Protection matrix with domains as objects
43
Protection Mechanisms II

• Access control list (ACL)
• storing protection matrix by column keeping only
  the nonempty elements
• associative with each object an (ordered) list
  containing all the domains that may access the
  object, and how
• Capability lists (capabilities)
• storing protection matrix by row keeping only the
  nonempty elements
• associate with each process a list of objects
  that may be accessed, along with an indication of
  which operations are permitted on each (its
  domain)
• protection of capability lists needed
• generic rights applicable to all objects, e.g.
  copy capability, copy object, remove capability,
  destroy object
• type manager modules
• rights amplification

44
Protection Mechanisms III

• Protection models
• Protection commands
• create object
• delete object
• create domain
• delete domain
• insert right
• remove right
• Authorized and unauthorized states
• Security policy
• no process may read any object whose level is
  higher than its own, but it may freely read
  objects at a lower level or at its own level
• no process may write information into any object
  whose level is lower that its own

45
Summary

• User point of view
• file system is a collection of files and
  directories, plus operations on them
• files can be read and written
• directories can be created and destroyed
• files can be moved from directory to directory
• file naming, structure, typing, access, and
  attributes
• Inside view
• disk block management
• free disk space management
• implementation techniques for files and
  directories
• Security and protection
• Next lecture?
• Input/Output