Virtual Memory, File-System Interface

1
Virtual Memory, File-System Interface
2
Background

• Virtual memory separation of user logical
  memory from physical memory.
• Only part of the program needs to be in memory
  for execution
• Logical address space can therefore be much
  larger than physical address space
• Allows address spaces to be shared by several
  processes
• Allows for more efficient process creation
• Virtual memory can be implemented via Demand
  paging

3
Virtual Memory Larger Than Physical Memory
?
4
Demand Paging

• Bring a page into memory only when it is needed
• Less I/O needed
• Less memory needed
• Faster response
• More users
• Page is needed ? reference to it
• invalid reference ? abort
• not-in-memory ? bring to memory
• Lazy swapper never swaps a page into memory
  unless page will be needed
• Swapper that deals with pages is a pager

5
Page Table When Some Pages Are Not in Main Memory
6
Handling a Page Fault
7
Process Creation

• Virtual memory allows other benefits
• Copy-on-Write (COW) more efficient process
  creation
• allows both parent and child processes to
  initially share same pages in memory
• If either process modifies a shared page, only
  then is the page copied

8
What happens if there is no free frame?

• Page replacement find some page in memory, but
  not really in use, swap it out
• Goal minimize number of page faults
• Only modified pages are written to disk to reduce
  overhead of page transfers

9
Basic Page Replacement

1. Find the location of the desired page on disk
2. Find a free frame - If there is a free
   frame, use it - If there is no free frame,
   use a page replacement algorithm to select a
   victim frame
3. Bring desired page into the (newly) free frame
   update the page and frame tables
4. Resume the process

10
Page Replacement Algorithms

• Want lowest page-fault rate
• Evaluate algorithm by running it on a particular
  string of memory references (reference string)
  and computing the number of page faults on that
  string
• In all our examples, the reference string is
• 1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5

11
First-In-First-Out (FIFO) Algorithm

• Reference string 1, 2, 3, 4, 1, 2, 5, 1, 2, 3,
  4, 5
• 3 frames (3 pages can be in memory at a time per
  process)
• 4 framesBeladys Anomaly more frames ? more
  page faults

1
1
4
5
2
2
1
3
9 page faults
3
3
2
4
1
1
5
4
2
2
1
10 page faults
5
3
3
2
4
4
3
12
Optimal Page Replacement

• Replace page that will not be used for longest
  period of time
• Used for measuring how well your algorithm
  performs

13
Least Recently Used (LRU) Page Replacement

• Every page entry has a counter every time page
  is referenced through this entry, copy the clock
  into the counter
• When a page needs to be changed, look at the
  counters to determine which to change

14
LRU Algorithm (Cont.)

• Stack implementation keep a stack of page
  numbers in a double link form
• Page referenced
• move it to the top
• requires 6 pointers to be changed

15
Use Stack to Record The Most Recent Page
References

• keep a stack of page numbers in a double link
  form
• Page referenced move it to the top, requires 6
  pointers to be changed

16
Counting Algorithms

• Keep a counter of the number of references that
  have been made to each page
• Least Frequently Used (LFU) Algorithm replaces
  page with smallest count
• Most Frequently Used (MFU) Algorithm based on
  the argument that the page with the smallest
  count was probably just brought in and has yet to
  be used

17
Frame Allocation

• Equal allocation For example, if there are 100
  frames and 5 processes, give each process 20
  frames.
• Proportional allocation Allocate according to
  the size of process

18
Thrashing

• If a process does not have enough pages, the
  page-fault rate is very high.
• Thrashing ? a process is busy swapping pages in
  and out
• Demand paging works because of locality model
• Process migrates from one locality to another
• Localities may overlap
• Why does thrashing occur?? size of locality gt
  total memory size

19
Working-Set Model

• ? ? working-set window ? a fixed number of page
  references Example 10,000 instruction
• WSSi (working set of Process Pi) total number
  of pages referenced in the most recent ? (varies
  in time)
• if ? too small will not encompass entire locality
• if ? too large will encompass several localities
• if ? ? ? will encompass entire program
• D ? WSSi ? total demand frames
• if D gt m ? Thrashing
• Policy if D gt m, then suspend one of the processes

20
Move on to File System

• To explain the function of file systems
• To describe the interfaces to file systems
• To explore file-system protection

21
File Concept

• A named collection of related information that is
  stored on secondary storage
• The smallest allotment of secondary storage
• A sequence of bits, bytes, lines or records
• Types
• Data
• numeric
• character
• binary
• Program

22
File Structure

• None - sequence of words, bytes
• Simple record structure
• Lines
• Fixed length
• Variable length
• Complex Structures
• Formatted document
• Relocatable load file executable files, library
  files
• Indexed file for fast access to data
• Can simulate last two with first method by
  inserting appropriate control characters

23
Example of Index and Relative Files
23
24
File Attributes

• Name only information kept in human-readable
  form
• Identifier unique tag (number) identifies file
  within file system
• Type needed for systems that support different
  types
• Location pointer to file location on device
• Size current file size
• Protection controls who can do reading,
  writing, executing
• Time, date, and user identification for
  creation/last modification/access, used for
  protection, security, and usage monitoring
• Information about files are kept in directory
  structure, which is maintained on the disk

25
File Operations

• File is an abstract data type with operations
  such as
• Create
• Write
• Read
• Reposition within file
• Delete
• Truncate
• Open(Fi) search the directory structure on disk
  for entry Fi, and move the content of entry to
  memory
• Close (Fi) move the content of entry Fi in
  memory to directory structure on disk

26
Open Files

• Several pieces of data are needed to manage open
  files
• File pointer pointer to last read/write
  location, per process that has the file open
• File-open count counter of number of times a
  file is open to allow removal of data from
  open-file table when last processes closes it
• Disk location of the file cache of data access
  information
• Access rights per-process access mode information

27
Open File Locking

• Provided by some operating systems and file
  systems
• Mediates access to a file
• Mandatory or advisory
• Mandatory access is denied depending on locks
  held and requested
• Advisory processes can find status of locks and
  decide what to do

28
File Types Name, Extension
29
Access Methods

• Sequential Access
• read next
• write next
• reset
• no read after last write
• (rewrite)

30
Simulation of Sequential Access on Direct-access
File

• Direct Access, n relative block number
• read n
• write n
• position to n
• read next
• write next
• rewrite n

31
Directory Structure

• Directory a collection of nodes containing
  information about all files

Directory
Files
F 1
F 2
F 3
F 4
F n
Both the directory structure and the files reside
on disk
32
Disk Structure

• Disk can be subdivided into partitions
• also known as minidisks, slices
• Disks or partitions can be protected against
  failure using RAID (Redundant Array
  of Independent Disks)
• Disk or partition can be used raw without a
  file system, or formatted with a file system
• Entity containing file system known as a volume
• Each volume containing file system also tracks
  that file systems info in device directory or
  volume table of contents
• general-purpose file systems vs special-purpose
  file systems

33
A Typical File-system Organization
34
Operations Performed on Directory

• Search for a file
• Create a file
• Delete a file
• List a directory
• Rename a file
• Traverse the file system

35
Organize the Directory (Logically) to Obtain

• Efficiency locating a file quickly
• Naming convenient to users
• Two users can have same name for different files
• The same file can have several different names
• Grouping logical grouping of files by
  properties, (e.g., all Java programs, all games,
  )

36
Single-Level Directory

• A single directory for all users

Unique naming problem Grouping problem
37
Two-Level Directory

• Separate directory for each user

• Path name
• Can have the same file name for different user
• Efficient searching
• No grouping capability

38
Tree-Structured Directories
39
Tree-Structured Directories (Cont)

• Efficient searching
• Grouping Capability
• Current directory (working directory)
• cd /spell/mail/prog

40
Tree-Structured Directories (Cont)

• Absolute or relative path name
• Creating a new file is done in current directory
• Delete a file
• rm ltfile-namegt
• Creating a new subdirectory is done in current
  directory
• mkdir ltdir-namegt
• Example if in current directory /mail
• mkdir count

mail
prog
copy
prt
exp
count
Deleting mail ? deleting the entire subtree
rooted by mail
41
Acyclic-Graph Directories

• Have shared subdirectories and files

42
Acyclic-Graph Directories (Cont.)

• Issues
• A file can have more than one path (aliasing
  problem)
• If dict deletes list ? dangling pointer
• Solutions
• Backpointers, so we can delete all pointers
• Count number of references to a file
• Implement shared files / directories
• New directory entry type
• Link another name (pointer) to an existing file
• Resolve the link follow pointer to locate the
  file

43
General Graph Directory
44
General Graph Directory (Cont.)

• How do we guarantee no cycles? (avoid infinite
  loops)
• Allow only links to files, not subdirectories
• Garbage collection delete items that have no
  reference to it
• Traverse file system and mark everything that can
  be accessed
• Collected everything that is not marked onto a
  list of free space
• Every time a new link is added, use a cycle
  detectionalgorithm to determine whether it is OK

45
File Sharing in Multiple User System

• Sharing of files on multi-user systems is
  desirable
• Sharing may be done through a protection scheme
• Identify users
• User IDs identify users, allowing permissions and
  protections to be per-user
• Group IDs allow users to be in groups, permitting
  group access rights

46
Protection

• File owner/creator should be able to control
• what can be done
• by whom
• Change owner user or group
• chgrp change group associated with file
• chown change owner of file
• Types of access
• Read
• Write
• Execute
• Append
• Delete
• List

47
Access Lists and Groups
chmod change access modes

• chmod 761 prog1.out
• Mode of access read, write, execute, setuid,
  setgid
• Three classes of users
• RWX
• a) owner access 7 ? 1 1 1 RWX
• b) group access 6 ? 1 1 0
• RWX
• c) public access 1 ? 0 0 1

48
Digit Permissions Binary Meaning
0 --- 000 All types of access are denied
1 --x 001 Execute access is allowed only
2 -w- 010 Write access is allowed only
3 -wx 011 Write and execute access are allowed
4 r-- 100 Read access is allowed only
5 r-x 101 Read and execute access are allowed
6 rw- 110 Read and write access are allowed
7 rwx 111 Everything is allowed
49
setuid, setgid access right

• Mode of access read, write, execute, setuid,
  setgid
• setuid, setgid Unix access rights flags that
  allow users to run an executable with permissions
  of the executable's owner or group.
• Used to allow users to run programs with
  temporarily elevated privileges in order to
  perform a specific task.
• When an executable file has been given setuid
  attribute, normal users who have permission to
  execute this file gain the privileges of the user
  who owns the file (commonly root) within the
  created process. When root privileges have been
  gained within the process, the application can
  then perform tasks on the system that regular
  users normally would be restricted from doing.
• E.g. passwd, chsh commands for changing password
  or login shell
• Need to modify system file /etc/passwd
• Another example program you used for submitting
  programs

49
50
File Sharing Remote File Systems

• Network allow file system access between systems
• Manually via FTP
• Automatically, seamlessly using distributed file
  systems
• Semi automatically via world wide web
• Client-server model allows clients to mount
  remote file systems from servers
• Server can serve multiple clients
• Client and user-on-client identification is
  insecure or complicated
• NFS is standard UNIX client-server file sharing
  protocol
• CIFS is standard Windows protocol
• Standard operating system file calls are
  translated into remote calls
• Distributed Information Systems (distributed
  naming services) such as LDAP, DNS, NIS, Active
  Directory implement unified access to information
  needed for remote computing

51
File Sharing Consistency Semantics

• Consistency semantics specify how multiple users
  are to access a shared file simultaneously
• Similar to Ch 7 process synchronization
  algorithms
• Tend to be less complex due to disk I/O and
  network latency (for remote file systems
• Andrew File System (AFS) implemented complex
  remote file sharing semantics
• Unix file system (UFS) implements
• Writes to an open file visible immediately to
  other users of the same open file
• Sharing file pointer to allow multiple users to
  read and write concurrently
• AFS has session semantics
• Writes only visible to sessions starting after
  the file is closed

52
File System Mounting

• A file system must be mounted before it can be
  accessed
• A unmounted file system is mounted at a mount
  point

52
53
Mount Point
53