Still More Multiprogramming Issues

1
Still More Multiprogramming Issues

• File Systems
• ECEN5043 Software Engineering of Multiprogram
  Systems
• University of Colorado

2
Overview

• File Systems
• Multimedia-type Complications
• (multimedia in the sense of gt 1 continuous media)
• Not your fathers text file format
• Process scheduler demands
• Extremely high data rates
• Smooth real-time play
• High storage needs -- (de)compression amount

3
Files -- Why we need them

• Running process can store/retrieve limited info
  within its own (virtual) address space
• When the process terminates, the info is lost
• When a computer crash kills a process, the info
  goes
• Multiple programs/processes need to share info
  simultaneously
• Shared data must persist

away
4
In other words, files

• store a very large amount of information
• allow data to survive the programs termination
• allow concurrent access by multiple programs

5
Analogy

• Client server relationship is the concurrent
  programming analog of the relationship between
  the subroutine caller and the subroutine itself.
• Server is typically accessing a file
• Each client request is an independent unit
• Multiple requests may be handled concurrently

6

• Client/server kinds of interactions occur
• within operating systems
• object-oriented systems
• networks
• databases
• etc.
• Common example There is a file server module
  that provides operations on a file.
• When a client process wants to access the file,
  it calls the operation in the appropriate file
  server module

7

• Shared-memory system
• File server typically implemented by a collection
  of subroutines and data structures that represent
  files (file descriptors)
• Interaction between a client process and a file
  would typically be implemented by subroutine
  calls.
• IF file is shared, important that it be written
  to by at most one client process at a time
• Shared file can safely be read concurrently

8
Same machine or not

• Clients are processes whether on same machine as
  server or not
• Shared-memory server is implemented by
  collection of subroutines
• Subroutines programmed using mutual exclusion and
  condition synchronization to protect critical
  sections and ensure execution in appropriate
  orders
• On distributed-memory environment, server is
  implemented by one or more processes, often a
  multithreaded program, one thread per client.

9
Communication

• Semaphores or monitors
• Message passing, remote procedure calls,
  rendezvous

10
Operating system concerns

• How the files are
• Structured
• Named
• Accessed
• Used
• Protected
• Implemented
• May memory-map file (by pages) -- systems tables
  make the disk file the backing store for a
  designated memory region.

11
Additional file concerns

• Robustness inspection checklist
• Verify appropriate usage of
• Create / Open / Rename
• Seek / read
• Write / Append
• Close / Delete
• Get / Set attributes

12
What are concerns for multi-program systems?

• (reference previous two slides)
• What information would be important to have in a
  requirements spec or a design spec for a program
  that will be part of a multi-program system?

13
Multiprogramming directory surprises

• If execution environment is simple with simple
  directory, care must be taken to ensure unique
  file names, especially when user-selected
• If process changes its working directory to
  simplify file pathname references ... what?
• Lots of variability in system call behavior for
  managing directories -- if called from within
  programs in a multi-program system ... what?

14
(Blurred) Sepa-ration of concerns

• Users (programmers)
• How files are named
• What operations are allowed
• What the directory tree looks like
• Other interface issues

• Implementers (of o.s.)
• How files and directories are stored
• How disk space is managed
• Efficiency and reliability

15
Programmed Aspirin reduces inflammation from
irritation

• When writing software to be portable to different
  kinds of operating environments, how can you
  minimize the pain associated with differences in
  files, directories, etc.?
• Similarly, multiprogram access to shared files?
• If using language support for safety in accessing
  shared files, what is good practice?

16
Follow the yellow brick road ...

• In a simple environment (no or little o.s.)
• Contiguous storage allocation is simple
• need address of the first block and the number of
  blocks in the file
• book presumes disk -- might be an in-memory
  file, might be executable in ROM in embedded
  system, etc.
• fragmentation issue -- compaction not realistic
  option on the fly
• great for CD-ROMs, DVDs, other write-once media
• Linked list allocation is ok with File Allocation
  Table for small files -- not as simple
• Remember text has details re index-nodes method

17
Shared (linked) files

• A multiprogram system design may consider shared
  files
• A shared file appears simultaneously in different
  directories belonging to different users
• Must solve
• If X appends to the file, new blocks must be
  visible to Y
• With links, files have gt1 path name -- same file
  is found via various path walks

18
Fundamental principle of disk space management re
block size
What can you conclude about performance and space
utilization?
19

• Can you apply that to other performance vs.
  management issues?

20
Why worse for multiprogram systems?

• File system reliability -- storage reliability
• backups restores -- will a restored file be
  where all programs expect to find it?
• techniques that minimize backup time tend to make
  recovery more complicated
• file system consistency -- what the text says is
  elementary compared to sophisticated storage
  management systems

21
Cache flow
tag index byte
16
tags tags
Example of 2-way associative cache
main memory
22
Cache flow
6 10 4
011010 1000100101 0100
16
1K
1000100101
011010
6A250
tags tags
Example of 2-way associative cache
main memory
23
Impacts

• If two programs are accessing the same data in
  memory, what impact will the presence of a cache
  have?
• What if you have a late enhancement to create a
  report which includes the execution over a large
  trio of arrays
• SumArray i ArrayOne i ArrayTwo i
• Would you be better off with three files?

24
Layers of complexity

• Typically in microprocessor world
• on-chip cache
• off-chip cache
• main memory
• secondary memory (disk)
• Both caches might work exactly the same -- check
  the caches sequentially.
• Main memory, however, may be managed with a
  paging algorithm ...

25
On the one hand ...

• If any essential byte in the cache is modified,
  it is written to disk immediately
• If any data byte in the cache is modified, it
  should be written almost as quickly
• UNIX -- soon (every 30 seconds)
• MS-DOS -- immediately
• Discuss the impact on performance.
• What are other issues to consider?

26
If caches are like little pages ...

• What would be an equivalent to the Transition
  Lookaside Buffer for a cache?
• Block read ahead
• Assume file is sequential
• Reset the sequential bit if a seek is performed
• These are techniques that could help performance
  improvement if needed in a more primitive
  environment that did not already support this

27
CP/M ran well in 16 KB of RAM ...

• Our purpose is generally not to understand how to
  write operating systems for large environments
• But rather, to understand
• issues addressed by operating systems as examples
  of multiprogram systems
• how solutions lead to new issues
• the simpler solutions in case they are needed in
  simpler environments without an adequate o.s.
• CP/M -- a rich source of simple solutions