CS444/CS544 Operating Systems

1
CS444/CS544Operating Systems

• Processes
• 1/24/2006
• Prof. Searleman
• jets_at_clarkson.edu

2
CS444/CS544 Spring 2006

• Operating System Structures
• Introduction to Processes
• NOTE
• Thursdays class will be held in the ITL
• (Science Center 334)

3
Operating System Structures

• Protected Mode of Execution (user vs kernel)
• System Call (software interrupts)
• Interrupts (hardware)
• Timer
• I/O devices
• Software exceptions
• trap - request for OS service
• faults - software errors that transfer control
  to the OS

4
Synchronization

• When we write a program, we think about adjacent
  instructions happening in order without
  interruption
• Weve seen lots of things that can interrupt the
  execution of a process (timers, I/O request
  completion, etc.)
• Most times this is ok the state of our process
  is restored and the illusion is maintained
• But sometimes it is really important that two
  things happen together with no interruption
• Specifically if two processes are sharing
  resources
• Example two processes updating a shared database
  of account balances one reads balance and adds
  100, one reads balance and removes 100

5
Hardware support for Synchronization

• Need a way to guarantee that a sequence of
  instructions occur at once at least with
  respect to other entities that are accessing the
  same data
• Solution 1 Disable Interrupts
• Until re-enabled, instruction sequence will run
  to completion
• Would you like to allow applications to do this?
• Solution 2 Provide Locks
• Acquire lock, perform sequence, release lock
• Sequence may be interrupted but interruption not
  visible to others because they wait to acquire
  the lock

6
Building Locks

• Acquiring a shared lock is the same problem as
  updating a shared bank balance
• Hardware can provide a grouping of instructions
  that it will guarantee to happen atomically
• Test and set, read/modify/write
• From these build locks, from locks build any
  atomic unit

Is lock free? (yes) Is lock free? (yes) Write
Ive got lock Write Ive got lock Proceed to
access Proceed to access Concurrent access
violating lock!
Read balance (300) Read balance (300) Decrement
100 (200) Increment 100 (400) Write balance
(200) Write balance (400) Withdrawal lost!
7
Overlapping I/O and Computation

• If we want the OS to be able to efficiently keep
  the CPU busy, then I/O devices need to be able to
  operate independently
• Even if CPU can do other work while I/O is
  pending, system is still inefficient if CPU
  constantly needs to check for I/O completion
  (polling)
• Interrupts
• DMA
• Buffering

8
Intel Architectures PIC

• Programmable Interrupt Controller (PIC) is a
  chip that offloads some interrupt processing from
  the main CPU
• Serves a referee to prioritize interrupt signals
  and allows devices to prevent conflicts
• Device interrupts go to the PIC PIC determines
  which device raised the interrupt Sends
  interrupt to the CPU with a value indicating the
  interrupt service routine to invoke
• If multiple interrupts, PIC will buffer them and
  send them one at a time to the CPU
• Treated by the main CPU as a peripheral

9
DMA

• Still if we want to transfer large chunks of
  data, CPU will still need to be very involved
• For each small chunk of data, CPU must write a
  command to the command and address registers and
  transfer data to/from the data register
• Very regular pattern
• DMA or Direct Memory Access automates this
  process and provides even greater overlap of
  computation and I/O
• Tell device controller with DMA Starting memory
  address and length and it will get each piece
  directly from memory as it needs it
• Scatter/gather list dont limit it to single
  start/length

10
Buffering

• Still more can be done to overlap computation and
  I/O
• What if I/O is slow enough and requested
  frequently enough, all processes may be waiting
  for I/O
• I/O bound vs compute bound jobs
• For writes, copy data to a buffer and then allow
  process to continue while data is written from
  buffer to device
• If system crashes?
• For reads, read data ahead in anticipation of
  demand

11
Memory Mapped I/O

• For each device, set aside a range of memory that
  will be mapped to the registers of the device
• The CPU thinks it is reading/writing memory
  locations (same instructions, same addressing
  scheme)
• Without memory mapped I/O, CPU needs a way to
  name each register on each device controller
• Special instructions? Device/register addresses?
• Required knowledge of number and type of devices
  at design time

12
Programmers/users demand functionality

• Operating systems provide commonly needed
  functionality
• Programmers want stable storage, want to be able
  to share contents with other apps gt file system
  with naming scheme shared by all processes
• Programmers dont want to deal with paging their
  own code and data in and out of limited physical
  memory (and want protection/isolation from other
  processes) gt virtual memory
• Programmers want running processes to be able to
  communicate (not complete protection and
  isolation) gt shared memory regions, pipes,
  sockets, events
• Users dont want a single task to be able to
  monopolize the CPU gt preemptive scheduling
• Users want to be able to designate high and low
  priority processes gt priority scheduling
• .

13
Application demands exceed OS functionality?

• Not all applications are happy with the operating
  systems services
• Many things an operating system does, application
  programmers could do on their own if they were
  sufficiently motivated
• Examples
• Databases traditionally ask for a raw disk
  partition and manage it themselves (who needs the
  FS?)
• User-level thread libraries can be more efficient
  than kernel level threads

14
Application Moves Into the OS

• If a computer system is going to be used, for one
  application, can avoid overhead of crossing
  user/kernel protection boundary by putting the
  application in the kernel

15
Driving forces for OS development?

• Many times platform implies operating system
  system hardware usually marketed more than OS
• Choice of OS for the PC platform is not the norm
• Even on PC platform, what drives OS development
• Application mix, stability, politics bigger
  factors than OS features?
• OS features driven by stability and ease of
  porting/writing apps
• All this implies OS you use every day doesnt
  follow the bleeding edge like hardware

16
Programs vs Processes

• A program is passive
• Sequence of commands waiting to be run
• A process is active
• An instance of program being executed
• There may be many processes running the same
  program
• Also called job or task

17
What makes up a process?

• Address space
• Code
• Data
• Stack (nesting of procedure calls made)
• Register values (including the PC)
• Resources allocated to the process
• Memory, open files, network connections

18
Address Space Map
Biggest Virtual Address
Ox0000
19
How is a process represented?

• Usually a process or task object
• Process Control Block
• When not running how does the OS remember
  everything needed to start this job running again
• Registers, Statistics, Working directory, Open
  files, User who owns process, Timers, Parent
  Process and sibling process ids
• In Linux, task_struct defined in
  include/linux/sched.h

20
struct task_struct / these are hardcoded -
don't touch / volatile long state / -1
unrunnable, 0 runnable, gt0 stopped / long
counter long priority unsigned long signal
unsigned long blocked / bitmap of masked
signals / unsigned long flags / per process
flags, defined below / int errno long
debugreg8 / Hardware debugging registers /
struct exec_domain exec_domain / various
fields / struct linux_binfmt binfmt struct
task_struct next_task, prev_task struct
task_struct next_run, prev_run unsigned long
saved_kernel_stack unsigned long
kernel_stack_page int exit_code, exit_signal
/ ??? / unsigned long personality int
dumpable1 int did_exec1 / shouldn't this
be pid_t? / int pid int pgrp int
tty_old_pgrp int session / boolean value for
session group leader / int leader int
groupsNGROUPS / pointers to (original)
parent process, youngest child, younger sibling,
older sibling, respectively. (p-gtfather can
be replaced with p-gtp_pptr-gtpid) / struct
task_struct p_opptr, p_pptr, p_cptr, p_ysptr,
p_osptr struct wait_queue wait_chldexit /
for wait4() / unsigned short
uid,euid,suid,fsuid unsigned short
gid,egid,sgid,fsgid unsigned long timeout,
policy, rt_priority unsigned long
it_real_value, it_prof_value, it_virt_value
unsigned long it_real_incr, it_prof_incr,
it_virt_incr struct timer_list real_timer

long utime, stime, cutime, cstime, start_time
/ mm fault and swap info this can arguably be
seen as either mm-specific or thread-specific /
unsigned long min_flt, maj_flt, nswap,
cmin_flt, cmaj_flt, cnswap int swappable1
unsigned long swap_address unsigned long
old_maj_flt / old value of maj_flt /
unsigned long dec_flt / page fault count of
the last time / unsigned long swap_cnt /
number of pages to swap on next pass / /
limits / struct rlimit rlimRLIM_NLIMITS
unsigned short used_math char comm16 /
file system info / int link_count struct
tty_struct tty / NULL if no tty / / ipc
stuff / struct sem_undo semundo struct
sem_queue semsleeping / ldt for this task -
used by Wine. If NULL, default_ldt is used /
struct desc_struct ldt / tss for this task
/ struct thread_struct tss / filesystem
information / struct fs_struct fs / open
file information / struct files_struct files
/ memory management info / struct mm_struct
mm / signal handlers / struct
signal_struct sig ifdef __SMP__ int
processor int last_processor int
lock_depth / Lock depth. We can context switch
in and out of holding a syscall kernel lock... /
endif
21
Management of PCBs

• PCBs are data structures (just like you are used
  to at user level)
• Space for them may be dynamically allocated as
  needed or perhaps a fixed sized array of PCBs for
  the maximum number of possible processes is
  allocated at init time
• As process is created, a PCB is assigned and
  initialized for it
• Often process id is an offset into an array of
  PCBs
• After process terminates, PCB is freed (sometimes
  kept around for parent to retrieve its exit
  status)

22
Process States

• During their lifetime, processes move between
  various states
• New just created
• Ready waiting for a turn to use the CPU
• Running currently executing on the CPU
• How many processes can be in this state? ?
• Waiting Unable to use the CPU because blocked
  waiting for an event
• Terminated/Zombie Finished executing but state
  maintained until parent process retrieves state

23
State Transitions
Schedule/unschedule
Ready
Terminated
New
Running
Request Resource or Service
Grant Resource
Waiting
24
State Queues

• OSs often maintain a number of queues of
  processes that represent the state of the
  processes
• All the runnable processes are linked together
  into one queue
• All the processes blocked (or perhaps blocked for
  a particular class of event) are linked together
• As a process changes state, it is unlinked from
  one queue and linked into another

25
State Queues
Tail ptr
Head ptr
Ready queue, queues per device, queue of all
processes,
26
Context Switch

• When a process is running, some of its state is
  stored directly in the CPU (register values,
  etc.)
• When the OS stops a process, it must save all of
  this hardware state somewhere (PCB) so that it
  can be restored again
• The act of saving one process hardware state and
  restoring anothers is called a context switch
• 100s or 1000s per second!

27
Context Switch
28
Schedulers

• Technically two kinds of schedulers
• Short-term scheduler (or CPU scheduler)
• Selects which process should be executed next and
  allocates CPU.
• Short-term scheduler is invoked very frequently
  (milliseconds) ? (must be fast).
• Long-term scheduler (or job scheduler)
• Selects which processes should be brought into
  the ready queue.
• Determines the degree of multiprogramming
• In reality this is usually you the human user

29
What kinds of processes are there?

• Compute bound/ IO bound
• Long-running/short-running
• Interactive/batch
• Large/small memory footprint
• Cooperating with other processes?
• How do we get all these different kinds of
  processes?

30
Family Tree

• Age old questions where do new processes come
  from?
• New processes are created when an existing
  process requests it
• Creating process called the parent created
  called the child
• Children of same parent called siblings
• Children often inherit privileges/attributes from
  their parent
• Working directory, Clone of address space

31
pstree

• init--18Xvnc
• -amd
• -atd
• -bdflush
• -crond
• -16deskguide_apple
• -8gconfd-1
• -gedit
• -18gnome-name-serv
• -16gnome-session
• -16gnome-smproxy
• -gnome-terminal--csh---gtop
• -gnome-pty-helpe
• -gnome-terminal--csh--gtop
• -tcsh
• -gnome-pty-helpe
• -gnome-terminal--csh---tcsh---xterm---csh
• -gnome-pty-helpe
• -gpm

init--18Xvnc -sshd--2sshd---csh---mc
-sshd---csh
-sshd---csh--more
-netstat -sshd---csh---pstree
-syslogd -16tasklist_applet
-xemacs -xfs---xfs -xinetd---fam
-xscreensaver---greynetic
-xscreensaver---hopalong -2xscreensaver--
-xscreensaver -xscreensaver---kumppa
-xscreensaver---spotlight
-xscreensaver---spiral -xscreensaver---nerv
erot -xscreensaver---strange
-xscreensaver---flame -xscreensaver---grav
-xscreensaver---lightning
-xscreensaver---penetrate
-xscreensaver---rotzoomer---xscreensaver-ge
-xscreensaver---deluxe -xscreensaver---rd-b
omb -xscreensaver---sonar
-xscreensaver---moire2 -ypbind---ypbind---2
ypbind
init--18Xvnc -16magicdev
-mdrecoveryd -migration_CPU0
-migration_CPU1 -6mingetty
-2nautilus---nautilus---8nautilus
-2nautilus---nautilus---10nautilus
-3nautilus---nautilus---9nautilus
-nautilus---nautilus---7nautilus
-7nautilus-histor -nautilus-mozill---nau
tilus-mozill---4nautilus-mozill
-8nautilus-news -8nautilus-notes
-7nautilus-throbb -ntpd
-13oafd -16panel -portmap
-16rhn-applet -rhn-applet---gnome_segv
-rhnsd -rpc.statd -rpciod
-14sawfish -2sawfish---rep
-scsi_eh_0 -scsi_eh_1 -sendmail
32
UNIX process creation

• Fork() system call
• Creates a new PCB and a new address space
• Initializes the new address space with a copy
  of the parents address space
• Initializes many other resources to copies of the
  parents (e.g. same open files)
• Places new process on the queue of runnable
  processes
• Fork gives two processes exactly alike
• Fork() returns twice to parent and child
• Returns childs process ID to the parent
• Returns 0 to the child

33
Example Code Snippet

• int main (int argc, char argv)
• int childPid
• childPid fork()
• if (childPid 0)
• printf(Child running\n)
• else
• printf(Parent running my child is d\n,
• childPid)

34
Output

• ./tryfork
• Parent running my child is 707
• Child running

35
Experiments Be Careful!

• Try putting an infinite loop in the childs
  portion ( do you return to the command shell?)
  and then looking for it in the ps output
• Try putting an infinite loop in the parents
  portion (do you return to the command shell?)
• Put an infinite loop in both
• try killing the child (look in the ps output for
  the child and the parent)
• Try killing the parent what happens to the
  child?

36
Exec

• How do we get a brand new process not just a copy
  of the parent?
• Exec () system call
• int exec (char prog, char argv)
• Exec
• Stops the current process
• Loads the program, prog, into the address space
• Passes the arguments specified in argv
• Places the PCB back on the ready queue
• Exec takes over the process
• There is no going back to it when it returns
• Try to exec something in your shell (example
  exec ls) when ls is done your shell is gone
  because ls replaced it!
• Note execvp will search users path automatically

37
Better way?

• Dont fork/exec seem a bit wasteful to you
• Why make a full copy of the parents virtual
  memory space if we are just going to overwrite it
  anyway?
• Function vfork creates a new process without
  fully copying the parents address space
• Parent suspended while child temporarily
  borrows its memory and thread of control
  (unlike fork)
• Man vfork for warnings about _exit vs exit
• Experiment Time N iterations of fork or vfork

38
Foreground vs Background

• int main (int argc, char argv)
• while (1)
• int childPid
• char cmdLine readCommandLine()
• if (userChooseExit(cmdLine))
• wait for all background jobs
• childPid fork()
• if (childPid 0)
• setSTDOUT_STDIN_STDERR(cmdLine)
• exec( getCommand(cmdLine))
• else if (runInForeground(cmdLine))
• wait(childPid)
• else
• Record childPid In list of background
  jobs