CPS110 / EE 153: Intro to Operating Systems

1
CPS110 / EE 153 Intro to Operating Systems

• Jeff Chase
• August 25, 2008

2
About the guy I got these slides from (Landon Cox)

• Background
• BS Math/CS Duke, 99
• PhD EECS Michigan, 05
• Research interests
• OS, p2p, economics, security, mobility
• Why am I a professor?
• Research and teaching are a lot of fun
• Explaining things improves my understanding

3
About me (Jeff Chase)

• Background
• BS Math/CS Dartmouth, back in the 1980s sometime
• PhD CS University of Washington (Seattle), 95
• Research interests
• OS, networked systems, Internet service
  infrastructure, utility computing, energy/green,
  cool new stuff
• Why am I a professor?
• Research and teaching are a lot of fun, etc.
• Explaining things improves my understanding
• Office with view of tower

4
Syllabus prerequisites (CPS)

• CPS 100
• Basic data structures
• Allocating memory on the stack versus from the
  heap
• CPS 104
• Basic computer architecture, ISAs
• Registers stack pointer, PC, general-purpose
• Virtual memory translation
• Page tables
• TLB, caching

5
Syllabus lectures and textbook

• Lecture notes on the web (125 pages)
• Exams based on content of lectures
• Textbooks
• Not required
• On-line Saltzer and Kaashoek
• Modern Operating Systems is OK
• Useful Storage, data, and information systems
  (15 on Amazon)

6
Syllabus discussion sections

• Two sections, starting next week
• MW 250- 405
• F 250 405 (sometimes)
• Teaching Assistant
• Amre Shakimov (shan_at_cs.duke.edu)
• Seasoned and energetic
• Undergraduate Teaching Assistant
• Matt Jacobson

7
Syllabus projects

• Where you will learn the most
• 4 projects
• 0 very simple intro to C
• 1 building a user-level thread package
• 2 building a virtual memory manager
• 3 hack into a vulnerable system
• Projects arent long, but are difficult
• Only 100-1,000 lines/code, but many hours
• Everything is in C
• Project 0 has been posted today

8
Syllabus homework problems

• Posted on web by Friday
• Should be done before discussion section
• Not graded, but count toward participation

9
Syllabus project groups

• All projects done in groups of 2 or 3
• Email groups to chase_at_cs.duke.edu
• By Friday (August 29)!
• Group members will rate each other
• Procedure for firing, quitting in syllabus

10
Syllabus project auto-grading

• All projects are auto-graded
• Allows groups to get immediate feedback
• Use submit110 script on cs machines
• One submission/group/day gets feedback
• Cant use to debug your project
• Any group members submission counts

11
More on the auto-grader

• Very limited feedback correct or incorrect
• Doesnt say what is wrong
• Still have to write a test suite (except P0)
• Dont rely on auto-grader feedback alone
• To get more useful feedback
• Come talk to us!
• We will provide many office hours every week
• (double office hours week before a deadline)

12
Syllabus project timelines

• Due at 6pm, accepted until 115959pm
• Auto-grader clock is the one that counts
• Last submission to auto-grader is final
• 3 late days/group/semester
• Intended for unexpected problems
• No extensions
• Start early!

13
Syllabus project collaboration

• Ok, among groups
• C syntax, course concepts
• What does this part of the handout mean?
• Not ok, among groups
• Design/writing of anothers program
• Includes prior class solutions
• How do I do this part of the handout?
• We use automated similarity-detection software
• Just changing the variable names wont save you
• If in doubt, ask me

14
Syllabus grades, exams

• Projects 35
• Midterm 30
• mid-October
• Final 30
• December
• Participation 5

15
Projects and exams

• The two are not independent
• Familiarity with projects is critical to doing
  well on exams
• I like to ask questions about projects on exams
• Extend Project X to include this functionality
• Know your project!
• You can assign roles to different people
• But each member must understand all aspects

16
Syllabus environment

• Linux/GNU environment
• Need to sign-up for term CS account
• Use the form on the cs.duke.edu/csl page
• Send CS login name to chase_at_cs.duke.edu
• Can login into linux.cs.duke.edu
• Use this account for all auto-grading

17
Syllabus getting help

• Newsgroup
• http//courses.duke.edu
• Office hours
• With me
• With Amre
• Dont email Amre or me directly
• Post to the newsgroup, which we monitor

18
Grades from last semester
Some kind of A
Some kind of B
sko D
19
Questions about the mechanics?
20
Goals for CPS 110

• First part demystify the operating system
• How does my computer start running?
• How does a program load into memory?
• Second part demystify Internet systems
• How does my email know where to go?
• Why is Google so fast?
• How is everything virtualized?

21
Duke course view of computers
Applications Ideas ? high-level programming
languages
CPS 1,6,100,108
compiling, reading programs off disk, getting
program into memory, reading keyboard, starting
the computer, saving files, filenames, networking
Whats missing?
CPS 110
Hardware Assembly language program ? gates
CPS 104
22
Thinking about interfaces

• Consider the Java language and its key word
  interface
• What is a Java object?
• List of methods and collection of internal state
• What is a Java interface?
• Set of methods associated with an object that a
  programmer can call
• What do those methods do?
• Invoke code (let the object do work on the
  callers behalf)
• Mutate the objects public/private state
• Why are interfaces useful?
• They provide an abstraction or simplification
• Callers dont have to know an object exact type

23
OS terminology

• Key terms interface, resource (cpu, mem, etc),
  abstraction, virtual
• Define an interface in terms of resources
• An interface is a set of primitives or operations
• Interfaces provides access to resources
• What do we mean by abstraction?
• How resources are presented to a client
• Can think of as an illusion that makes resources
  easier to program
• What does it mean to virtualize something?
• Provides an abstraction (simple way to manipulate
  resources)
• (mostly) disallow direct access to
  reality/resources

24
What is an operating system?

• Program that runs on CPU, (mostly) like any other
• Virtual interface should be simpler than physical

25
What is an operating system?
Applications
What interface does the OS present?
Virtual machine Interface
OS
Physical machine Interface
Hardware
What interface does the hardware present?
Instruction set Load/store, mem, regs
26
Hardware-software stack
Applications
OS
Hardware
27
OS vs user-level programs
OS runs first, calls program Programs run until
they return control to OS (by themselves or
forced by hardware) Then OS calls another program
How do programs start? Tasks outside program?
(net recv) How to prevent CPU hogging?
Key question who calls whom?
28
Functions of the OS

• Illusionist
• Makes computer seem nicer than it really is
• Examples?
• Programs seem to have their own CPU
• AFS single, unified file system
• Name data with human-readable names
• Directories
• Packets get lost OS makes net look reliable
• Disk is slow OS makes it look fast via caching

29
Functions of the OS

• Illusionist
• Makes computer seem nicer than it really is
• Government
• Divides hardware resources among competing
  programs
• What hardware resources does the OS manage?
• Processor
• Memory
• Network
• Disk

30
Functions of the OS

• Illusionist
• Makes computer seem nicer than it really is
• Government
• Divides hardware resources among competing
  programs
• Taxes programs (OS needs CPU, memory to run)
• Taken for granted when it works, cursed when it
  breaks

31
Why study operating systems?

• Very few of you will ever write one
• Illusionist, govn functions appear in many
  domains
• Google provides the illusion of a single web
  server
• Word does background spell checking
• Design principles
• Proper abstractions, caching, indirection
• Concurrency, naming, atomicity, authentication
• Protection, resource multiplexing (fairness)
• How does OS create the illusions we know/love?

32
Hints for designing systems

• What is a system?
• Components, interconnections
• Interfaces, environment
• Systems do something for their environs
• Exhibit this behavior via interface
• Cleanly divides the world in two
• Parts of the system the environment

33
Systems from 10,000 feet
Component
Component
Component
Component
System
Environment aka the client
34
Why is designing systems hard?

• Emergent properties
• Cant predict all component interactions
• Millennium bridge
• Synchronized stepping leads to swaying
• Swaying leads to more forceful synchronized
  stepping
• Leads to more swaying
• Propagation of effects
• Incommensurate scaling
• Trade-offs

35
Why is designing systems hard?

• Emergent properties
• Propagation of effects
• Want a better ride so increase the tire size
• Need a larger trunk for the larger spare
• Need to move the back seat forward
• Need to make front seats thinner
• Leads to worse driver comfort than before
• Incommensurate scaling
• Trade-offs

36
Why is designing systems hard?

• Emergent properties
• Propagation of effects
• Incommensurate scaling
• Consider the giant mouse
• Weight size3 (volume)
• Bone strength size2 (cross section area)
• An elephant sized mouse is not sustainable
• Trade-offs

37
Why is designing systems hard?

• Emergent properties
• Propagation of effects
• Incommensurate scaling
• Trade-offs
• Waterbed effect
• Push on one end, and the other goes up
• Spam filters and smoke detectors
• False positives vs false negatives

38
Why is designing systems hard?

• Emergent properties
• Propagation of effects
• Incommensurate scaling
• Trade-offs
• In the immortal words of HT Kung
• Systems hard. Must work harder.

39
History of operating systems

• History dominated by two trends
• Increasingly inexpensive hardware
• Increased software complexity
• Microsoft embodies tension between these trends
• MS gained 90 market share by running on cheap hw
• Supporting all that hardware complicates the OS
• (3rd-party drivers responsible for vast majority
  of crashes)
• How is Apples strategy different?
• Jobs chooses the hardware you will run
• HW-to-app control reduces complexity, choice,
  discount

40
First phase single operator

• One goal make it work
• Interactive (user has entire machine to herself)
• Users sign up, get room for two hours at a time
• OS is really just a library compiled into
  program

What is wrong with this timeline? CPU utilization
is awful Since CPUs were expensive, this mattered
41
Second phase batch processing

• Goal improve CPU, I/O utilization
• Machine is no longer interactive
• Users submit program (stack of cards) to queue
• One job at a time, CPU idle during I/O, I/O idle
  during CPU
• OS is a batch monitor library of services
• Loads program, runs program, prints results
• Loads next program

42
Second phase batch processing

• Goal improve CPU, I/O utilization
• Machine is no longer interactive
• Users submit program (stack of cards) to queue
• One job at a time, CPU idle during I/O, I/O idle
  during CPU
• What key OS function starts to matter now?
• Protection programs must not corrupt monitor
• Programs must relinquish CPU to monitor

43
Second phase batch processing

• Goal improve CPU, I/O utilization
• Machine is no longer interactive
• Users submit program (stack of cards) to queue
• One job at a time, CPU idle during I/O, I/O idle
  during CPU
• Why wasnt protection an issue before?
• No batch monitor to corrupt
• Person in lab coat took CPU back from program

44
Third phase multi-program batch

• Goal overlap CPU, I/O
• When one job is reading from disk, run another
  job on CPU
• Use DMA interrupts to allow background I/O
• DMA devices write to program memory
• Interrupts devices can tell CPU the I/O is done

Job 1
Job 2
45
Third phase multi-program batch

• Goal overlap CPU, I/O
• What are the OSs new responsibilities?
• Switch between processes
• Manage multiple I/Os across devices
• Protect processes from each other

Job 1
Job 2
46
Fourth phase time-sharing

• Goal keep efficiency, restore interactivity
• Key insight humans are really just slow I/O
  devices
• Switch between programs during think-time

Job 1

• Increased complexity
• Many jobs
• Outstanding reqs
• Many job sources

Job 2
Job 3
47
Fifth phase personal computing

• What are PC operating systems most like?
• As PC prices dropped, single-operator became
  feasible
• OS was again just a library of services (MS-DOS)
• With one user, do jobs need to time-share?
• Early PC OSes could only do one thing at a time
• Everything waited while printing/loading a
  program (Mac lt X)
• Need protection if Im the only one using the PC?
• Protect me from myself (or my buggy software)
• Early PCs provided no protection
• (why Windows before XP, Mac before X were awful)
• PC operating systems are basically time-sharing
  OSes now

48
Operating system complexity

• Windows XP
• gt 40 million lines of code
• Most of this code is device drivers (not written
  by MS)
• Windows NT took 7 years to develop
• Only worked well years after it shipped
• Windows 2000
• Shipped with 63,000 potential known defects
• Hot research area
• Simplify, automatically find OS bugs