Course Goal

1
Introduction
2
Course Goal

• A hands-on course
• There is a lot of theory that we could go into
• Instead well take a pragmatic approach and
  experience general principles of concurrent
  programming, with a high-performance emphasis
• By the end of the class you should be able to
• Write correct and efficient multi-threaded code
  in C and Java using all the state-of-the-art
  tools and techniques
• Understand how to accelerate code via
  multi-threading
• Have notions of and some experience with
  distributed memory computing

3
Course Website

• Located at
• http//navet.ics.hawaii.edu/casanova/courses/ics4
  32_fall08
• Linked from my homepage
• Accessible from the ICS homepage
• Contains
• All lecture notes (posted at least one week
  before the corresponding lecture)
• No textbook in this course
• Pointers to useful on-line material
• All assignments
• Announcements
• A link to the PDF Syllabus

4
Lectures and Office Hours

• Lectures Tue Thu, 300PM-415PM
• Office Hours Mon/Wed, 3PM-415PM
• POST 310C
• phone [phone]

5
Assignments

• Programming Assignments
• 1 or 2 weeks
• Write code
• Run it
• Sometimes write a brief report on the
  performance/behavior
• a few graphs
• a sentence/paragraph about the graphs when
  required
• Turn in the source code and the report (via
  e-mail)

6
Last Assignment / Project

• The last programming assignment will be longer (3
  weeks)
• We will
• Take a real-world piece of software
• Try to use anything weve learned during the
  class to make it faster
• I will benchmark the turned-in code for all
  students
• I will create an (anonymous) ranking
• Extra-credit will be awarded based on achieved
  speed-up
• Data from last time I did this

7
Exams, Grading

• Exams
• One midterm exam
• One comprehensive final exam
• Grading
• Programming Assignments 60 (not all equal)
• Midterm 15
• Final 25

8
Academic Dishonesty

• The PDF syllabus contains the usual academic
  dishonesty statement
• Programming Assignments
• Some code may be found on the Web
• True for the first question of some classic
  assignment
• Difficult to work with code you havent developed
  yourself!
• So although this kind of cheating makes life
  easier for Question 1, it makes it much harder
  for 2, 3, etc.
• And exam questions will be in part about the
  programming assignments
• Using the Web for references and examples is OK
• I encourage you to do your own research on-line
  regarding concurrent programming and come up with
  questions, suggestions for things we should look
  at, etc.

9

• Questions about the Syllabus?

10
Concurrency

• Definition Execution of multiple tasks at the
  same time
• You are used to writing non-concurrent, or
  sequential, programs
• At any point, you could stop the program and say
  exactly which execution is being executed, what
  the calling sequence is, etc.
• And there is a single answer to the above
• In a concurrent program, you design the program
  in terms of tasks, where each task as a life of
  its own
• Each task has a specific job to do
• Tasks may need to talk to each other
• Tasks are in different regions of the code a the
  same time
• A different way of thinking/programming
• The state of the program has more than one
  dimension

11
Concurrent Programs

• A program consists of multiple files/modules/class
  es/functions

12
Concurrent Programs

• A sequential program does this

13
Concurrent Programs

• A concurrent program does this
• a blue task
• a red task

14
Concurrent Programs

• or this
• a blue task
• a red task
• a green task

15
Concurrent Programs

• Thinking about what a concurrent program does is
  more difficult than for a sequential program
• One must keep a mental picture of what each task
  is doing
• Questions like While task 1 is in function f
  where is task 2? are difficult
• Two executions of the same program may not be
  identical
• Well explain this in more details
• As a result, concurrent programs are
• Sometimes more difficult to design for
  correctness
• Sometimes more difficult to read
• Almost always more difficult to debug
• So, why do we bother at all?

16
Why Concurrency?

• Some programs are naturally concurrent
• e.g., a video game in which I have
  computer-generated autonomous characters
• Each character is a task
• e.g., an ecology simulation
• Each animal is a task
• e.g., a program that plays music and shows some
  cool visualization
• The player is a task, the visualization is a task
• If we had a programming language in which we
  could specify concurrent tasks, then such
  programs would be must simpler to write

17
Concurrent GUI

• A common application of concurrent programming is
  for designing Graphical User Interfaces (GUIs)
• Example
• Say you want to write a program that renders 3-D
  objects
• You have a clickable button to launch the
  rendering
• But rendering takes a long time
• You dont want the GUI to appear frozen while
  rendering
• For instance, you want the Quit, Cancel
  buttons to still work, at the least

18
Concurrent GUI

• One way to avoid the frozen problem is to write
  your code with breaking down a task into
  sub-tasks and interleaving execution of sub-tasks
• Typically, code fragments will be written in
  C/C-like pseudo-code or in Java, without
  declarations, etc.

void render(...) for (step0 steplt100
step) this.doSomeRendering(...) if
(gui.cancelButton.clicked()) break
19
Concurrent GUI

• There are many reasons why you dont want to have
  to do the task interleaving by hand
• Its cumbersome
• What if you want to do 7 tasks?
• Its no always doable
• Perhaps the task is not easily done in multiple
  distinct steps
• Perhaps the task is a call to a library which
  code you cant or dont want to modify
• What if some tasks have some real-time
  requirements?
• e.g., you want to have an animated symbol that
  changes every t milliseconds but other tasks may
  call library functions that take more than t
  milliseconds

20
Concurrent GUI

• Youd rather write your code like this

render.start() while (render.notDone()) if
(gui.cancelButton.clicked())
render.kill() break
sleep(0.001) task render(...) ...
21
Concurrent Tasks Abstraction

• Fortunately, the underlying system can support
  the concurrent tasks abstraction
• the interleaving can be done for you
• and better than anything you could do by hand
• This can be done by
• A special library
• A virtual machine like the JVM
• The Operating System
• A combination of some of the above
• In this class we will
• Understand how to use concurrent tasks
• Discuss how it all works underneath
• Look at several benefits of concurrent tasks

22
Simultaneous tasks?

• Can we really have simultaneous concurrent tasks?
• Well talk about this at length in a future
  lecture, but basically there are two kinds of
  concurrency
• True concurrency two or more things happen at
  the same instant in time
• False concurrency only one thing happens at a
  time, but the illusion of concurrency is given by
  rapid context switching
• context switching the interleaving from a couple
  of slides ago, but done transparently by the O/S

23
True/False Concurrency

• Consider a program that defines two concurrent
  tasks, T1 and T2
• On a single-processor system, only one task can
  use the CPU
• The concurrent tasks use false concurrency
• On a multi-processor system, each task can be on
  a different core
• The concurrent tasks use true concurrency

24
True/False Concurrency
one processor

• False concurrency between the red task, the green
  task, and the blue task

25
True/False Concurrency
one processor
one processor

• True concurrency between the yellow task and the
  green task, the grey and the blue, etc.

26
True/False Concurrency

• The programmer shouldnt have to care/know
  whether concurrency will be true or false
• Typically, the programmer doesnt know on which
  computer the program will run!
• A concurrent program with 10 tasks should work on
  a single-core processor, a quad-core processor, a
  32-core processor, etc.
• However, there is an impact on performance

27
Concurrency and Performance

• True concurrency, whenever possible, leads to
  better performance than false concurrency
• Two processors cores used concurrently
• False concurrency is still useful for creating
  more interactive applications
• And in fact, what at first glance looks like
  false concurrency (e.g., one processor core) can
  in fact still use hardware concurrently
• One processor core used concurrently with the
  disk
• One processor core used concurrently with the
  memory
• One processor core used concurrently with the
  network
• We will talk at length about opportunities for
  concurrent use of hardware resources

28
Example Concurrent App

• Consider an application that reads files that
  contain images and enhances the images
• A sequential execution would look like this

. . .
read img1
enhance img1
read img2
enhance img2
read img3
enhance img3

• While an image is being read, the CPU is mostly
  idle
• Perhaps ok if you run other applications at the
  same time, otherwise its wasteful

29
Example Concurrent App

• Rewrite the application as two concurrent tasks
• Task 1 Reads images
• Task 2 Enhances images
• Now the execution (could) look like this

. . .
enhance img1
enhance img2
enhance img3
. . .
read img1
read img2
read img3

• Essentially, the cost of reading images is hidden
  after the first one

30
Example Concurrent App
. . .
enhance img1
enhance img2
enhance img3
enhance img4
. . .
read img1
read img2
read img3
read img4
read img5
read img6
read img7
read img8

• In our scenario, image reading takes less time
  than image enhancing
• This can lead to a problem a limited number of
  images can be held in memory
• If one tries to keep too many in memory, then the
  application will start swapping things back to
  disk!
• See your virtual memory lectures (ICS431, ICS412)

31
Example Concurrent App

• Solution 1 Read only one image ahead of time
• Requires some synchronization between tasks
• which we will see how to do

. . .
enhance img1
enhance img2
enhance img3
enhance img4
read img1
read img2
read img3
read img4
read img5

• Problem If we have images of different sizes,
  then reading image i1 may take longer than
  processing image i
• i3 above leads to idle time

32
Example Concurrent App

• Solution 2 Read only N image ahead of time
• Making sure that N images always fit in memory

. . .
enhance img1
enhance img2
enhance img3
enhance img4
read img1
read img2
read img3
read img4
read img5

• In the above example, N 3
• If images are very different, it could be
  difficult to determine the smallest N
• Best bet just keep at most X MBytes of image
  data in memory

33
Example Concurrent App

• What if enhancing takes less time than reading?

. . .
enhance img1
enhance img2
enhance img3
. . .
read img1
read img2
read img3

• The cost of enhancing images is hidden after the
  first one
• Good news One doesnt have to know which
  operation takes less time ahead of time
• Difficult to know anyway, depends on the computer
• Lesson Just create concurrent tasks and make
  sure memory doesnt become a problem

34
Example Concurrent App

• The previous application is really a 2-stage
  software pipeline
• Pipelining in hardware for a RISC processor (see
  ICS431)
• Each instruction consists of X one-cycle stages
• Two instructions can be in different stages at
  the same time at no extra cost
• But extra complexity of the CPU design
• Performance is best when all stages take the same
  time
• Similar concept here, but in software
• Application design is more complicated
• Things would be easy if image reading and image
  enhancing were to take the same exact time
• Rarely the case (unlike for a RISC process)
• When stages dont take the same time, we can do
  things ahead of time, e.g., hold up to N images
  in memory
• Not an option in hardware pipelining

35
No Extra Cost??
enhance img2

• In hardware two instructions can be in different
  stages at the same time at no performance cost
• In software its not generally 100 true
• A task that reads data from disk still needs to
  execute some instructions at the CPU
• But they are not very frequent because the task
  spends most of its time waiting for the disk
• Furthermore, running more than one task at a time
  has some overhead
• The interleaving of instruction requires some
  extra work by the CPU, O/S
• So we lose a little bit in general

read img3
enhance img2
read img3
extra
36
Example Concurrent App

• The ideal picture looks like this

. . .
enhance img1
enhance img2
enhance img3
enhance img4
read img1
read img2
read img3
read img4
read img5

• The real picture could look like this

. . .
enhance img1
enhance img2
enhance img3
enhance img4
read img1
read img2
read img3
read img4
read img5
37
Example Concurrent App

• The big question is how much performance benefit
  can we really get from concurrency?
• Its a difficult question because the answer
• depends on the application
• depends on the computer
• depends on the operating system
• depends on the language
• One could rely on benchmarks and other
  application runs to get a reasonable guess
• Fortunately, concurrency is rarely a bad idea as
  long as one knows how to write good concurrent
  code
• Which is what we will learn in this course

38
Other Concurrent Apps

• In many cases concurrency can help get better
  performance by using multiple hardware resources
  at the same time
• Examples
• A concurrent Web browser Network CPU
• A concurrent Web server Network CPU Disk
• An image renderer CPU1 CPU2 CPU3
• All the above applications can also use
  concurrency for interactivity/responsiveness
• Example
• You dont want a Web server to appear frozen to
  Web clients because it is currently serving a
  long request
• Especially true in 3-tier apps like airline
  reservation systems
• Lesson Concurrency is useful in many ways
• In fact, some argue that create a bunch of
  concurrent tasks and throw them in cant be a
  bad idea

39
Why Should You Care?

• Having faster and more responsive apps is good
• The advent of multi-core processors has a
  profound effect on the way software is written
• More opportunities for concurrency for
  performance
• Cannot count on increasing clockrates anymore to
  increase performance like weve been doing in the
  past
• Therefore
• Knowing how to write good concurrent code is
  extremely important for todays computer
  scientist/programmer
• Many companies are engaged in making their code
  concurrent as we speak
• Theyre looking for people to do it

40
In the News and the Industry

• Herb Sutter, A Fundamental Turn Toward
  Concurrency in Software Dr. Dobb's Journal ,
  March 2005
• Video Games that are CPU-bound (not GPU-bound)
• The SourceTM Engine (Half Life) current
  development almost completely focuses on
  multi-threading
• Quote from an interview The short summary is
  that creating efficient and powerful
  multithreaded code is extremely difficult, and
  there's a very real possibility that developers
  will need to throw away a lot of their existing
  code base.
• Harmotion Game Engine wins 1st place in Intels
  multithreading gamecontest
• Photoshop has some multithreading, but could use
  more
• You can Google for variations on the word
  multithreading and youll find many pointers to
  companies working on making their applications
  concurrent

41
Why this course?

• Concurrency is increasingly important in industry
• In 2 years it went from that cool thing that
  some people know how to do to that fundamental
  thing youd better know how to do
• Writing concurrent programs is difficult when one
  has never been exposed to concurrency
• Some people put the shock of going from
  sequential to concurrent in the same category as
• Going from imperative programming to Functional
  or Logic programming
• C to LISP or Prolog
• Going from imperative programming to Object
  Oriented programming
• Going from C to Java
• I am not sure I agree, but its clearly a new way
  of thinking
• After taking this course you should be proficient
  in concurrent programming

42
Concurrency in this Course

• Well look at the three important/difficult
  issues with concurrency
• Correctness
• Responsiveness
• Performance
• Well look at
• Java Threads
• Java the new java.util.concurrent package
• C pthreads
• C OpenMP