More on PThreads

1
More on PThreads

• Material taken from
• Introduction to Parallel Processing
• By Grama, Gupta, Karypis, Kumar

2
Programming Shared Address Space

• All memory is accessible to all processors
• Process based models assume that memory is
  private and hidden from other processors (high
  overhead to access)
• Lightweight processes (threads) assume all memory
  is global
• Threads support faster memory manipulation
  (preferred model for parallel programming)

3
What are Threads?

• Basic code segment w/o threadsfor(row0 rowltn
  row) for(col0 colltn col) crowcol
  dot_product(get_row(a,
  row),
  get_col(b,col))

4
What are Threads?

• Basic code segment with threadsfor(row0 rowltn
  row) for(col0 colltn col) crowcol
  create_thread(
  dot_product(get_row(a,row),
  get_col(b,col)))

5
What are Threads?

• Notice that the thread model will allow each row
  column product to be computed in parallel if
  multiple processors exist.
• The underlying systems must schedule the tasks on
  multiple processors

6
Logical Memory Model of a Thread

• In order to execute the previous code, each
  thread needs access to matrices a, b, and c.
• How we think of memory is below

7
Logical Memory Model of a Thread

• Actual memory model
• M represents local stack based memory

8
Why use Threads?

• Software Portability
• Can be developed on serial machines and run on
  parallel machines w/o change
• Latency Hiding
• A major overhead is memory, I/O, and
  communication latency
• When one thread is blocked, another can proceed
• While one thread waiting on communication another
  can operate

9
Why use Threads?

• Scheduling and Load Balancing
• Threaded APIs allow the programmer to specify a
  large number of concurrent tasks
• Minimizes idling overhead
• Frees the programmer from explicit scheduling and
  load balancing
• Done by the OS

10
Why use Threads?

• Ease of Programming
• Easier to write than message passing programs
  using tools like MPI and PVM
• Be aware that we may lose some efficiencies that
  we once had
• POSIX threads have become a standard and are in
  wide use

11
Thread Basics Creation and Termination

• Threads are created using the pthread_create
  functionint pthread_create( pthread_t
  thread_handle, const pthread_attr_t
  attribute, void (thread_function)
  (void), void arg)

12
Thread Basics Creation and Termination

• Pthread_create will create a single thread that
  corresponds to the invocation of thread_function
• The attribute argument is usually NULL, but can
  specify things like scheduling type
• A unique id associated with the thread is
  returned to thread_handle

13
Thread Basics Creation and Termination

• The arg field is used to pass parameters to the
  thread
• We must be careful since thread creation can
  preempt its creator on a single processor system
• On successful creation pthread_create will return
  0, otherwise an error code is returned

14
Threaded pi Computation

• We will next look at code to compute pi based on
  random numbers
• Use a unit square and count the number of points
  that fall within the largest circle inscribed on
  the square

15
Threaded pi Computation

• Since the area of a circle ( ) is equal to (
  ), and the area of a square is 1 x 1, the
  fraction of points in the circle should approach
• A simple threaded strategy for generating the
  value of assigns a fixed number of points to
  each thread

16
Threaded pi Computation

• Each thread will generate random points and keeps
  track of the number of points that land in the
  circle.
• Upon termination of all threads, their counts are
  combined to compute the final value.
• We know how to create threads, we just need to
  use another function to implement the final join

17
Threaded pi Computation

• The pthread_join functionint pthread_join
  ( pthread_t thread, void ptr)
• A call to this function waits for the termination
  of the thread whose id is given by the parameter
  thread

18
Threaded pi Computation

• The actual computation proceeds as follows
• Read in desired number of threads
• Read in the desired number of sample points
• Divide points between the threads
• Use an array hits to hold the count of hits
  (points outside the circle) by thread
• Create number of threads and call compute_pi

19
Threaded pi calculation

• http//www.cs.sdstate.edu/hamerg/csc750/threadedp
  i.c

20
Thread Basics Creation and Termination

• rand_r
• Reentrant
• Safe to reenter if disrupted
• 0..RAND_MAX
• rand_no_x (y) are 0..1
• Circle radius is 0.5 (0..0.5)
• False Sharing
• Second version updates shared resource not local
  resource
• 100,000,000 points
• 4 threads 10x longer
• 1 thread about the same

21
Monte Carlo Pi Calculation

• Estimate Pi by throwing darts at a unit square
• Calculate percentage that fall in the unit circle
• Area of square r2 1
• Area of circle quadrant ¼ p r2 p/4
• Randomly throw darts at x,y positions
• If x2 y2 lt 1, then point is inside circle
• Compute ratio
• points inside / points total
• p 4ratio