Bulk Synchronous Parallel Computing

1
Bulk Synchronous Parallel Computing

• Trevor Schaub
• Jim Sellers

This presentation was prepared for Professor
Stefan Dobrev in partial fulfillment of the
requirements for the course CSI 4140
2
Outline

• Model description
• Why it is interesting/important
• Relationship to real life
• Some basic techniques, simple sample program to
  demonstrate how the approach to algorithm design
  differs from PRAM and MPP

3
Why do we need change?

• Diverse parallel computers and models
• Shared mem. vs. message passing, no clear winner
• Use network locality vs. dont
• Conclusion parallel code is often computer
  specific. Need unifying model

4
Why BSP?

• Provides consistent, general framework to develop
  scaleable software
• Similar to OO a way of thinking and writing
  programs (paradigm)
• Not system specific
• Proceed in supersteps
• Decouples communication and synchronization
• The deadlock problem is minimized

5
What is BSP?

• BSP is a process of dividing a task into steps
  called supersteps
• Each process performs the task by doing the
  supersteps in order
• The fundamental concept is the superstep itself,
  and its structure

6
Whats a superstep?

1. Processes perform as much as possible using local
   data
2. Communication and synchronization
3. Move on to next superstep

superstep

7
Dissection of a superstep
Threads/Processes
T I M E
Local computation
Global Communication
Synchronization
8
Model description

• What are the benefits of a superstep?
• Easier debugging
• Can look at supersteps in isolation
• Removes problem of deadlock
• Separate computation from communication
• Allows reasoning of the correctness of the code
  nearly as easy as sequential code.
• Easier shift for programmers used to sequential
  programming

9
Relationship to real life

• Is BSP in use today?
• Yes, sometimes without even realizing it.
• When should you use BSP?
• Becomes more useful with more communication
• Benefits vary with the problem

10
Basic techniques

• Identify the sections that can be that need
  communication or synchronization
• Modify program so that sync. happens after
  communication
• Separate ones are grouped where possible
• Before each communication group there is a global
  barrier
• The code between the end of each communication
  group becomes a superstep

11
Example

• See code example

12
References

• http//approximity.com/papers/ptfopt/node31.html
• http//www.cs.ucf.edu/csdept/faculty/goudreau/cop5
  937_fall96/COP5937.html
• http//web.comlab.ox.ac.uk/oucl/research/highlight
  s/bsp_computing.html
• http//www.math.uu.nl/people/bisselin/pas1_prev.ht
  ml
• http//www.math.ruu.nl/people/bisselin/software.ht
  ml
• http//web.comlab.ox.ac.uk/oucl/work/bill.mccoll/o
  parl.html
• http//www.cs.hmc.edu/courses/2001/spring/cs156/ht
  ml12/slides12.pdf
• http//www.cag.lcs.mit.edu/bayanihan/papers/javapd
  c99/html/node2.html
• http//www.byte.com/art/9611/sec5/art5.htm

13
(No Transcript)