Parallel and Distributed Algorithms Spring 2005

1
Parallel and Distributed Algorithms Spring 2005

• Johnnie W. Baker

2
Presentations

• Professor Johnnie W. Baker
• Instructor
• Will give most presentations
• Guest Lecturers from Parallel Processing Group
• Occasional lecture in areas of expertise
• Occasionally cover classes when I am away
• Hopefully, we will have a grader from this group.

3
Two Primary Textbooks

• Parallel Programming in C with MPI and OpenMP
• Michael Quinn, author
• Published by McGraw Hill in 2004
• Used in both PDC and PDA
• Parallel Computation Models and Methods
• Selim Akl, author
• Prentice Hall, 1997
• Access to an online copy will be provided.
• There will also be some supplementary handouts.

4
Additional References

• There will also be some supplementary handouts
  provided, as needed.
• Another excellent reference (not required) for
  parts of the PDA course is
• Introduction to Parallel Computing (Second
  Edition)
• Authors are Grama, Gupta, Karypis, Kumar
• Addison Wesley, 2003.
• A more advanced reference for some topics in
  Quinn
• A useful online textbook (more applicable to PDC)
  is
• Designing and Building Parallel Programs
• Ian Foster, Addison Wesley, 1995
• The website for this book is http//www-unix.mcs.a
  nl.gov/dbpp/

5
Two Complementary Courses

• Parallel Distributed Computing (Fall)
• Parallel Architectures
• Parallel Languages
• Parallel Programming
• Algorithm Examples for some architectures
• Parallel Distributed Algorithms (Spring)
• Important Models of Computation
• Designing Efficient Algorithms for Various Models
• PDC and PDA can be taken in either order
• More natural for PDC to be taken first
• However, students often take PDA first

6
Limited Overlap in PDC PDA

• Allows PDC and PDA to be taken in either order.
• Performance Evaluation and Limits for Parallel
  Computation
• Some general topics required for both courses.
• More practical coverage needed for programming in
  PDC
• More theoretical considerations in PDA
• MPI Language
• Covered as a programming language in PDC
• Only a subset used in algorithms covered in PDA
• No programming assignments in PDA

7
Prerequisites

• The prerequisite for this course is
• A course in the design and analysis of algorithms
  such as CS 4/56101.
• Or Permission
• Alternately, students who have the following
  course should also have an adequate background
  for this course.
• CS 6/76105 Parallel and Distributed Computing

8
Assignments and Grading

• Homework assignments
• Problems assigned for most chapters
• Probably 5-7 different assignments
• No programming assignments
• Course Grade
• Based on homework, midterm, and final
• Approximate weights (assuming grader)
• Homework 40
• Midterm Exam 30
• Final Exam 30

9
Major Topics Covered in PDA(Not necessarily in
order covered)

• General topics
• Analysis of parallel computation
• Limits for parallel computation
• PRAM model and algorithms
• Algorithms for some important interconnection
  networks
• e.g. linear arrays, 2D mesh, hypercube
• Bus-Based models and typical algorithms
• Task/Channel Model algorithms (using MPI)
• BSP (Bulk Synchronous Model) and algorithms
• KSUs associative model and algorithms

10
Major Topics in Companion Course (PDC)

• Fundamental concepts in parallel computation.
• Synchronous Computation
• SIMD, Vector, Pipeline Computing
• Associative and Multi-Associative Computing
• ASC Language and Programming
• MultiC Language and Programming
• Fortran 90 and HPF Languages
• Asynchronous (MIMD) Shared Memory Computation
• OpenMP language
• Symmetric Multiprocessors or SMPs
• Asynchronous (MIMD) Distributed Memory
  Computation
• Communications
• MPI Language and Programming
• Architectures
• Interconnection Networks (synchronous and
  asynchronous)
• Specific Computer Examples for above computation
  paradigms
• MIMD-SIMD Comparisons in Real-Time Applications