CS 524

1
CS 524 High-Performance Computing

• Outline

2
Description (1)

• Introduction to high-performance computing
  architectures and programming environments
• Performance enhancements for single-processor
  computers
• Data locality, cache optimizations, data
  dependences, loop optimizations
• Parallel computer architectures
• Parallel computing performance metrics
• Message-passing programming using MPI
• Shared-memory programming using OpenMP and
  P-threads
• Dense matrix algorithms
• Sparse linear system algorithms

3
Description (2)

• Emphasis on code optimizations and programming
  issues
• This is not a hardware design course
• Programming in C/C only
• Programming under Unix/Linux environment

4
Goals

• Mastery of fundamental performance issues for
  parallel and serial computing
• Familiarity with shared-memory and message
  passing architectures and models for parallel
  program development
• Hands-on experience in designing and implementing
  efficient algorithms for high-performance
  computers
• Familiarity with the current state-of-art in
  parallel programming environments, portable
  software libraries, and program development

5
Before Taking This Course

• You should be comfortable with
• Data structures and algorithms
• CS213 is a prerequisite
• You should be comfortable with (matrix) algorithm
  descriptions and implementations in a high-level
  programming language
• Computer organization
• CS223 is a prerequisite
• CS423 is NOT a prerequisite
• Programming in C/C
• Majority of work under Unix/Linux environment
• Linear algebra
• Matrix operations, systems of linear equations

6
After Taking this Course

• You should be able to
• understand the basic architectures of single- and
  multi-processor computers
• develop and implement programs for
  distributed-memory and shared-memory parallel
  computers
• evaluate and enhance performance on single- and
  multi-processor computers
• develop programs in MPI and OpenMP

7
Grading

• Points distribution
• Assignments ( 5) 35
• Quizzes ( 5) 10
• Midterm exam 25
• Final exam (comprehensive) 30

8
Policies (1)

• Quizzes
• Quizzes may be announced 1 to 2 days in advance
  or they may be unannounced
• Sharing
• No copying is allowed for assignments.
  Discussions are encouraged however, you must
  submit your own work
• Violators can face mark reduction and/or reported
  to Disciplinary Committee for action
• Plagiarism
• Do NOT pass someone elses work as yours! Write
  in your words and cite the reference. This
  applies to code as well.
• Violators can face mark reduction and/or reported
  to Disciplinary Committee for action

9
Policies (2)

• Submission policy
• Submissions are due at the day and time specified
• Late penalties 1 day 10 2 day late 20
  not accepted after 2 days
• An extension will be granted only if there is a
  need and when requested several days in advance.

10
Summarized Course Contents

• Introduction to single- and multi-processor
  computing models and architectures
• Cache tuning and characterization
• Data locality, data dependences, loop
  optimization
• Parallel computer performance metrics,
  interconnection networks
• Message-passage programming in MPI
• Shared-memory programming in OpenMP and P-Threads
• Parallel dense and sparse matrix algorithms

11
Course Material

• Texts
• Introduction to Parallel Computing Design and
  Analysis of Algorithms, V. Kumar, A. Grama, A.
  Gupta, G. Karypis, Benjamin/Cummins Publishing
  Co., 1994.
• Using MPI Portable Parallel Programming with
  the Message-Passing Interface, 2nd Ed., W. Gropp,
  E. Lusk, A. Skjellum, MIT Press, 1999.
• Parallel Programming in OpenMP, R. Chandra et
  al., Academic Press, 2001.
• Techniques for Optimizing Applications High
  Performance Computing, R.P. Garg and I. Sharapov,
  Sun Microsystems Press, 2002.
• Introduction to Parallel Processing, M.
  Sasikumar, D. Shikhare, P.R. Prakash,
  Prentice-Hall of India, 2000.

12
Course Web Site

• For announcements, lecture slides, handouts,
  assignments, quiz solutions, web resources
• http//suraj.lums.edu.pk/cs524w03/
• The resource page has links to information
  available on the Web. It is basically a meta-list
  for finding further information.

13
Other Stuff

• How to contact me?
• Office hours 2.30 to 4.00 PM TR (office 132)
• E-mail akarim_at_lums.edu.pk
• Stop by my office
• By appointment
• Philosophy
• Knowledge cannot be taught it is learned.
• Be excited. That is the best way to learn. I
  cannot teach everything in class. Develop an
  inquisitive mind, ask questions, and go beyond
  what is required.
• I dont believe in strict grading. But there has
  to be a way of rewarding performance.

14
Reference Books in LUMS Library

• High Performance Cluster Computing Programming
  and Applications, Volume 2, Rajkumar Buyya
  (Editor), 1999.
• Parallel Programming Techniques and Applications
  Using Networked Workstations and Parallel
  Computers, B. Wilkinson and M. Allen, 1999,
  5.3W686P.
• Parallel Computer Architecture A
  Hardware/Software Approach, D.E. Culler et al.,
  1999.
• Computer Organization, Hamacher et al., 2002.
• Computer Organization and Architectural Design
  for Performance, W. Stalling, 2000.
• Using MPI-2 Advanced Features of the Message
  Passing Interface, Gropp et al., 1999.
• How to Build a Beowulf, T.L. Sterling et al.,
  1999.
• Beowulf Cluster Computing With Linux/Windows,
  T.L. Sterling (ed.), 2002, 5.43 B481.
• Parallel I/O for High Performance Computing, J.M.
  May, 2001.