CS315B Assignment 2: Application Mapping for Parallelism

1
CS315BAssignment 2Application Mapping for
Parallelism
Bill Dally, Christos Kozyrakis and Kunle
Olukotun Stanford University http//www.stanford.
edu/class/cs315b/
2
The Work

• Pick an important application or algorithm
• One from our list or one you are passionate about
• Develop and analyze parallelization approaches
• Where is the parallelism and what is the nature
  of parallelism
• Data, functional, static, dynamic, etc.
• Where and how much the communication and memory
  access
• What is the computation/communication ratio
• Communication pattern
• Communication/local access ratio
• What is the locality in the application
• How do parallelism, locality, and communication
  interact
• Suggest implications to parallel programming
  languages
• Analyze implications on parallel hardware
• Bring up challenges, questions, experiments,
  alternatives,

3
UCB Dwarfs
4
How to Evaluate an Application

• Consider alternative application mappings
• What works for 10s, 100s, or 1000s of nodes?
• What happens if dataset varies
• Size, characteristics etc
• Consider the parallel language mapping
• What programming primitives are important?
• How well do existing HLL capture application?
• How would you map it to a HL DSL?
• Give grade
• Implications to the runtime system
• What kind of scheduling is needed?
• What kind of locality optimization might work?
• What kind of communication management is needed?

5
How to Evaluate an Application

• Implications to hardware architecture
• How easy is it to partition data
• Shared vs separate address space
• What is the impact of cache coherence on
  programming and performance
• What computation primitives are important
• What communication support is important
• Granularity, topology, bandwidth/latency
  requirements,
• What synchronization support is important
• Locks, barriers, transactions
• Develop an analytical model that captures the
  performance with each parallelization approach
• Parameters dataset size (N), number of
  processors (P), overhead of communication/synchron
  ization,
• Use the model to evaluate what if scenarios
• Processor counts, latencies, bandwidths

6
The Rules

• Work alone
• Prepare a 10 min talk for 5/24 and 5/31
• Strict adherence to time limit
• Prepare a longer talk to make available online
• Include important references and links
• Important note
• Your goal should be to produce new knowledge (new
  alternatives, better analysis, new challenges,
  new implications) not just review the app and how
  people parallelized it so far!!