OpenMP Parallelization Tips

1
OpenMP Parallelization Tips

• Shared Memory Programming Class
• COSC 6397

2
Merging Parallel Regions

• Main.c
• pragma omp parallel default(shared)
• pragma omp single
• getUserParameters()
• genScalData(SDGdata)
• computeGraph(G, SDGdata)
• .

3
Tips

• Use reduction as much as possible
• pragma omp for reduction(numStrWtEdges)
• Add a name to your critical section
• pragma omp critical(permLock)
• Use fine-grained Locks
• An array of locks
• Omp_test_lock()

4
Nowait

• Nowait is your friend! Beware of data dependency!
• Overlap works
• pragma omp single nowait
• pragma omp for

5
Flexible Sync

• Barrier is expensive
• Use Flush to implement post and wait sync.
• pragma omp parallel
• pragma omp single nowait
• dosomething()
• Donetrue
• pragma omp flush(Done)
• while(!Done)
• pragma omp flush(Done)

6
Privatization

• Use threadprivate, private
• Avoid memory contention
• Improve data locality

7
SPMD style

• Use thread ID to distribute work
• pragma omp parallel
• if(myid 0)
• dothis
• else
• doelse

8
Load balance

• Dynamic/Guided schedule
• Nested parallelism
• pragma omp parallel
• nworkload()
• pragma omp parallel num_threads(n)

9
Performance tools

• Use performance tools to gather the detailed
  performance information
• Tau
• Perfsuite
• Papi
• Kojak
• Opari

10
Debugging

• Much harder than sequential
• Switch data view between threads
• Data race is the problem
• Get help from tools
• Set thread number to 1
• Add barriers to suspicious codes
• Change omp for to omp single
• Incremental parallelization
• Relative debugging approach