OpenMP%20II

1
OpenMP II

• Laxmikant Kale
• CS320
• Spring 2003

2
OpenMP review

• Mechanisms provided by OpenMP Implementations
• Loop Parallelism
• Parallel loops
• Data sharing declarations (shared, private,
  firstprivate,lastprivate)
• Critical sections
• Schedule clause
• Other
• Parallel clause
• Locks and barriers
• Strategies for optimizing performance of
  programs
• Generating more parallelism
• Restructuring loops
• Efficiency
• Privatization
• Memory layout

3
Shared Address Space Model

• All memory is accessible to all processes
• Processes are mapped to processors, typically by
  a symmetric OS
• Coordination among processes
• by sharing variables
• Avoid stepping on toes
• using locks and barriers

4
So, do we understand the prog. Model?

• Consider the following code

a 1 if (b 0) if (z 0) z 1
t1 a 0
b 1 if (a 0) if (z0) z 2
t2 b 0
store a load b load z store z store t store a
store b load a load z store z store t store b
1 3 - - - -
2 4 - - -
4 1 5 7 10 11
2 3 6 8 9 12
1 2 5 6 7 8
3 4 - - -
Expectation (if z, t began as (0,0)) they can
be (0,0) (1,1) or (2,2) but not (1,2), or (2,1).
If each processor allows its instructions to be
out of order (as long as its own results are
consistent), the result can be wrong.
For example the store from processor A may get
delayed.
5
Sequential consistency

• So, we want to state that the implementation
  should disallow such reordering (of one
  processors instructions)
• As seen by other processors
• I.e. it is not enough for processor A to issue
  its operation in order, they must be seen as
  completed by others in the same order
• But we dont want to restrict the freedom of the
  processor any more than really necessary
• Speed will suffer
• Sequential consistency
• A parallel program should behave as if there is
  one processor and one memory (and no cache)
• I.e. the results should be as if the instructions
  were interleaved in some order

6
Sequential consistency

• More precisely
• operational semantics
• behave as if there is a single FIFO queue of
  memory operations coming from all processors (and
  there is no cache)
• Now, the architect must keep this contract in
  mind while building a machine, but the programmer
  has a concrete understanding of what to expect
  from their programs
• and it agrees with their intuitions (for most
  people)..
• The architect is NOT required to build such a
  FIFO queue
• Just make sure the system behaves as if there is
  one.

7
Another example
Proc 1
Proc 2
a1 b 1
while (b0) // wait print a
We should not see a 0 printed, right?
But a and b may be in different memory modules
(or caches) and the change in b may become
visible to the second process before the change
in a
Sequential consistency forces the machine
(designer) to make a visible before b is visible
8
Computing pi (Pthreads) Declarations
/ pgm.c / include ltpthread.hgt include
ltstdlib.hgt include ltstdio.hgt define nThreads
4 define nSamples 1000000 typedef struct
_shared_value pthread_mutex_t lock int
value shared_value shared_value sval
9
Function in each thread
void doWork(void id) size_t tid (size_t)
id int nsucc, ntrials, i ntrials
nSamples/nThreads nsucc 0
srand48((long) tid) for(i0iltntrialsi)
double x drand48() double y
drand48() if((xx yy) lt 1.0)
nsucc pthread_mutex_lock((sval.lock))
sval.value nsucc pthread_mutex_unlock((sval
.lock)) return 0
10
Main function
int main(int argc, char argv) pthread_t
tidsnThreads size_t i double est
pthread_mutex_init((sval.lock), NULL)
sval.value 0 printf("Creating Threads\n")
for(i0iltnThreadsi)
pthread_create(tidsi, NULL, doWork, (void )
i) printf("Created Threads... waiting for
them to complete\n") for(i0iltnThreadsi)
pthread_join(tidsi, NULL) printf("Threads
Completed...\n") est 4.0 ((double)
sval.value / (double) nSamples)
printf("Estimated Value of PI lf\n", est)
exit(0)
Init lock/s
Create threads
Wait for threads to complete
11
Compiling Makefile
Makefile for solaris FLAGS -mt for
Origin2000 FLAGS pgm pgm.c cc -o
pgm (FLAGS) pgm.c -lpthread clean rm
-f pgm .o