Message Passing and MPI - PowerPoint PPT Presentation

About This Presentation

Title:

Message Passing and MPI

Description:

Message Passing and MPI. A SOR Example. for some number of ... Willy Zwaenepoel: missing initialization of a and b. MPI Matrix Multiply (w/o Index Translation) ... – PowerPoint PPT presentation

Number of Views:48

Avg rating:3.0/5.0

Slides: 34

Provided by: CITI54

Learn more at: http://charm.cs.uiuc.edu

Category:

more less

Transcript and Presenter's Notes

Title: Message Passing and MPI

1
Message Passing and MPI
2
A SOR Example

for some number of timesteps/iterations
for (i0 iltn i )
for( j0 jltn, j )
tempij 0.25
( gridi-1j gridi1j
gridij-1 gridij1 )
for( i0 iltn i )
for( j0 jltn j )
gridij tempij

3
A SOR Example OpenMP

for some number of timesteps/iterations
pragma omp parallel for private(j)
for (i0 iltn i )
for( j0 jltn, j )
tempij 0.25
( gridi-1j gridi1j
gridij-1 gridij1 )
pragma omp parallel for private(j)
for( i0 iltn i )
for( j0 jltn j )
gridij tempij

4
Shared Memory
grid
temp
1
1
2
2
3
3
4
4
proc1
proc2
proc3
proc4
5
What does the user have to do?

This is what we said for shared memory
Decide how to decompose the computation into
parallel parts.
Create (and destroy) processes to support that
decomposition (implicitly or explicitly).
Add synchronization to make sure dependences are
covered.
Is the same true for message passing?

6
Message Passing No shared memory!
mem1
mem2
mem3
mem4
proc1
proc2
proc3
proc4
network
7
Data Distribution (only middle processes shown)
grid
grid
2
3
temp
temp
2
3
proc2
proc3
8
Is this going to work?

Same code as we used for shared memory
for( ifrom iltto i)
for( j0 jltn j )
tempij 0.25( gridi-1j
gridi1j
gridij-1 gridij1)

9
Is this going to work?

Same code as we used for shared memory
for( ifrom iltto i )
for( j0 jltn j )
tempij 0.25( gridi-1j
gridi1j
gridij-1 gridij1)
No, we need extra boundary elements for grid.

10
Data Distribution (only middle processes)
grid
grid
2
3
temp
temp
2
3
proc2
proc3
11
Is this going to work?

Same code as we used for shared memory
for( ifrom iltto i)
for( j0 jltn j )
tempij 0.25( gridi-1j
gridi1j
gridij-1 gridij1)

12
Is this going to work?

Same code as we used for shared memory
for( ifrom iltto i)
for( j0 jltn j )
tempij 0.25( gridi-1j
gridi1j
gridij-1 gridij1)
No, on the next iteration we need boundary
elements from our neighbors.

13
Data Communication (only middle processes)
grid
grid
proc2
proc3
14
Is this now going to work?

Same code as we used for shared memory
for( ifrom iltto i )
for( j0 jltn j )
tempij 0.25( gridi-1j
gridi1j
gridij-1 gridij1)

15
Is this now going to work?

Same code as we used for shared memory
for( ifrom iltto i )
for( j0 jltn j )
tempij 0.25( gridi-1j
gridi1j
gridij-1 gridij1)
No, we need to translate the indices.

16
Index Translation

for( i0 iltn/p i)
for( j0 jltn j )
tempij 0.25( gridi-1j
gridi1j
gridij-1 gridij1)
Remember, all variables are local.

17
Index Translation is Optional

Allocate the full arrays on each processor.
Leave indices alone.
Higher memory use.
Sometimes necessary (see later).

18
What does the user need to do?

Divide up his program in parallel parts.
Create and destroy processes to do above.
Partition and distribute the data.
Communicate data at the right time.
(Sometimes) perform index translation.
Still need to do synchronization?
Sometimes, but many times goes hand in hand with
data communication.

19
Message Passing Systems

Provide process creation and destruction.
Provide message passing facilities (send and
receive, in various flavors) to distribute and
communicate data.
Provide additional synchronization facilities.

20
MPI (Message Passing Interface)