pSeries

1
Linda Parallel Model
Carlos P. Sosa Chemistry and Life Sciences
Solutions Development IBM High-Performance
Computing
pSeries
2
Outline

• Parallel Programs
• Linda Model
• Tuple Space
• Tuples
• Operations
• Template
• Examples
• Network Linda

3
Parallel Programs

• Programs run faster by dividing the work among
  different processors. Time to solution!
• Blocks of work have to be independent from each
  other. Simultaneous execution.
• Parallel programming model

4
Parallelizing Applications

• Developing and debugging sequential program
• Transforming the sequential program into a
  parallel program
• Optimizing the parallel program

5
Sequential to Parallel

• Where does parallelism come from?
• Is there any extra cost associated with
  parallelism?

6
Parallel Overhead

• Parallel programs require an additional effort
• of constructing and coordinating the separate
• parts

Overhead is dominated by the communication
between discrete program processes
7
Definition
Granularity ratio of computation to
communication
Coarse-grained parallelism Fine-grained
parallelism
8
Parallel Programming Models

• Message Passing
• Distributed Data Structures

9
Message Passing Matrix Multiplication
receive
send
Pi-1
Pi
Pi1
send
Ai-1
Ai
Ai1
Bj
Bj-1
Bj1
Ci-1
Ci
Ci1
receive
receive
send
10
Distributed Data Structures Matrix Multiplication
A
Shared Data Space
Tasks
C
B
Workers
11
Linda Model

• Linda calls the shared data space tuple space
• C-Linda is an implementation of the Linda model
  using C programming language
• Fortran-Linda is an implementation of the Linda
  model using Fortran programming language

12
Tuple Space
Linda programmers dont need to worry about

• How tuple space is set up
• Where is physically located
• How data moves between it and running processes

13
Tuples
Data moves to and from tuple space as tuples
Tuples data structures of tuple space
C Form (Simple, 1)
Fortran Form (Simple, 1)

• Sequence up to 16 typed fields
• Comma-seoarated list of items
• Enclosed in parentheses

14
Linda Operations
15
Examples
Fortran Form out(cube, 4,64)
C Form out(cube, 4,64)
Data tuples (passive tuples)
16
eval

• Creates a process tuple consisting of the fields
  specified as its argument and then returns
• Implicitly creates a process to evaluate each
  argument

Fortran Form eval(test, i,f(i))
C Form eval(test, i,f(i))
Live tuples
17
Eval Initiates Worker Processes
C Form for (i0, iltNWORKERS, i)
eval(worker, worker())
Fortran Form do 5 I 1, NWORKERS eval
(worker, worker()) 5 continue
18
rd in
in
rd
Access data in tuple space
rd reads a tuple from tuple space in removes
a tuple from tuple space
19
Template
rd in take a template as an argument
Template specifies what sort of tuple to retrieve
C Form (Simple, 1)
Fortran Form (Simple, 1)
20
Matching Templates
A template matches a tuple when

• They both have the same number of fields
• The types, values, and lengths of all actuals in
  the template are the same as those of the
  corresponding fields in the tuple
• The types and lengths of all formats in the
  template match the types and lengths of the
  corresponding fields in the tuple

21
Examples
C Form (cube, 8,512)
Fortran Form (cube, 8,512)
Tuples
rd(cube, 8,?i)
rd(cube, 8,?i)
Templates
in(cube, j,?i)
in(cube, j,?i)
22
Hello World! (sequential)
Main () Printf(Hello, world\n)
23
Hello World! (parallel)
real_main(argc,argv) int argc char
argv int nworker, j, hello() nworkeratoi(a
rgv1) for (j0 jltnworker j)
eval(worker,hello(j))) for (j0 jltnworker
j) in (done) printf(hello_world is
finished\n) return(0)
24
hello(i)
hello(i) int I printf(Hello, world from
number d\n,i) out(done) return(0)
25
Network Linda

• Process scheduling
• Executable location
• Heterogeneity

Flexible utility for executing Linda Programs on
networks
ntsnet
Network Linda runs the real_main process on the
local system, and evaled worker processes run on
remote hosts
26
Simplest Case

• Current directory is a commonly mounted directory
  (NFS mounted)
• Current working directory is accessible by the
  same pathname from every host that the binary is
  going to be executed

27
Steps to Run Network Linda

• Make sure that the bin subdirectory of the Linda
  distribution tree is in the search path
• Define the set of hosts to be used for program
  execution by creating the file .tsnet.config in
  your home directory
• Compile the program clc o hello_world
  hello_world.cl
• Run the program ntsnet hello_world 4

28
.tsnet.config File
Tsnet.Appl.nodelist masternode001 node001 node002
29
What ntsnet Does

• Parsing the command line options and
  configuration file entries
• Querying remote systems for load averages
• Locating local executable files
• Determining what set of nodes to run on, based on
  its scheduling algorithm
• Determining working directories and executable
  file locations on remote nodes, using map
  translation and the associated configuration
  files
• Copying executable files to remote nodes (if
  necessary)
• Initiating remote processes via rsh ( or ssh )
• Waiting for normal or abnormal termination
  conditions during program execution
• Shutting down all remote processes at program
  termination
• Removing executables from remote systems (if
  applicable)

30
Using ntsnet
ntsnet options executable arguments
ntsnets options
Command line arguments for the specified network
program
Binary file to run on local node
31
options
32
Examples
1. ntsnet maxprocspernode 3 hello_world
2. ntsnet nodelist masternode001 node001
node002 hello_world 3
3. Tsnet.Appl.nodelist masternode001 node001
node002
4. Tsnet.Appl.nodelist _at_nodelist node008