Data Parallel Languages

1
Data Parallel Languages

• ASC
• multiC
• Fortran 90 HPF

2
Contents

• ASC Programming Language
• Multi-C Language
• Fortran 90 and High Performance Fortran

3
References for Data Parallel Languages

• Jerry Potter, Associative Computing - A
  Programming Paradigm for Massively Parallel
  Computers, Plenum Publishing Company, 1992
• Jerry Potter, ASC Primer, 42 pages, posted at
  parallel website under downloads at
  http//zserver.cs.kent.edu/PACL/downloads.
• Ian Foster, Designing and Building Parallel
  Programs Concepts and Tools for Parallel
  Software Engineering, Addison Wesley, 1995,
  Online at http//www-unix.mcs.anl.gov/dbpp/text/bo
  ok.html
• The multiC Programming Language, Preliminary
  Documentation, WaveTracer, PUB-00001-001-00.80,
  Jan. 1991.
• The multiC Programming Language, User
  Documentation, WaveTracer, PUB-00001-001-1.02,
  June 1992.

4
The ASC Language

• Michael C. Scherger Johnnie Baker
• Department of Computer Science
• Kent State University
• October 2003

5
Contents

• Data Types and Variables
• Operator Notation
• Control Structures
• Looping
• Accessing Values in Parallel Variables
• Associations and Setscope
• Input and Output
• Performance Monitor
• Subroutines and other topics
• Software Location and Installation
• Basic Program Structure

6
Data Types and Variables

• ASC has eight data types
• int (i.e., integer), real, hex (i.e., base 16),
  oct (i.e., base 8), bin (i.e., binary), card
  (i.e., cardinal), char (i.e., character),
  logical, index.
• Card is used for unsigned integer data.
• Variables can either be scalar or parallel.
• Scalar variables are in the IS. Parallel
  variables are in the cells.
• Parallel variables have the suffix at the
  end of the identifier.
• Can specify the length (in bits) of parallel
  variables. Default word length works fine for
  most programs.

7
Comparison of Logical Parallel and Index Parallel

• A index parallel variable selects a single scalar
  value from a parallel variable.
• A logical parallel variable L is normally used to
  store a logical value resulting from a search
  such as
• L A .eq. B
• ASC implementation simplifies usage by not
  formally distinguishing between the two.
• The correct type, based on usage, should be
  selected to improve understanding.

8
Array Dimensions

• A parallel variable can have up to 3 dimensions
• First dimension is , the parallel dimension
• The array numbering is zero-based, so the
  declaration
• int parallel A,2
• creates the following 1dimensional variables
• A,0, A,1, A,2

9
Mixed Mode Operations

• Mixed mode operations are supported and their
  result has the natural mode. For example, given
  declarations
• int scalar a, b, c
• int parallel p, q, r, t,4
• index parallel x, y
• then
• c a b is a scalar integer
• q a p is a parallel integer variable
• a px is a integer value
• r tx,23p is a parallel integer
  variable
• x p .eq. r is an index parallel
  variable
• More examples are given on page 9-10 of ASC Primer

10
Operator Notation

• Relational Operators
• less than is denoted by .lt. or lt
• equal is denoted .eq. or
• not equal is denoted by .ne. or !
• Logical Operators
• not is denoted by .not. or !
• or is denoted by .or. OR
• and is denoted by .and. or
• Arithmetic Operators
• Addition is denoted by
• Multiplication is denoted by
• Division is denoted by /
• For more information, see page 40 of ASC Primer

11
The IF-THEN-ELSE Control Structure

• Scalar version similar to sequential programming
  languages.
• Either the body of the IF or the body of the ELSE
  is executed, but not both.
• Parallel version normally executes both bodies.
• First finds the responders for the conditional
• If there are any responders, the responding PEs
  execute the body of the IF.
• Next identifies the non-responders for the
  conditional
• If there are non-responders, those PEs executes
  the body of the ELSE.
• The PEs that are idle when the IF statement is
  encountered remain idle during this execution
• This control structure is also a masking
  operation.

12
Parallel IF-THEN-ELSE Example

• if A .eq. 2
• then A 5
• else A 0
• endif

13
IF-THEN-ELSENANY Control Statement

• Similar to the IF-THEN-ELSE, except that only one
  of the two bodies of code is executed.
• First the logical expression following the IF
  is evaluated
• If there is at least one responder, then the
  responding processors execute the body of the IF.
• However, if there were no responders to the
  logical expression, then all processors that were
  initially active at the start of this command
  execute the body of the ELSENANY
• EXAMPLE
• IF A .GE. 2 and A .LT. 4 THEN
• C 1
• ELSENANY
• C 9
• ENDIF
• See page 16-17 of ASC Primer for more information

14
ANY-ELSENANY Control Statement

• If there is at least one responder, then all
  active processors (i.e., ones active at the start
  of this command) execute the statements inside
  the any-block.
• If there are no responders, then all active
  processors execute the statements inside the
  elsenany block
• any can be used alone (without the elsenany)
• Example
• any A .eq. 10
• B 11
• elsenany
• B 100
• endany
• For more information, see pages 17-19 of ASC
  Primer.

15
Looping

• Sequential looping
• Loop-Until
• See example on page 20 of ASC Primer
• Sequential looping is an infrequently used
  operation in ASC
• Two types of parallel looping constructs
• Parallel For-Loop
• Conditional is evaluated only once.
• Example on page 21.
• Parallel While-Loop
• Conditional is evaluated every iteration.
• Example on page 21-22.

16
Parallel For Loop

• For construct
• Often used when a process must be repeated for
  each cell that satisfies a certain condition.
• The index variable is available throughout the
  body of the for statement
• The index value of for is only evalulated
  initially
• Active responders are evaluated one after another
  until all have been processed.

17
For Loop Example

• Example
• sum 0
• for x in A .eq. 2
• sum sum B
• endfor x
• Trace for example

18
While Loop

• Unlike the for statement, this construct
  re-evaluates the logical conditional statement
  prior to each execution of the body of the while.
  
• The bit array resulting from the evaluation of
  the conditional statement is assigned to the
  index parallel variable on each pass.
• The index parallel array is available for use
  within the body each loop and can be changed
  within the body.
• The iteration is terminated when there are no
  responders when the conditional statement is
  tested.
• That is, all zeros in the index parallel
  variable.
• Study example and trace in the ASC Primer
  carefully to make sure you understand the while
  construct.

19
The associate statement

• Creates an association between parallel variables
  and a parallel logical variables.
• There are no structs or classes in ASC.
• See earlier example in sample program.
• Example
• associate tail, head, weight with
  graph
• See page 8 of ASC primer for more information.

20
Get Statement

• Used to retrieve a value from a specific field in
  a parallel variable satisfying a specific
  conditional statement.
• Example
• get x in tail .eq. 1
• valx 0
• endget x
• The index variable x in example will identify one
  active responder (if one exists) that satisfies
  conditional test.
• The index x is available to use within body of
  get.
• Study the trace of this example in on page 24 of
  ASC Primer to make sure its action is clear.

21
Next Statement

• Similar to get statement, except next updates the
  set of responders each time it is called.
• Unlike get, two successive calls to next is
  expected to select two distinct processors (and
  two distinct association records).
• Can be used inside a loop to sequentially process
  each responder.
• See page 22-23 of ASC Primer for more details.

22
Maximum and Minimum Values

• The maxval and minval functions
• maxval returns the maximum value of the specified
  items among the active responders.
• Similarly, minval returns the minimum value.
• Example
• if (tail .neq. 1) then
• k maxval( weight)
• endif
• See trace of example on pg 27 of ASC Primer.
• The maxdex and mindex functions
• Returns the index of one processor where a
  maximum or minimum occurs.
• If maximum/minimum value occurs at more than one
  location, an arbitrary selection is made as to
  which index is returned.

23
Setscope Endsetscope Commands

• Resets the parallel mask register
• setscope jumps out of current mask setting to
  another mask setting.
• One use is to reactivate currently inactive
  processors.
• Also allow an immediate return to a previously
  calculated mask, such as an association.
• Is an unstructured command such as go-to and
  jumps from current environment to a new
  environment.
• Use sparingly
• endsetscope resets mask to preceding setting.
• See example on page 15 of ASC Primer.

24
Parallel read Statement

• Works only with parallel variables.
• Must have an ASSOCIATE statement.
• Input is from interactive user input or from a
  data file identified in the command line.
• Input file must have blank line at end of data
  set to terminate input.
• Format
• read tail, head, weight in graph
• Read (file) Tail, Head, Weight in
  Graph
• See page 12-13 of ASC Primer for example
• Also, see examples from class notes and primer

25
Parallel print Statement

• Must have an associate statement.
• Does not output user specified strings.
• I.e. User text messages.
• Only outputs the values of parallel variables.
• Example
• print tail, head, weight in graph
• Print (file) tail, head, weight in
  graph
• Provides a dump of the entire parallel variable
  graph
• Dump of association results follow tail, head,
  weight
• a b 40
• a c 30
• See page 13 in the ASC primer and examples

26
Input / Output

• The msg command
• Used to display user text messages.
• Used to display values of scalar variables.
• Used to display a dump of the parallel variables.
• The entire parallel variable contents printed
• Status of active responders or association
  variables ignored
• Format msg string list
• Msg The answers are max BBX B
• See Page 13-14 of ASC Primer

27
Scalar variable input

• Static input can be handled in the code.
• Also, define or deflog statements can be used to
  handle static input.
• Dynamic input is currently not supported
  directly, but can be accomplished as follows
• Reserve a parallel variable dummy (of desired
  type) for input.
• Reserve a parallel index variable used.
• Values to be stored in scalar variables are first
  read into dummy using a parallel-read and then
  transferred using get or next to the appropriate
  scalar variable.
• Example
• read dummy in usedx
• get x in used
• scalar-variable dummyx
• endget x
• NOTE Dont need to use associate statement to
  associate dummy with used. Omission causes no
  problems as no check is currently made.

28
Dynamic Storage Allocation

• allocate is used to identify a processor whose
  association record is currently unused.
• Will be used to store a new association record
• Creates a parallel index that points to the
  processor selected
• release is used to de-allocate storage of
  specified records in an association
• Can release multiple records simultaneously.
• Example
• char parallel node, parent
• logical parallel tree
• index parallel x
• associate node, level, parent with
  tree
• ......
• allocate x in tree
• nodex B
• endallocate x
• release parent .eq. A from tree.

29
Performance Monitor

• Keeps track of number of scalar and parallel
  operations.
• It is turned on and off using the perform
  statement
• PERFORM 1
• PERFORM 0
• The number of scalar and parallel operations can
  be printed using the msg command
• MSG Number of parallel and scalar operations
  are PA_PERFORM SC_PERFORM
• The ASC Monitor is important for evaluation and
  comparison of various ASC algorithms and
  software.
• It can also be used to estimate or determine
  running time.
• See Pg 30-31 of ASC Primer for more information

30
Additional Features

• Restricted subroutine capability is currently
  available
• See call and include on pg 25-7 of ASC Primer.
• ASC has a rather simple subroutine capability.
• While not difficult, the subroutine details will
  not be covered in slides.
• Assignment will not require use of subroutines.
• Use of personal pronouns and articles in ASC make
  code easier to read and shorter.
• See page 29 of ASC Primer.
• Again, the details are not covered in slides.

31
Software
Anyprog.asc

• Compiler and Emulator
• DOS/Windows, UNIX (Linux)
• WaveTracer
• Connection Machine
• http//zserver.cs.kent.edu/PACL/downloads.htm
• Use any text editor.
• Careful on moving files between DOS and UNIX!

-e -wt -cm
ASC Compiler
Anyprog.iob
-e -wt -cm
ASC Emulator
Standard I/O
File I/O
32
Software

• Example
• To compile your program
• asc1.exe e shapes.asc
• To execute your program
• asc2.exe e shapes.iob
• asc2.exe e shapes.iob lt shapes.dat
• asc2.exe e shapes.iob lt shapes.dat gt shapes.out

33
Basic Program Structure

• Main program_name
• Constants
• Variables
• Associations
• Body
• End

34
Basic Program Structure

• Example
• Consider an ASC Program that computes the area of
  various simple shapes (circle, rectangle,
  triangle).
• Here is an example shapes.asc
• Here is the data shapes.dat
• Here is the shapes.out
• NOTE Above links are only active during the
  slide show.