Semantics of Minimally Synchronous Parallel ML

1
Semantics of Minimally Synchronous Parallel ML

• Myrto Arapini, Frédéric Loulergue, Frédéric Gava
  and Frédéric Dabrowski
• LACL, Paris, France

2
Outline

• Introduction
• Pure functional minimally synchronous parallel
  programming
• High level semantics
• Distributed evaluation
• Conclusions and future work

3
Introduction
4
Our previous workBulk Synchronous Parallelism
Functional Programming BSML

• Bulk Synchronous Parallelism
• Scalability
• Portability
• Simple cost model
• Functional Programming
• High level features (higher order functions,
  pattern matching, concrete types, etc.)
• Programs proofs
• Safety of the environment

5
What is MSPML ?Why MSPML ?

• Minimally Synchronous Parallel ML
• ? BSML without synchronization barriers
• Comparison of efficiency BSML/MSPML
• Further extensions restricted or impossible in
  BSML (divide-and-conquer, nesting of parallel
  values)
• Basis for an additional level to BSML for Grid
  computing BSML on each nodes (clusters) MSPML
  for coordination

6
Pure Functional Minimally Synchronous Parallel
Programming
7
The MSPML Library

• Parallel ML library for the Objective Caml
  language
• operations on a parallel data structureAbtract
  type ? par
• access to the machine parameters
• p unit ? int
• p() number of processes

8
Creation of Parallel Vectors

• mkpar (int ? ?) ? ? par
• (mkpar f )

9
Pointwise Parallel Application

• apply (? ? ?) par ? ? par ? ? par
• apply

10
Example (1)

• let replicate x mkpar(fun pid-gtx)replicate a
  -gt a par
• (replicate 5)

11
Example (2)

• let parfun f v apply (replicate f) vparfun
  (a-gtb)-gta par-gtb par
• parfun (fun x-gtx1) parallel_vector

f
12
Communication Operation get

• get ? par?int par ?? par

13
Example (3)

• let bcast_direct root parallel_vector if
  not(within_bounds root)then raise Bcast_Error
  else get parallel_vector (replicate root)
• bcast_direct int-gta par-gta par

14
Global Conditional

• if parallel_vector at n then else
• if
• at n then E1 else E2
• E1

n
15
Implementation of MSPML

• MSPML v 0.05
• F. Loulergue
• Library for Ocaml (uses threads and TCP/IP)
• October 2003
• http//mspml.free.fr

16
High level semantics
17
Terms

• Functional semantics can be evaluated
  sequentially or in parallel
• Terms

18
Values and judgements

• Values
• Judgement
• e ? v
• Term  e  evaluates to value  v 

19
Some rules
20
Distributed evaluation
21
Informal presentation (1)

• MSPML programs seen as SPMD programs
• The parallel machine runs p copies of the same
  MSPML program
• Rules for local reduction rule for
  communication
• For example at processor i the expression
• (mkpar f) is evaluated to (f i)

22
Informal presentation (2)

• Proc. 0 1 2

0,v
0,v
empty
0,v
A BIT LATER
Local computation
get v 1
request 0 1
v
Communication environment
23
Informal presentation (3)

• Proc. 0 1 2

0,v 1,w 2,w
0,v
0,v 1,w
empty
0,v
Local computation
Not Ready !!
request 2 0
Communication environment
24
Terms and judgments

• New term request i j the processor asks the
  value stored at processor j during the ith step
• Step each step is ended by a call to get
• Judgment At process i, the term ed with
  communication environment Ec is evaluate to ed
  with new communication environment Ec

25
Local computation rules
26
Communication rule
27
Conclusion and Future Work

• Conclusion
• Minimally Synchronous Parallel MLFunctional
  semantics Deadlock free
• Two semantics
• High level semantics (programmers view)
• Distributed evaluation (implementations view)
• Implementation
• Future Work
• MPI Implementation
• Comparison with BSMLlib
• Extensions parallel composition, etc.