Design of an intelligent MPI execution environments

1
Design of an intelligent MPI execution
environments

• Kyung-Lang Park
• (2003. 3. 19)

2
Contents

• 2nd year project overview
• Intelligent MPI execution environments
• Message handling for high performance MPI
• Future work

3
2nd year project overview
4
Intelligent MPI execution environment

• Design objective
• Auto selection of efficient nodes
• Fast startup
• High Performance
• Requirements
• Co-allocation and Co-scheduling
• Management of computing nodes in whole testbed
• Efficient scheduling algorithm
• Reference Condor-G, Ninf-G, Multi clustered
  PBS/LSF
• Support Globus Tool Kit 3.0

5
Resource management architecture
2
?
RSL specialization
RSL
3
1
4
Application
Information Service
Queries
Info
Ground RSL
mpirun
DUROC
5
globusrun
globusrun
globusrun
Simple ground RSL
6
Local resource managers
GRAM
GRAM
GRAM
rsh
PBS
Condor
Fork/default
fork
7
6
Conventional MPI architecture
MDS
Kernel

User Request (mpirun)
GRAM
PBS mon
PBS mon
PBS Serv
DUROC
Machine file
PBS Serv
GRAM
PBS mon
PBS mon

Kernel
MDS
7
Proposed MPI architecture
MDS
Kernel

User Request (mpirun)
PBS mon
PBS mon
GRAM
PBS Serv
DUROC
CoScheduler
GRAM
PBS Serv
PBS mon
PBS mon

Kernel
MDS
8
Proposed scheduling algorithm

• Not completed yet, but will leverage those three
  idea
• Intelligent topology selection (ongoing paper)
• Point-based scheduling (kohs paper)
• Agent-based monitoring (kohs paper)
• Efficient Linux kernel interaction

9
Need of Information Service

• We need information service and MDS is not
  sufficient
• Need accurate network status
• All nodes should have same data
• How to make information service
• Use MDS (need to create network information
  provider)
• Use old NMC (need to upgrade)
• Request to information service developer

10
Strategy

• Utilize exist components and framework as much as
  possible
• Modify core algorithm

11
GTK 3.0 Support

• GTK 3.0 doesnt support MPI yet
• MPICH 2.0 is coming soon
• Probably it can support GTK 3.0 (or not)
• Alternatives
• Make MPI service in GTK 3.0
• Use old MPICH-G2 with Globus 2.X
• Make interface (wrapper) between old MPICH-G2 and
  Globus 3.0

12
Future work

• Analyze Condor-G and Ninf-G framework
• Complete scheduling architecture and algorithm
• Topology selection will be presented next time.
• Analyze the Globus Toolkit 3.0
• We will make GTK 3.0 seminar on April 2 (Wed.)