Intelligent Topology Selection for MPI

1
Intelligent TopologySelection for MPI

• Kyung-Lang Park
• (2003.4.9)

2
Contents

• Introduction
• Background
• Topology selection
• Burden process based topology selection
• Application specific topology selection
• Conclusion

3
Introduction

• The Grid is a emerging technology
• MPI becomes more significant
• Grid-enabled MPI has a lot of new features
• Multi-domain, geometrically distributed
• Performance remain difficult
• New concept is needed
• Process topology

4
Definition

• Topology selection is a kind of resource
  selection procedure
• Resource selection
• Cluster selection (global scheduler)
• Node selection (local jobmanager)
• Topology selection is a combination method of
  cluster and node selection, whose operation is
  adaptive to applications

5
What is topology selection?
Cluster A
Cluster B
200 ms
100 ms latency
300 ms
The Change ofExecution Environment
.
Cluster C
6
What is topology selection?
Cluster B
Cluster A
Cluster B
100 ms
200 ms
100 ms
300 ms
Cluster C
Cluster C
Cluster C
There are many choices. And, application
performance must be different each other.
7
How to design topology selection

• Design objective High performance
• Basic concept
• Use of minimum number of clusters
• Give priority to the cluster which has the lowest
  average value of latencies to other clusters
• Additional methodology
• Burden process based topology selection (BPTC)
• Burden channel based topology selection (BCTC)

8
Burden process based TS

• Idea
• Burden process should be placed on the
  cluster/node which provide good network
  performance
• How to find out burden process?
• How to provide network status?

9
Burden channel based TS

• Idea
• Burden channel should be placed into inside of
  clusters (two processes which have heavy
  communication should be placed into the same
  cluster)
• How to find out burden channel?
• How to provide network status?

10
Conclusion

• MPI is getting more popular, but performance and
  easy-of-use still remain difficult points
• Scheduling or resource selection become important
  
• Topology selection, which is a kind of resource
  selection, can provide better performance
• (Need implementation and experiments)