Comparative Study of Parallel Performance Visualization Tools

1
Comparative Study of Parallel Performance
Visualization Tools

• By J. Ramphis Castro
• December 4, 2002

2
Presentation Overview

• Introduction
• Evaluation Criteria
• Aims
• ParaGraph
• VAmpir
• Conclusions

3
Introduction

• The reasons for poor performance of parallel
  message-passing codes can be varied and complex
  and users need to be able to understand and
  correct performance problems.
• Performance tools can help by monitoring a
  program's execution and producing performance
  data that can be analyzed to locate and
  understand areas of poor performance.

4
Introduction (cont)

• There are many performance visualization tools
  available to the parallel programming community.
  The purpose of this review is to analyze the
  distinctive features that separate them from each
  other.

5
Evaluation Criteria

• There must always be some guidelines with which
  to separate and gauge the value of a given piece
  of software. Therefore, we used numerous review
  standards in order to accurately gauge the
  individual softwares value.

6
Usability

• Intuitive easy-to-use interface
• Adequate documentation and support
• On-line help and man pages

7
Portability

• Amount of platforms it can be used on
• Ease of building for a new host
• MPI implementations and languages it could handle

8
Versatility

• Ability to analyze performance data in different
  ways and to display performance information using
  different views
• Ability for new analysis methods and views to be
  added easily

9
Scalability

• Ability to handle large numbers of processes
• Ability large or long-running programs
• Acceptable overhead increases

10
Maturity

• Handles error conditions without crashing
• Few or no penalties by incurring in mistakes
  while using the program
• Level of support
• Size of user base

11
Automated Instrumentation and Monitoring System
(AIMS)

• AIMS is a software toolkit for measurement and
  analysis of Fortran 77 and C message-passing
  programs written using the NX, PVM, or MPI
  communication libraries.

12
Usability

• Software distribution included readme files and
  man pages
• User guide available from main web page
• User interface was functional, not aesthetically
  pleasing

13
(No Transcript)
14
(No Transcript)
15
Portability

• Platforms (HP, SGI, IBM, and SUN)
• University of Tennessee is porting AIMS to DoD
  MSRC
• An ASCI effort at LANL is porting AIMS to ASCI
• Data files are in ASCII and thus portable
• AIMS supporting HPF is being developed jointly
  with the Portland Group
• Developers of AIMS are working on a port for the
  SGI Origin 2000

16
Versatility

• Instrumentation done automatically (xinstrument)
• There are a few display types VK, tally, MK, and
  Xisk. (provide different display choices)

17
Scalability

• PC is the utility name that attempts to remove
  overhead
• No data available on the tools impact on
  overhead

18
Maturity

• From what information we could find about AIMS
  support and development we can conclude that it
  is still a relatively young program and counts on
  minimal developer support
• There a numerous projects to port AIMS to
  different platforms

19
Verdict on AIMS

• Although there are a few research groups trying
  to improve upon its availability to different
  platforms and its robustness, AIMS suffers from
  reliability issues.
• Has good features for all around visualization.

20
ParaGraph

• This tool was created for the purpose of
  providing a detailed, dynamic, graphical
  animation of the behavior of message-passing
  parallel programs and graphical summaries of
  their performance.

21
Usability

• Most features can be accessed through Point and
  Click operations
• Easy manipulation of the trace file in use
• Offers intuitive appeal and aesthetic quality of
  its displays
• Vast amount of documentation available from the
  developers

22
Portability

• Based on X Windows
• Is available for most unix based systems
• Its availability to different platforms rivals
  even the most accomplished of commercial tools

23
Versatility

• Three categories - utilization, communication,
  and task information
• 25 different displays in its current form
• Allows for users to create their own
  visualization windows

24
Scalability

• Due to its association and dependency with PICL,
  its overhead is basically a function of the
  frequency and volume of communication traffic
• Visualization still provides accurate and useful
  information

25
Maturity

• Fairly mature tool
• Minor bugs (features)
• PG has a display limitation of how many
  processors can be displayed at one time (1024)

26
Verdict on PG

• Regardless of its few setbacks and limitations,
  PG is a full featured and powerful visualization
  tool for small to medium sized clusters. Its
  ease of use, display features, developer support,
  open-source nature, and its huge community
  support (which allows for numerous improvements
  over a shorter period of time) it is the best
  visualization tool available freely over the
  internet short of developing one yourself.

27
Visualization and Analysis of MPI Programs
(Vampir)

• VAMPIR is a graphical tool for analyzing the
  performance and message passing characteristics
  of parallel programs that use the MPI and/or
  SHMEM communication libraries. It is the top
  commercial tool available today and is developed
  and supported by Pallas.

28
Usability

• On-the-fly segment zoom
• Built-in snapshot capabilities
• Graph to source code linking
• Shortcuts to most of the commands and displays
• Point and Click interface

29
(No Transcript)
30
(No Transcript)
31
Portability

• Architectures supported is simply impressive
• Supported on T3E, IBM SP2, SGI PowerChallenge,
  and Sun Solaris (just to name a few)
• Only limited by its demand on any specific
  platform or research group

32
Versatility

• The amount of displays available is just
  staggering (if you need it, they probably have it
  or are in the process of making it)

33
Scalability

• Causes the least amount of disturbance on the
  program behavior out of all the tools (its
  effects are more noticeable on large clusters)
• Logistic curve overhead

34
Maturity

• Monthly fee, they can also provide 24/7 customer
  service and support
• Has new releases or patches almost every month
• Robustness is one of its best qualities
  (i.e.-systems without a globally consistent
  clock, VAMPIRtrace automatically corrects clock
  offset and skew so that timestamps in the merged
  trace file are consistent)
• Cannot benefit from their user base efforts to
  improve vampir at a faster rate

35
Verdict on Vampir

• If you have the money required to purchase and
  maintain such a product, vampir is the best
  visualization tool available on the market today.
  It has the most features, easiest and most
  intuitive UI, and the largest support
  infrastructure (funded by its customers).
• Works best for very large cluster, where savings
  in overhead are more noticeable
• Displays contain a lot of information and legends
  are very informative
• Visualization features are easy to use and
  understand

36
Overall Conclusion

• AIMS seemed like a very promising tool, but its
  lack of support, available platforms and features
  is very disappointing
• Vampir has the most features, support and
  platform availability out of all the tools. Its
  display flexibility and the sheer volume of data
  it can display accurately on screen at any given
  time, make it the perfect tool for very large
  cluster of computers.

37
Overall Conclusion (cont)

• ParaGraph is the clear winner. Nothing more can
  be said of a FREE tool that rivals Vampir in
  features, robustness and versatility whose
  open-source nature and design structure allow it
  to be a dynamic and highly adaptive tool for
  todays fast paced research market.