Work Package 1 Requirements Report

1
Work Package 1 Requirements Report

• This report discusses the requirements for
  automatic performance analysis tools.
• Outline
• Intro.
• Goal of project is to identify requirements for
  automatic performance analysis systems and the
  issues associated with building and applying them
• Context for Requirements Analysis
• Two views of performance
• (3) Forms of output from a performance analysis
  task
• (8) Sources of data
• Why analyze performance?
• What can be automated?
• Non-linear process of performance analysis
• Scenario Factors
• Lists important questions to ask regarding how,
  where, and why tools will be used
• Gives 3 examples
• Summary of Requirements
• High Level Requirements
• Derived Requirements

2
1. Introduction

• Aim of APART
• Promote the efficient use of parallel computers
  by enhancing the support for automatic
  performance analysis provided to application
  developers.
• Requirements analysis will consider these points
  of view
• Programming interface
• Application areas
• Application programmers
• Available performance monitoring support
• Future machine designs
• Approach guided by questions
• What is the nature of performance analysis?
• What is the purpose of performance analysis?
• What could be automated in performance analysis?

3
2. Context for Requirements Analysis

• Peformance analysis involves complex systems.
  Focus on two views of performance to simplify
• Hardware utilization
• Trade off between accuracy and cost-of-monitoring
• Automated systems benefit from ability to manage
  the overall data collection
• Focus on factors users can control
• Relation of performance data to source code
• Need to focus on performance data a user is
  capable of manipulating
• Multiple forms
• Raw hardware utilization for entire program
• Raw hardware data to source code
• Interpreted data summaries, abstract statements
  (ex. false sharing on some variable is occuring
  in some region), possible even make statements
  about algorithm itself
• Data may be relevant to many parallel programming
  paradigms (ex. execution time, single processor
  features) or paradigm specific (ex. MPI time
  spent in message passing calls)
• There are dificulties relating data to source
  code due to increasing comples machine
  architectures, and increasingly abstract
  languages

4
2. Context for Requirements Analysis cont

• Many of the features of performance analysis are
  common, but will have specialized implementations
  for specific languages and machines
• Could use an Object-Oriented representation to
  capture commonality in base classes
• Types
• Statically analyzable data
• Dynamically gathered data from instrumented code
• Data from architectural simulators
• User-supplied data
• Machine parameter data
• History data (above listed data from previous
  revisions in performance tuning process)
• History data from entirely different
  applications and architectures
• Sources
• Previously gathered data
• Monitoring data available from hardware counters,
  instrumented source code, and software libraries
• Invocations of other tools
• A mechanism is needed for determining which data
  is to be gathered/use

5
2. Context for Requirements Analysis cont

• What is the purpose of performance analysis?
• Locate, identify, and quantify major causes of
  performance loss in relation to source code
• Make suggestions as to what steps can be taken to
  resolve causes of performance loss
• Quantify how well the application is executing on
  the hardware
• What could be automated?
• The search for performance properties
• Performance properties must be defined in terms
  of data potentially available
• Process of query formulation and execution
• The gathering of data required for the search
• Process of Performance Analysis
• Accept user input
• Gather static performance related data
• Instrument the application
• Monitor the execution of the program
• Analyze static data, and monitored data
• Make the results available to the user

6
3. Scenario Factors

• Describes a set of factors that characterize a
  scenario in which a performance tool would be
  used
• Aim is to delimit the scope of the analysis
  problem and to extract specific requriments by
  considering scenarios
• A series of questions

7
3. Scenario Factors

• What is the nature of the task?
• Are there any external process within which the
  task is taking place
• What is the nature of the code?
• What source language(s) does the program use?
• What is the nature of the platforms on which the
  application is to be executed?
• What bounds the performance of the application?

• What are the Environmental factors affecting
  performance
• What is the nature of the execution envrionment?
• What performance metrics are important?
• What is the compilation and build process?
• What performance analysis tools are available?

8
3. Scenario Factors

• Examples
• A Traditional HPC Application
• A Message Passing Application
• An Agent-based Application

9
4. Summary of Requirements

• High Level Requirements
• What should a tool provide?
• Derived Requirements
• What is necessary to facilitate the high level
  requirements?
• Extensible data model
• Formal notation for the description of
  performance properties
• Formal language in which to describe the search
  process
• Formal description of interfaces to existing
  tools
• A recommendation system to suggest next steps in
  both the performance analysis and performance
  tuning task

10
A Appendix

• A brief overview of issues relating parallel
  performance analysis to uni-processor performance
  analysis