Instrumentation and Profiling

1
Instrumentation and Profiling

• David Kaeli
• Department of Electrical and Computer Engineering
• Northeastern University
• Boston, MA
• kaeli_at_ece.neu.edu

2
Application Areas

• Trace generation
• Performance profiling
• Dynamic optimization
• Binary translation
• Power/energy management
• Architecture/hardware design studies
• Virtualization
• Software validation and testing
• Information security

3
Definitions

• Instrumentation A hardware or software probe
  that provides access to either machine state or
  program state
• Profiling The ability to capture information
  available through instrumentation
• Tracing One form of profiling that captures
  execution information in the form of a sequential
  record
• Static Techniques applied before a program runs
  (i.e., at compile time)
• Dynamic Techniques applied as the program
  executes (i.e., at run time)

4
Definitions

• Trace-driven simulation
• Using a previously collected trace as input to a
  simulator
• Execution-driven simulation
• Tracing and simulating concurrently
• Tracing Level -
• Source high-level language program
• Object unlinked binary file
• Executable linked, runable, binary file

5
Definitions

• Binary translation translating between two
  binary executable formats
• Binary optimization modifying the binary to
  produce an optimized binary
• Emulation - One software program, executing
  software binary, potentially targeting a
  different ISA

6
Why are Instrumentation and Profiling important
topics?

• Allow for pre-hardware evaluation
• Enable performance tuning after the hardware has
  been delivered
• Enable software to adapt to changes in workload
  usage
• Provide quantitative software testing quality
• Provide for power estimation
• Enable increased security
• Enable next-generation hardware design criteria

7
History of Instrumentation

• IBM Mainframe Instrumentation Systems (1981)
• Instrumentation implemented in customized
  hardware (Logic Support Subsystem)
• Provides performance data directly to the
  customer
• Allowed for load balancing and capacity planning
• Also provided a level of online diagnostic system
• Hang detection
• Recovery
• Intel In-Circuit-Emulators
• Started with Intel 80386 in mid-80s
• Externalized signals to ICE pins
• Allowed for external control of hardware
  functions (e.g., L1 caching)
• Hardware Counters
• VAX 8800, Pentiums, PowerPCs, SPARCs, Alphas
• On most CPUs today

8
History of Instrumentation

SDS Sigma 7 UCLA 1967
9
History of Profiling

• Hardware tracing
• Early memory studies (Belady, Denning)
• VAX-Tracer (Emer and Clark ISCA 1984)
• LLATT (IBM 1985)
• ATUM (Stanford, Anant Agarwal 1989)
• TRAPEDS (Stunkel, Univ. of Illinois 1989)
• Trace/Trap (IBM - Sigmetrics 1989)
• Logic Analyzers
• Used to debug early hardware
• Used to tune programs (very difficult to do)
• Used to obtain traces for architectural studies
• Counter readers
• VAX8800 Histogram Monitors
• DCPI Digital Equipment (Alpha)

10
History of Profiling

1979 Hewlett Packard Model 85 Used for control
and data acquisition
11
Issues with Instrumentation and Profiling

• Overhead what type of perturbation does the
  instrumentation and profiling introduce?
• Accuracy does our profile reflect the actual
  execution?
• Completeness do I capture OS, interrupts,
  libraries??
• Portability is my approach ISA and OS
  independent?
• Input independence does my output change for
  different inputs?
• Capacity how much can I capture?
• Speed do I catch the fastest event of interest?
• Cost is this worth the investment?

12
Today, Profiling, Simulation and Tracing have
become mainstream research areas

• Cost of simulation is too high
• Need for increased performance accuracy
• Need for software security
• Need to model increasingly complex designs
• New workshops and symposia
• IEEE International Symposium on Workload
  Characterization annual conference
• Workshop on Binary Instrumentation and Analysis
  (held with PACT in 2005 and 2006, special issue
  of Dec 2005 ACM SIGARCH News)
• ACM Symposium on Code Generation and Optimization
  (feedback directed optimization) annual
  conference
• ACM Transactions on Architecture and Code
  Optimization high quality ACM journal

13
This class will

• Present a range of technologies that fall under
  the broad definition of profiling and
  instrumentation
• Present some recent implementations that utilize
  these technologies to improve performance,
  scalability, reliability, power and security
• Involve readings from the literature and a lab
  assignment

14
Your grade in this class will be based on

• Your course project 40
• Your paper review 40
• Your participation in class 20
• Questions and discussion are encouraged!