F1: Simulative Performance Prediction with FASE

1
F1 Simulative Performance Prediction with FASE

• Alan D. George, Ph.D.
• Professor of ECE, University of Florida
• Casey Reardon
• Ph.D. Student, University of Florida

2
Outline

• Project Goals, Motivations, Challenges
• Background and Related Research
• Project Team Members (faculty students)
• Y1 Tasks
• Overview of Y1 Tasks
• Task 1 Extend simulation modeling framework
• Task 2 Validate system models
• Task 3 Evaluate simulative performance
  prediction
• Y1 Milestones, Deliverables, Budget
• Budget min. max. of memberships recommended
• Conclusions, Member Benefits

3
Project Goals, Motivations, Challenges

• Goals
• Develop concepts and first integrated tool for
  simulative performance prediction of complex RC
  systems apps
• Explore design tradeoffs of complex,
  multi-paradigm systems applications (HPC or
  HPEC) via simulation modeling
• Motivations
• Provide an efficient, comprehensive method of
  evaluating and prototyping RC systems
• Facilitate fast system design tradeoffs
• Enable application mapping/decomposition analyses
  without hardware or software implementations
• Challenges
• Design a framework to accurately model a wide
  range of current and future RC systems and
  applications
• Balance speed and fidelity when designing a
  modeling approach

4
Background Related Research

• Prediction of RC system performance has been
  largely analytical to date
• No previous work available on high-fidelity
  simulative performance prediction of
  dual-paradigm systems apps

FASE Process Diagram

• Build upon recent research success at Florida
• FASE Fast and Accurate Simulation Environment
• Two-phase discrete-event simulation design
• Pre-Simulation auto-characterization of apps via
  MPI code parsing
• Simulation trace-driven simulation scalable to
  large systems apps
• Successfully supported DOD NASA projects (HPC
  HPEC, but no RC!)
• Mission-Level Designer (MLD) used as simulation
  modeling tool
• Supports hierarchical C-based model design in
  discrete-event environment

Basis provided by operational modeling and
simulation suite and published journal and
conference papers and theses at UF.
5
Project Team Members

• Faculty
• Dr. Alan D. George
• Professor of ECE, University of Florida
• Students
• Casey Reardon student project leader
• 3rd year doctoral student, University of Florida
• BS in ECE, Duke University, 2004
• UF Presidential Fellow
• Mark Oden
• BS/MS student, University of Florida
• TBD
• Optional third graduate student
• Undergraduate student (volunteer) TBD

6
Overview of Y1 Tasks

• Three primary tasks planned for Y1
• Task 1 Extend simulation modeling framework
• Design/build/test extension to FASE as a
  simulation modeling framework for virtual
  prototyping of complex RC systems apps
• Design/build/test models for several disparate
  classes of RC systems
• of classes cases determined by of
  membership votes
• Task 2 Validate system models
• Design micro-benchmarks, then calibrate system
  models with experimental data from
  micro-benchmarking tests
• Task 3 Evaluate simulative performance
  prediction
• Demonstrate modeling capabilities with key
  applications and systems recommended by task
  sponsors (HPC or HPEC)
• of scenarios experiments determined by of
  membership votes

7
Task 1 Extend Simulation Framework

• Design, build, and test initial modeling
  framework
• Define and model disparate classes of RC systems
• e.g. loosely vs. tightly coupled resources
• Extend FASE to support modeling of dual-paradigm
  systems
• Generalized high-level, black-box models for RC
  devices
• Manual insertion of RC events into trace scripts
• High-fidelity network and transport models
• Explore steps to allow framework to be enhanced
  in future (Y2)
• Automatic characterization of RC events (via
  standard API calls)
• Detailed, specific RC component models

Initial Modeling Design
Black-box Model Concept
8
Task 2 Validate System Models

• Build and conduct micro-benchmark tests on
  various experimental platforms in laboratory
• Use micro-benchmarks to gather data on key system
  behaviors
• e.g. I/O metrics with RC devices/subsystems
• Use experimental data to calibrate/validate
  system models
• Evaluate and compare alternative models for
  device I/O, memory access, etc.
• Develop automated method to determine optimized
  model parameters from experimental data
• Allow easy, systematic calibration of models to
  any system

Model Calibration Results for RC Device I/O
9
Task 3 Evaluate Simulative Prediction

• Perform simulative case studies with key
  applications and systems
• Use sponsor feedback to determine target
  scenarios for virtual case-studies
• Explore, evaluate, and demonstrate performance
  prediction capabilities of extended tool
• Use virtual prototyping to predict effects of
  hardware changes to application performance
• Analyze multiple algorithm decompositions to
  find optimal mapping of applications to systems

10
Y1 Milestones, Deliverables, Budget

• Milestones
• Completion of models for disparate RC systems
  (May 07)
• Validation of systems via micro-benchmarks (July
  07)
• Completion of performance prediction studies (Dec
  07)
• Deliverables
• Midterm and final reports documenting research
  methods, progress, results, and analysis
• Discrete-event model libraries (compatible with
  commercial MLD tool from ML Design Technologies
  Inc.)
• One or two scholarly conference and/or journal
  publications
• Budget
• 2-3 CHREC memberships
• 2 memberships allows baseline completion of all
  three tasks
• 3 memberships allows extended set of system
  classes/cases in Task 1, extended set of mission
  application scenarios in Task 3

11
Conclusions Member Benefits

• Conclusions
• Design, build, and validate first simulative tool
  for performance prediction involving complex RC
  systems and applications
• Discrete-event simulations with black-box RC
  component models
• Emphasize balance of speed and fidelity in
  modeling approach
• RC system and apps modeling provides useful
  performance prediction capabilities for RC w/ HPC
  or HPEC needs
• Observe effects of tuning hardware capabilities
  of RC resources for specific applications
• Analyze alternative algorithm decompositions
  without developing hardware or software
  implementations
• Member Benefits
• Direct influence over selection of target systems
  to be modeled
• Direct influence over selection of scenarios and
  applications featured in simulative prediction
  studies
• Access to research results from system
  simulations, experimental benchmarking, and
  related deliverables