The MicroGrid: A Scientific Tool for Modeling Grids

1
The MicroGrid A Scientific Tool for Modeling
Grids

• Andrew A. Chien
• SAIC Chair Professor
• Department of Computer Science and Engineering
• University of California, San Diego
• April 30, 2001

2
Outline

• Motivation
• What is a MicroGrid?
• Validating Models
• Status
• Future Work

3
Motivation

• Need tools to study complex dynamic Grid behavior
• complex non-linear dynamic behavior
• Tightly couple communication, computing, and
  storage resources
• Performance, Availability, Failure
• Complementary approaches useful, but insufficient
• MacroGrids
• Limitations of scale and actual configuration
• Major logistical efforts
• Other Simulations
• Network-only (internet/networking)
• Application level (simple resource models)
• Enable design of robust, reliable, good
  performing Grids and Grid applications

4
Grid Application Developer
Cactus
Zeus-MP
Tardis
Netsolve
GTomo
SF-Express
Distributed Viz

• How will my software behave on the projected
  hardware configuration? (performance)
• How will it behave dynamically? (robustness)
• How will it interact with other Grid applications
  an uses of the system?
• How can I make this a robust, stable, reusable
  application?

5
Grid System Software Developer
GrADS
NWS
PPS
Globus
Nimrod
Grid Researchers

• Libraries network, performance instrumentation,
  runtime environment (e.g. Globus)
• Program Preparation System dynamic compilers,
  runtime, etc.
• Do these things work and how well?
• With what applications and what range of
  applications?

6
Grid System Administrator

• What if I change my resource access policies?
• What if I add/take away these resources?
• What if I change the price charged for
  resources?
• What happened to my Grid when it melted down last
  week?

7
MicroGrid Goals

• Runtime environment for GrADS experiments (a la
  MacroGrid)
• Develop technology and tools to support
  specialized Grid communities (a la MacroGrid)
• Realistic modelling of a broad range of Grid
  systems, applications, environments, and dynamic
  behavior
• Execution of real applications (tools and
  middleware)
• Scale to large experiments
• High fidelity simulation, support variety of
  speed fidelity
• Network, compute, memory, disk
• Observable, repeatable behavior

8
Outline

• Motivation
• What is a MicroGrid?
• Validating Models
• Status
• Future Work

9
MicroGrid Modeling
Grid Application
Virtual Grid
MicroGrid Software
LAN Workgroup
Scalable Cluster
Heterogeneous Environment

• A scientific tool for modeling Computational
  Grids
• Run arbitrary Grid applications on any virtual
  Grid resources
• Allow the study of complex dynamic behavior of
  large systems

10
MicroGrid Today

• Processor speed modeling
• Memory size modeling
• Virtualized Resource description (GIS/MDS)
• Network Virtualization
• Online Network Simulation
• gt runs the Globus 1.1.3 software
• gt runs Globus applications on a Linux/Alpha
  testbed

11
Using a MicroGrid

• Find some physical resources
• Configure a Virtual Grid
• Submit a Globus Job to it
• Observe Execution (which occurs in virtual time)
• DeConfigure the Virtual Grid

12
Outline

• Motivation
• What is a MicroGrid?
• Validating Models
• Status
• Future Work

13
MicroGrid Validation

• Simulate an benchmarks and applications
• various Grid systems
• Run simulations on the physical hardware
• Compare to published results

14
Validation on Micro-benchmarks

• Memory Capacity Modeling
• Processor Speed Modeling
• NSE Network Modeling
• Each resource model is validated

15
Validation on NPB Benchmarks

• Comparison to published cluster NPB results
• Set parameters based on known published relative
  resource performance -- processor and network
  performance
• Alpha cluster (Alphas 100Mbit Ethernet) and
  HPVM cluster
• Overall execution time matches within 4

16
NPB over WAN

• vBNS
• A fictional Cluster
• Varying WAN bandwidth

17
NPB over WAN (Cont.)

• No background network traffic
• Performance is insensitive to network bandwidth
• Shows a simulation of hypothetical cluster on WAN

18
Internal Behavior of NPB

• Autopilot tools for Program Tracing (in MicroGrid
  environment)
• Traces from MicroGrid and real Grid
• Match within 5

19
Validation on Large Applications

• Cactus PDE Solver Framework on Alpha cluster
• WaveToy program, various Matrix sizes
• Execution time matches within 7

20
Outline

• Motivation
• What is a MicroGrid?
• Validating Models
• Status
• Future Work

21
MicroGrid Today

• Uses Globus 1.1.3
• Supports Globus 1.1.3 applications and tools
• Incorporates models for
• Processor speed
• Memory capacity
• Virtualized Resource Description (GIS/MDS)
• Network Virtualization
• Online Network Simulation
• Used via standard submission interfaces
• Not yet available for external users, improving
  robustness and adding modules

22
What have we learned?

• Demonstrated accurate simulation of Grid
  environments and applications
• Demonstrated ability to support existing
  applications and tools (critical for significant
  experiments)
• Existing network simulation tools are inadequate
• Existing network traffic models are inadequate
• Deriving network configuration information is
  challenging
• Extrapolation of results is a major challenge due
  to nonlinearity of behavior

23
What have we learned? (cont)

• Theres a LOT more work to be done to support
• large-scale, high speed simulations,
• with flexible choice of resource models,
• simulating a wide range of environments (config,
  background activity, etc.), and
• executing on a wide range of physical hardware
  resources.

24
Milestones

• Year 1
• Develop Initial Version of MicroGrid toolkit
• Empirical study of application behavior based on
  MicroGrid toolkit
• Year 2
• GrADS runtime environment and applications on the
  MicroGrid (in progress)

25
Outline

• Motivation
• What is a MicroGrid?
• Validating Models
• Status
• Future Work

26
Ongoing and Future Activities

• System Development (Better MicroGrid)
• Scalable On-line Network simulation Xin Paff
  Liu
• Variable speed simulation (efficiency) Ranjita
  Bhagwan
• Network Traffic Modeling (background coupled
  load) Xianan Zhang
• Disk Speed Modeling (I/O intensive workloads)
  Huaxia Xia
• Other current activities (Validation, Software)
• Scalapack modeling Match GrADS results
• Cactus modeling Match GrADS results
• Porting to x86 Linux
• Robustify and package for external release

27
Summary

• Demonstrated that MicroGrid approach can produce
  accurate results in modeling
• Grid applications
• Grid infrastructures
• Dynamic behavior
• Working software
• Significant validation
• Micro-benchmarks Full benchmarks Applications
• Need to get MicroGrid software to the next
  level of capability

28
MicroGrid Team

• Dr. Andrew Chien (PI)
• Graduate Students
• Xin Paff Liu, Ranjita Bhagwan, Xianan Zhang,
  Huaxia Xia
• Former
• Dr. Hyo Jung Song (Postdoc)
• Dr. Kenjiro Taura (U Tokyo Professor)
• Dennis Jakobsen (MS)
• For more information see
• http//hipersoft.rice.edu/grads/project/micro.html
  
• http//www-csag.ucsd.edu/

29
(No Transcript)