Grid Applications

1
Grid Applications

• Andrew A. Chien
• CSE 225, Winter Quarter 1999

2
Last Time

• Technology Prologue
• Heterogeneous Distributed Computing
• Metacomputing
• Fast Networks, Cheap Computing, Killer Micros
• HPDC and Grids
• Aggregate Performance on Single Apps
• High Performance and Distributed PERFORMANCE
• Course Overview
• Discussion

3
Administrative Announcements

• Email signup list (well set up a reflector)
• Group formation (at end of class)
• Undergraduates -- not at this time
• Graduates -- no P/NP option, all students should
  take for grade (same amount of work for all)
• Handouts, Lecture 1 up on the course web
• Homework 1 assignment out today (teams of 2-3)

4
Outline

• Applications for the Grid
• Distributed Supercomputing
• Realtime Distributed Instrumentation
• Data Intensive Computing
• Tele-immersion
• Performance Requirements
• Grid Services Requirements

5
Distributed Supercomputing

• Excessive computational requirements, met only by
  combining multiple high capacity resources
  (national/world supercomputer)
• Applications exploit capacity, special
  performance character, and significant bandwidth
  hierarchies to deliver performance over wide area
  networks

6
QCRD, 3D-React
230Mflops 115Mflops
LHSF 512MB 400MB Output 3.5 C90 Hours
ASY
250 Mflops - Cray 1280Mflops -- MPP machine
LOG-D 512MB 1.5 C90 Hours

• Ab Initio Quantum Chemical reaction dynamics --
  solving a 6D Schrodinger equation for 3 atoms!!!!
• Exploit platform characteristics, pipeline
  communication (80MB/144secs sustained 4.2MB/s
  33.6 Mbits/sec)
• Wu/Kuppermann, Caltech

7
Other Distributed Supercomputing

• Global Climate Modelling
• Driver is Kyoto Accords (global warming)
• Impetus behind a new DOE Initiative
• SF-Express -- DI Simulation
• Driver is military simulation,exercises, training
• Real markets are any training or large scale
  simulation
• 100,000 entities is modest, a city? SD freeways,
  people, cell phones, etc

8
DS Characteristics

• Just cant wait to run larger
• Science -- competition, knowledge, technology,
  commerce, health, wealth, etc.
• Business -- many of the same reasons, often less
  structured applications (asset pricing, design,
  deployment simulation, markets, data mining,
  etc.)
• How large a factor can this buy you?
• How well does this work with a geometrically
  improving technology base?

9
Real-time Distributed Instrumentation

• Instruments Advanced Light source, Particle
  Accelerator, electron microscope, MRI machine
• Computation for data processing, control, remote
  data storage and processing

10
Problem Classes

• Health care imaging systems
• High cost instrument and infrastructure
• High data rates and data cataloguing and
  diagnosis
• Online desirable/required
• High energy physics experiments
• one of a kind instruments (and long waits)
• high data rates and data cataloging,
  online/offline analysis
• Remote Microscopy control
• Remote control of instrumentation (space, time,
  scale), safety?
• Collaboration

11
Real-time Requirements

• Preprocess the data (must keep up)
• to storage
• Sink the data (must keep up)
• to storage
• Control based on the data
• must accept and analyze the data
• to storage as well
• gt control is the most difficult case
• gt shared control is even harder

12
Electron Microscopy
Video
Remote Computing Resources
Control

• 1.5 MeV facility at Natl Ctr for Electron
  Microscopy
• Images are video rates (resolution depends on
  instrument)
• Not amenable to high compression rates used for
  typical video (lossy) some compression possible
• Remote analysis, focus, calibration, object
  detection, tracking, etc.

13
RTDI Characteristics

• Special (expensive) remote instruments
• High data rates Real-time requirements (BW)
• Closed-loop control Real-time requirements
  (latency)
• Can involve large archival databases on or
  offline
• gt amongst the most challenging Grid applications

14
Data Intensive Computing

• Guest lecture later, Reagan Moore of SDSC

15
Tele-immersion

• Combination of immersive virtual reality over a
  network where any element can be remote
• Avatars, natural/artificial interaction, many
  modes
• Computationally and network intensive

16
(No Transcript)
17
Typical Architecture

• Back-end computation and data server
• Front-end CAVE immersive environment driver
• Funky front-end peripherals

18
Performance Requirements

• VR Data Rates (GB/sec)
• Database (GB/sec) -- arbitrary
• Computation (TeraOps) -- arbitrary
• Multiperspective, adaptive, etc.

19
TI Characteristics

• Effectively infinite computing and bandwidth
  requirements (limited by end-nodes scaleup)
• do it better or richer
• Variety of input/output devices focused on human
  interaction
• Hierarchies of data rates, all high, push toward
  natural interaction?
• Networks increase flexibility of decomposition
  and collaboration/sharing
• Low latency desirable for closed loop control and
  collaborative interaction
• Coupled with any of the other types

20
Grid Services Requirements Perspective

• All require basic support for access, initiation,
  and run
• All require coscheduling
• Some benefit from wealth of network resources
  (faster network, better application)
• Some require predictable network performance(QoS)
• Some require integration of large data
  archives/repositories
• Some require heterogeneous computer types
• Some require integration of weird digital devices
  (instruments, sensors, actuators)

21
Summary

• Many flavors of grid applications
• gt Reflect making digital systems collaborative,
  immersive, with remote reach and integrating
  computation and data within those as an enabler
• All involve networks as an extender of reach and
  interaction
• All involve use of computers as extender of
  capability (remoteness)
• All have scalable requirements to the limits of
  available

22
Readings for Next Time

• Grid Book, Chs 4-6 (Grid Application Examples)
• Cluster Networking Papers
• The Virtual Interface Architecture, IEEE Micro,
  18(2), Mar/April 1998, pp. 66-75
• Efficient Layering for High Speed Communication
  Fast Messages 2.x. Proceedings of the 7th High
  Performance Distributed Computing (HPDC7)
  conference (Chicago, Illinois), July 28-31, 1998.
• Design and Evaluation of an HPVM-based Windows NT
  Supercomputer. Submitted for publication.
• Optional
• Design Challenges for High Performance Network
  Interfaces, IEEE Computer, November 1998
• Virtual Interface Architecture Specification.
  V1.0, http//www.viarch.org