Dynamic Grid Computing: The Cactus Worm

1
Dynamic Grid ComputingThe Cactus Worm

• The Egrid Collaboration
• Represented by Ed Seidel
• Albert Einstein Institute
• Egrid_at_egrid.org
• cactus_at_cactuscode.org

2
Grid Computing a new paradigm

• Computational Resources Scattered Across the
  World
• Compute servers
• Handhelds
• File servers
• Networks
• Playstations, etc
• How to take advantage of this for
• scientific/engineering simulations?
• Harness multiple sites and devices
• Simulations at new level of complexity
• and scale

3
Components for Grid Computing

• Resources Egrid (www.egrid.org)
• A Virtual Organization in Europe for
• Grid Computing
• Over a dozen sites across Europe
• Many different machines
• Infrastructure Globus Metacomputing Toolkit
• Develops fundamental technologies needed to build
  computational grids. 
• Security logins, data transfer
• Communication
• Information (GRIS, GIIS)

4
Components for Grid Computing

• Application Cactus Computational Toolkit
• Modular Toolkit for Parallel Computation
• Numerical/Computational Infrastructure to solve
  PDEs
• Enables Grid applications of many types
• www.cactuscode.org

5
Grid Computing Scenarios The Vision

• Distributed Computing Sit here, compute there,
  monitor and steer
• Managing intelligent
• parameter surveys
• jobs to new machines, e.g. analysis tasks
• Dynamic staging seeking out and moving to
  faster/larger/cheaper machines as they become
  available
• Scripting capabilities (management, launching new
  jobs, checking out new code, etc)
• Dynamic load balancing

6
Application Code as Information Server/Gatherer

• Code should be aware of its environment
• What resources are out there?
• What is their current state?
• What is my allocation?
• What is the bandwidth and latency between sites?
• How can I adjust myself to take advantage of the
  current state?
• Code should be able to make decisions on its own
• A slow part of my simulation can run
  asynchronouslyspawn it off!
• New, more powerful resources just became
  availablemigrate there!
• Code should be able to publish this information
  to central server for tracking, monitoring,
  steering...
• Cactus has modules to enable this for any
  application

7
Cactus Worm Illustration of basic scenario

• Cactus simulation starts, launched from a portal
• Queries a Grid Information Server, finds
  available resources
• Migrates itself to next site
• Uses some logic to choose next resource
• Securely starts up remote simulation
• Transfers memory contents to remote simulation
  (using streaming HDF5, scp, GASS, whatever)
• Registers new location to server, terminates
  previous simulation
• User tracks and monitors with continuous remote
  viz and control using thorn http, streaming data,
  etc...
• Continues around Europe, and so on
• If we can do this, much of what we want can be
  done!

8
Grand Picture we are very close!
Viz of data from previous simulations in SF café
Remote steering and monitoring from airport
Remote Viz in St Louis
Remote Viz and steering from Berlin
DataGrid/DPSS Downsampling
IsoSurfaces
http
Streaming Data
T3E Garching
Origin NCSA
Globus
Simulations launched from Cactus Portal
Grid enabled Cactus runs on distributed machines
9
Next

• Tonight Global Grid Forum BOF
• Tomorrow manchester Booth at 1030
• Thanks to Sun for sponsoring us!