Building Visualization Clusters with Rocks

1
Building Visualization Clusters with Rocks

• SDSC Innovation Award Funded
• Mason Katz
• 06-30-04

2
Sales Pitch

• NPACI Rocks has become a clustering standard
• Currently 1 of June Top500 list
• Expanding to Application Vertical Markets
• Visualization Rocks Users
• NCMIR - tele-science group
• NCSA - visualization group
• Potentially OptIPuter
• Build on SDSC/Rocks name
• Allow PIs to build there own visualization labs
• COTS (no facility required)
• NCSA is a key partner here
• OSCAR vs. Rocks issues (may help bridge gap)
• Just need money to build a testbed
• Build system for developers
• Give to application when done

3
Rocks Background
4
NPACI Rocks (open source clustering
distribution)www.rocksclusters.org

• Technology transfer of commodity clustering to
  application scientists
• make clusters easy
• Scientists can build their own supercomputers and
  migrate up to national centers as needed
• Rocks is a cluster on a CD
• Red Hat Linux
• Clustering software (PBS, SGE, Ganglia, NMI)
• Highly programmatic software configuration
  management
• Core software technology for several campus
  projects
• BIRN
• Center for Theoretical Biological Physics
• EOL
• GEON
• NBCR
• OptIPuter
• First Software release Nov, 2000
• Supports x86, Opteron, and Itanium

5
Basic Architecture
Front-end Node(s)
Power Distribution (Net addressable units as
option)
Public Ethernet
Ethernet Switching Complex
Optional Application Interconnect
6
Rocks Registry (optional for users)
Top500 Nov03
Top500 June 03
7
Over 51 TFlops Aggregate Peak

• Earth Simulator - 41 TFlops
• Thunder - 23 TFlops
• ASCI Q - 21 Tflops
• Not a fair comparison, but good indication of
  Rocks impact

8
System Configuration Graph

• Laptop / Desktop
• Appliances
• Final classes
• Node types
• Desktop IsA
• standalone
• Laptop IsA
• standalone
• pcmcia
• One Framework
• Many Appliances

9
Complete Rocks Configuration Graph
10
What is a Roll

• Motivation
• Im concerned Rocks is becoming everything for
  everyone - rocks mailing list
• Building a cluster should be like ordering a
  car. I want the sports package, but not the
  leather seats, - z4 owning rocks developer
• We need to let go of Rocks but hold onto the core
• Recruit more external open-source developers
• Only trust ourselves with fundamental
  architecture and implementation
• We want to move the SGE but need to still support
  PBS
• Rolls
• Optional configuration and software
• Just another CD for installed (think application
  pack)
• SGE and PBS are different Rolls
• User chooses scheduler
• PBS Roll supported by Norway
• SGE Roll supported by Singapore (and us)
• Rolls give us more flexibility and less work
• IEEE Cluster 2004 - Rolls Modifying a Standard
  System Installer to Support User-Customizable
  Cluster Frontend Appliances

11
Extensible Rocks

• Over a dozen Rolls already created (e. g.)
• SGE, PBS
• Grid (NMI stack)
• Java
• AMD (32bit libraries for Opteron)
• Condor
• SCE
• Several third party Rolls have started
• Quadrics (rumored)
• SGE V6.beta
• NIMROD
• BIRN
• DB2
• Rocks is done
• The core is basically stable and needs continued
  support
• Rolls allow us to develop new ideas
• Application Domain specific
• For example Visualization

12
Vis Roll
13
Early Work NCSA

• LCD Cluster
• Custom framing
• One PC / tile
• Portable (luggable)
• SC 2001 Demo
• NCSA Software
• Pixel Blaster
• Display Wall In-A-Box
• OSCAR based
• Never fully released
• Contact David Semeraro
• Funding for travel
• All around Vis Guru
• Playing with Rocks

14
NCMIR

• Using Rocks
• Hand configured a visualization cluster
• Administered the machine to the point of
  instability - David Lee
• Automation is needed

15
COTS Vis GeoWall

• LCD Clusters
• One PC / tile
• Gigabit Ethernet
• Optional Stereo Glasses
• Portable
• Commercial Frame (Reason)
• Applications
• Large remote sensing
• Volume Rendering
• Seismic Interpretation
• Brain mapping (NCMIR)
• Electronic Visualization Lab
• Jason Leigh (UIC)

16
Eye Candy (NCMIR)
17
Project Status

• Funding for Hardware Only
• 3 x 3 tile display
• 10 servers video card
• SDSC/Rocks testbed
• To be used for development
• To be Lost to production use (meteor model)
• Hand configured Rocks Vis Clusters
• NCSA and NCMIR
• Documented configuration changes
• Complete list of software required
• Hardware coming
• Shuttle PCs arrived and tested with Rocks
• Commercial Display Wall ordered (Reason Co)
• NEC delays
• 2 - 4 weeks

18
Project Plan

• July
• Build our Vis cluster
• Start development on Roll
• August
• Release beta version of Roll
• Testing with NCMIR
• September
• Release 1.0
• Start being used by
• NCMIR
• OptIPuter
• NCSA
• Future
• Continue evolution (Renderfarm?)
• Can we give away ownership to another group?