xgrid@MIT: An innovative campus grid prototype

1
xgrid_at_MIT An innovative campus grid prototype

• Adam Kocoloski and Mike Miller
• Massachusetts Institute of Technology
• STAR collaboration

2
Outline

• Motivation for (another) campus grid
• What is unique about xgrid_at_MIT
• Introduction to Apples xgrid system
• Integrating xgrid into SUMS
• Results and deliverables
• Future plans

3
Typical user analyses

1. Run n identical processes with different seeds
2. Analyze n files in m processes (mltn)

User Input () Policy . dispatcher
4
How it all began

• Wouldnt it be great to harvest wasted cycles?
• But
• Not another batch system
• No scripting
• No overhead for setup
• No admin access to others machines
• No overhead for maintenance
• Someone said
• try that xgrid button

• Apple xgrid a new HE(N)P grid platform?
• Single vendor, proprietary, built into OS X 10.4
• No requirements on clients and agents
• Uniquely scalable
• OSG interface
• Standard STAR-GRID interface
• No need to learn xgrid syntax
• No scripting needed
• Business as usual
• ? A unique campus grid

5
xgrid_at_MIT layout
SUMS
6
Apples xgrid the promises

• Instant configuration
• A clickable setup
• Anywhere (that port 4111 is open)
• Even behind BNL firewall
• Single sign on capable (KDC)
• Separate authentication for clients, agents
• Scalable and stable
• No hard limit on number of agents
• Prompt controller auto-recovery from crash
• Management made easy
• Once authenticated, agents auto-detected by
  controller

7
Integrating xgrid into SUMS

• Abstract user task request into xml
• Auto generate scripts
• Test/choose queues
• Create sandbox
• Submit tasksjobs
• Resubmit failed jobs
• Retrieve results
• Clean jobs from queue

8
Growing the grid

• Challenges
• No control of apps installed on agents!
• Jobs run as user nobody
• No static linking under OS X
• Admin must provide
• NFS/AFP accessible libraries
• Grid build recipe
• Controller cannot retrieve 10GB text file
  transfer
• Established
• XgridDispatcher integrated into standard SUMS
  development
• dedicated controller
• dedicated NFS/AFP fileservers
• 1 dedicated agent
• 4 harvested desktop agents
• handful of laptops (PPC and Intel!)
• ? 19 cpu for 42 GHz
• tiny fraction of available MIT machines
• Stability
• Only 1 (predictable) admin intervention in 3
  months!

9
Performance and deliverables

• 33-44 GHz range
• Currently handful of users
• But 1/2 day job becomes 1/2 hour
• Smooth processing of O(100GB) datasets,
  impossible on laptop

• Deliverables
• Subset of 2006 offline calibration for STAR
  Calorimeter
• Various suite of simulation studies
• Extensive analyses for impending publication
• Data analysis for hep-ex/0608030, submitted to PRL

10
Future plans

• Next 2 months
• Extended testing of prototype
• Already integrated Intel architecture
• Adapt single task xml submission
• Integrate MIT-IST Apple cluster
• Testing by string, HEP theorists, Neutrino group
• Demonstrate OSG capability
• Port STAR libs to OS X
• Validate simultaneous SUMS-submission to
  xgrid_at_mit, PDSF, BNL, etc
• By next year
• Require single-sign on user authentication
• Dedicated, scalable backbone
• x10 in dedicated CPU
• Xsan Xserve RAID (5 TB)
• Harvest MIT
• Website under development
• launch Xgrid_at_MIT_at_home

• Capable of ltcpugt 300 GHz in next 12 months
• One of largest Apple campus grids
• Largest HE(N)P Apple grid?

11
Conclusions

• Marriage of two technologies
• Apple xgrid
• SUMS user interface
• For MIT-STAR users
• essentially free grid
• business as usual
• Prototype
• All from existing machines
• No FTE administrator!
• Stable, immediately useful
• Unique OSG growth potential
• Dedicated, I/O capable backbone
• Harvested, cpu-capable _at_home component

12
Backup