The CSIR Cluster Computing Centre

1
The CSIR Cluster Computing Centre

• Albert Gazendam, on behalf of CSIR's C4 team
• agazendam_at_csir.co.za
• 6 April 2006
• SCAW 2006

2
Problem statement

• Energizing research and associated infrastructure
  at CSIR
• Preparing CSIR's research community for CHPC
• Designing, building and operating a shared HPC
  resource
• Lowering the entry barriers to HPC
• Accommodating unknown and exotic user application
  requirements
• Low management and administrative intervention
• Limited budget (around R2M)

3
C4 in the national HPC context
4
Application areas already planning use

• Aeronautics air flow simulations
• Bioinformatics genetic pattern analysis and
  molecular modeling
• Geophysics crack propagation modeling
• Materials simulation of material properties
• Logistics analysis of geo-referenced datasets

5
Application areas expressing interest

• Cyberforensics analysis of huge digital
  evidence sets
• Construction engineering particulate material
  simulations
• Environmental sciences earth system modeling
• Earth observation pattern recognition
• Human language technologies various
  computational problems

6
Procurement and commissioning timescales

• RFI to vendor community - late October 2005
• RFP to vendor community - mid December 2005
• Final negotiations start late February 2006
• Equipment orders placed - early April 2006
• Installation starts - mid May 2006
• Installation concludes - late May 2006
• Operational in alpha state - early June 2006
• Operational in beta state - late June 2006
• Operational in production state - early July 2006

7
System diagram and main cabinets
8
Technical characteristics (1)

• 46 x dedicated cluster nodes, each with 2 x dual
  core 2.6GHz Opteron processors, 4GB of RAM and 2
  x 160GB SATA HDDs
• 1 x node with 4 x dual core 2.6GHz Opteron
  processors, 64GB of RAM and 2 x 72GB SCSI HDDs
• 4 x utility servers with same configuration as
  cluster nodes
• 2 TB of raw storage space, in storage system
  scalable to gt40TB
• All of above on common KVM network
• Space for 128 standard desktop PCs configured as
  additional cluster nodes

9
Technical characteristics (2)

• GbE local networking, and 10GbE fiber link
  between buildings
• Available PCIe slots on cluster nodes for
  upgradability to Infiniband or another high
  performance interconnect
• Reconfigurable VLANs and firewall rules to create
  multiple simultaneous virtual clusters,
  catering for exotic user requirements
• All of above (and previous slide) in
  air-conditioned datacenter with UPS system and
  backup diesel generators
• 4 x workstations with same configuration as
  cluster nodes
• 2 x viz workstations with similar configuration
  as cluster nodes, but equipped with NVIDIA Quadro
  FX3450 256MB PCIe cards

10
Usage and access model

• The C4 is envisaged as a open and shared
  scientific computing resource with zero entry
  barriers for all CSIR researchers. In order to
  encourage participation from throughout the CSIR
  no formal access requirements will be imposed to
  interested parties. In the spirit of an
  infrastructure commons it is expected that
  serious user groups will contribute to the upkeep
  and sustainability of C4 through various
  mechanisms discussed in the C4 operational plan.
  There will be two classes of user groups
  exploratory and serious. All users will however
  utilize C4's resources on a first-come-first-serve
  based by means of a job and resource allocation
  queue, for which scheduling rules will be
  implemented as the need arise.

11
Conclusion

• Most powerful computing resource in the CSIR
  (500 gigaflops)
• Probably one of the 10 most powerful computing
  resources in South Africa
• Extremely flexible architecture capable of
  dynamic reconfiguration
• Designed around open source software and COTS
  hardware