CERN Clusters

1
CERN Clusters

• Tim Smith CERN/IT

2
Heterogeneous Clusters

• 10 years evolution
• HP-UX, IRIX, AIX, DUX, Solaris, WNT
• 4 years ago
• Linux additions / replacements
• 37 clusters configurations !
• e.g. CMS
• Interactive Solaris, Linux
• Batch Solaris, HP-UX, Linux

3
RISC Decommissioning
4
The Rise and Fall of PC Clusters
5
The Rise and Fall of PC Clusters
Elonex III
TechAS
Elonex II
Siemens
HP
Elonex I
Cogestra
6
Component Architecture
High capacitybackboneswitch
Application Server
100/1000baseT switch
CPU
CPU
CPU
CPU
CPU
Disk Server
1000baseT switch
Tape Server
Tape Server
Tape Server
Tape Server
7
Concentrated Facilities

• Interactive Cluster
• 50 bi-processor PCs 512 MB, 440-800 MHz
• Batch Cluster with chaotic access
• 280 bi-processor PCs 0.1-1 GB, 440-800 MHz
• Batch Cluster with scheduled access
• 190 bi-processor PCs 512 MB, 600-800 MHz
• Tape and Disk server Clusters

8
Chaotic Clusters
lxbatch001
lxbatch001
DNS load balancing
lxbatch001
lxbatch001
lxbatch001
lxbatch001
lxbatch001
lxbatch001
lxbatch001
lxbatch001
lxplus001
lxplus001
lxplus001
LSF
lxplus001
lxplus001
lxplus001
rfio
lxplus001
lxplus001
lxplus001
tape001
rfio
tape001
disk001
disk001
9
Chaotic Clusters
Public_queues
lxbatch001
lxbatch001
lxbatch001
lxbatch001
DNS load balancing
ATLAS_queues
lxbatch001
lxbatch001
lxbatch001
lxbatch001
lxbatch001
lxbatch001
CMS_queues
lxbatch001
lxbatch001
lxbatch001
lxbatch001
lxbatch001
lxbatch001
lxbatch001
lxbatch001
lxbatch001
lxbatch001
production_queues
lxplus001
lxplus001
lxplus001
LSF
lxplus001
lxplus001
lxplus001
rfio
lxplus001
lxplus001
lxplus001
tape001
rfio
tape001
disk001
disk001
10
Scheduled Cluster
11
Management Techniques I

• KickStart JumpStart (Linux Solaris)
• System installation
• ANIS
• installation automation (bootp, dhcp, tftp, nfs)
• SUE
• System post installation and configuration
• ASIS
• Application installation (3 GB local)

12
Management Techniques II

• Console Mgmt
• PCM (DEC PolyConsole Manager)
• Console Concentrators
• Cross Wiring serial ports
• Etherlite, VACM
• Power Mgmt
• NONE
• Monitoring
• SURE, perfmon, remperf,