Computer Systems Lab

1
Linux Clusters

• David Thompson
• thomas_at_cs.wisc.edu
• http//www.cs.wisc.edu/ \ thomas/madlug

2
Overview

• The Computer Systems Lab (CSL)
• Clusters
• The condor/db cluster
• Scalable Linux Administration

3
Overview

• The Computer Systems Lab (CSL)
• Clusters
• The condor/db cluster
• Scalable Linux Administration

4
Computer Systems Lab

• Purpose
• Staff
• Resources

5
Purpose

• To support the research and teaching missions of
  the Department of Computer Sciences

6
Staff

• 8 Full Time
• 12 - 20 Part Time

7
Responsibilities

• Networks
• Gigabit, 100BaseT, ATM, FDDI
• Cisco, Foundry routers
• 3com, HP, Cisco switches

8
Responsibilities (cont.)

• Operating Systems
• Solaris, Linux, Digital Unix, AIX, IRIX, NT
• Applications
• compilers, dbs, simulators, email, image
  processing....

9
Responsibilities (cont.)

• 641 software packages installed
• 69 Gbytes
• multiple version
• each package installed for several architectures
• several thousand builds

10
Responsibilities - (cont.)

• Workstations
• 600 PCs (including cluster)
• 200 Sparcs
• 15 Alphas
• others
• 5600 User home directories
• 69 Gbytes

11
Responsibilities (more)

• AFS
• 1 Tbyte of ubiquitous file space
• 14 File Servers, 3 db Servers
• 95 client cache hit rates
• Backups
• 2 week epoch cycle (1 Tb)
• Daily incs

12
Overview

• The Computer Systems Lab (CSL)
• Clusters
• The condor/db cluster
• Scalable Linux Administration

13
Clusters

• Definitions
• Architectures
• Example
• Applications

14
Definitions

• NOW - Network of workstations
• COW - Cluster of workstations
• Some degree of network isolation
• Dedicated function

15
Architectures

• N-dimensional arrays
• previous next neighbor
• hypercube
• Simple Network

16
Architectures

• Distributed
• MPI
• PVM
• condor

17
Examples

• The Hive
• http//newton.gsfc.nasa.gov/thehive/

18
Examples - The Hive
19
Examples - The Hive (cont.)
20
Redundant Networks
http//einstein.drexel.edu/beowulf/Beowulf.html
21
http//www.cs.nmsu.edu/pcl/
22
Cluster Applications

• Image Analysis
• http//newton.gsfc.nasa.gov/thehive/thehive_dir/ti
  lton.html
• Parallel Virtual File System (PVFS)
• http//ece.clemson.edu/parl/pvfs/
• Speech Recognition
• http//noel.feld.cvut.cz/magi/

23
Cluster Applications (cont.)

• Physics
• Viscoelasticity
• http//www.meca.ucl.ac.be/memawww/deepflow/
• Seismology
• http//weland.esd.mun.ca/index.html
• Big Bang
• http//www.phy.duke.edu/muller/BRAHMA/index.html

24
Cluster Applications (cont.)

• Physics (cont.)
• Laser Interferometer Gravitational-Wave
  Observatory (LIGO)
• http//www.ligo.caltech.edu/
• NA49 (??)
• http//na49info.cern.ch/
• Large Acceptance Hadron Detector for an
  Investigation of Pb-induced Reactions at the CERN
  SPS

25
Overview

• The Computer Systems Lab (CSL)
• Clusters
• The condor/db cluster
• Scalable Linux Administration

26
Computer Science Cluster

• Two connected clusters
• Dual Xeon 550mHz, 512k cache, 1 Gig RAM, Ultra 2
  SCSI 9 Gig boot disk, tulip network
• 64 node compute cluster
• 36 node db cluster with 4 extra 9 Gig disks and
  GNIC-II Gigabit ethernet
• Red Hat Linux 6.1, kernel 2.2.12

27
Cluster Architecture
28
Cluster Picture
29
Overview

• The Computer Systems Lab (CSL)
• Clusters
• The condor/db cluster
• Scalable Linux Administration

30
Scalable Linux Administration

• What
• Why
• Installation
• Maintenance

31
Scalable Admin - What

• Leverage
• Control systems
• Remote monitoring
• Operating system upgrades
• Centralized Services
• kerberos, afs, logging

32
Scalable Admin - Why

• Consistent user view
• Available applications
• Stability
• Predictable Admin Environment
• Security

33
Scalable Admin - Installation

• Red Hat Kickstart
• Configuration file
• network config, nfs locations, disk layout, RPMs
  to install
• Boot disk, nfs, or bootp/dhcp
• Post-install script
• redhat-6.1/i386/doc/HOWTO/KickStart-HOWTO

34
Sample Kickstart Script

• Id ks.cfg,v 1.3 1999/10/07 185724 thomas
  Exp
• lang en_US
• network --bootproto bootp
• nfs --server pinstall.cs.wisc.edu --dir
  /install/redhat-6.0/i386
• keyboard us
• zerombr yes
• clearpart --all
• part / --size 100
• part /tmp --size 300
• part /var --size 75
• part /usr --size 570
• part swap --size 127
• part /var/vice/cache --size 120
• part /local --size 2 --grow --maxsize 4000

35
Scalable Admin - Maintenance

• Update RPMS
• Create list of RPMs, versions, and files to
  install
• Each computer updates based on list
• Special files
• package (afs)
• cfengine (gnu)
• config files (filedist)

36
Linux Clusters

• David Thompson
• thomas_at_cs.wisc.edu
• http//www.cs.wisc.edu/thomas/madlug