ScotGRIDGlasgow History

1
ScotGRID-Glasgow History

• Xmas 2001 IBM Deliver kitWe got a solution
  from IBM - a package deal of hardware, selected
  software and help getting started.
• http//www.redbooks.ibm.com/redbooks/SG246623.html
  
• http//www.redbooks.ibm.com/pubs/pdfs/redbooks/sg2
  46623.pdf
• Spring 2002 Accommodation refurbished
• Summer 2002 Start of operation according to
  IBMs solution

2
ScotGRID-Glasgow(1)
3
ScotGRID-Glasgow(2)
4
Batch

• IBMs solution incorporates a traditional Batch
  Job Processing System
• the JOB is the basic unit of non-interactive work
  - a few minutes to many hours of processing
• JOBs are described in a Job Description Language
• SUBMITTING the JOB is the process of notifying
  the Batch System of a new JOB to add to its
  queue.
• The BATCH system EXECUTEs/RUNs the JOB when it
  deems there are sufficient free resources and it
  is the JOBs turn to get resources

5
Batch Details

• The Batch System used is OPENPBS
• http//www.openpbs.org/
• The Job Description Language is Shell Scripts
  with special comments
• Jobs are submitted using the qsub command
• The location of the job output is determined by
  the special PBS shell script comments
• Jobs are scheduled for execution by the maui
  plugin to OpenPBS
• http//www.supercluster.org/

6
Sample Job

• !/bin/tcsh
• PBS -N OptorSim
• PBS -l nice19
• PBS -m abe
• PBS -M wbell_at_a5.ph.gla.ac.uk
• PBS -o masternode/home/wbell/jobresults/stdout
• PBS -e masternode/home/wbell/jobresults/stderr
• setenv JAVA_HOME /usr/java/j2sdk1.4.1
• setenv PATH JAVA_HOME/binPATH
• setenv CLASSPATH JAVA_HOME/lib.
• echo "This is an OptorSim test"

See manpages for qsub(1B) and pbs_resources_linux(
7B)
7
Schematic
Masternode
Storage Nodes
Head Nodes
Campus Backbone
Internet VLAN
10.0.0.0 VLAN
100 Mbps
1000 Mbps
59 Compute Nodes
8
Job Environment

• RedHat Linux 7.2
• Disks - local to Compute Node
• 15 Gbytes in temporary /scr/PBS_JOBID
• Disks - shared using nfs
• home directories and user data is stored on 9 x
  100 Gbyte partitions on the storage nodes
• available on all nodes as /storage/data01,2,3,4,5,
  6,7,8,9
• allocations approved by Prof. A.T.Doyle

9
Login Arrangements

• accounts are approved by Prof. A.T.Doyle
• need user email address
• access is via ssh/scp to
• headnode1.scotgrid.ac.uk
• headnode2.scotgrid.ac.uk
• ( masternode.scotgrid.ac.uk )

10
Caveats

• security - ScotGRID Glasgow is very low security
  and must not be used for any sensitive material -
  especially anything covered by the Data
  Protection Act
• integrity - at present no backups are being made
  - all data is vulnerable
• some disk partititions are currently experiencing
  problems leading to loss of data

11
Future (1)

• The transfer of the 5 Tbyte store of
  ScotGRID-Edinburgh to ScotGRID-Glasgow will
  require major changes to the way data is stored -
  replacing the 3 storage nodes with a single big
  file server
• integration with additional Bio and Edikt kit
  would require even more radical changes to data
  storage - starting to use SAN technology to share
  access to disks

12
Future (2)

• ScotGRID-Glasgow is currently moving to Grid
  based operations - this makes a big change to how
  users relate to ScotGRID-Glasgow.
• Users would prepare jobs and submit them from
  some Grid Client box e.g. their desktop
• Users would authenticate using X509 certificates
  - not passwords
• Users might not have personal accounts on
  ScotGRID-Glasgow but use pool accounts