Deployment of an LCG Infrastructure in Australia

1
Deployment of an LCG Infrastructure in
Australia How-To Setup the LCG Grid Middleware
A beginner's perspective Marco La
Rosa mlarosa_at_physics.unimelb.edu.au eScience
2005 Workshop Deploying Production Grids
Beyond the Hype
2
Deployment of an LCG Infrastructure in Australia

• Australian Government provides funding for a
  National High Performance Computing facility.
• Australian Partnership for Advanced Computing
  (APAC)
• APAC has partner computing facilities in most of
  the states
• ac3, ANU, CSIRO, iVEC, QPSF, SAPAC, TPAC, VPAC
• State facilities support researchers from their
  member institutions
• All working together on the
• Australian National Grid Program

3
Deployment of an LCG Infrastructure in Australia

• APAC provides funding for application level
  projects
• High Energy Physics Application area within the
• National Grid Program
• Deploy an LCG infrastructure within Australia
• Develop expertise in the deployment and use of
  the LCG middleware
• Work with the KEK Computing Facility (Japan) and
  members of the Belle experiment in the federation
  of resources
• Deploy the Belle Monte Carlo Analysis on the Grid

4
Deployment of an LCG Infrastructure in Australia

• Current state of Deployment
• Compute Element deployed as a Xen Virtual Machine
  at the
• Victorian Partnership of Advanced Computing
  (VPAC)
• Supported by
• User Interface, Resource Broker, Global-BDII,
  Storage Element and VOMS at the School of
  Physics, The University of Melbourne
• Deployed as Xen Virtual Machines
• Large tax on resources to get started in Grid -
• easier to get started if using Virtual Machine
  technology.
• National Grid Program is using 'gateway' machines
  which host different Grid interfaces to resources
  based on application area requirements

5
Deployment of an LCG Infrastructure in Australia

• Beyond the Hype
• Grid tools not available on the cluster compute
  nodes (Worker Nodes)
• Policy
• Compute Nodes should not need access to the
  outside world
• User accounts one account grid-belle
• Shared facility will the whole facility belong
  to LCG?
• We have an excellent relationship with VPAC
• support our application area, provide computing
  and human resources
• Use the Compute Element as a Globus interface
• LCG CE easier to setup than Globus gatekeeper!
• LCG CE supported by ALL of CERN and members!

6
Deployment of an LCG Infrastructure in Australia

• Operating System Issues (RedHat, Suse, Scientific
  Linux 3)
• Athena is supported on Scientific Linux 3 (it's
  not about the middleware)
• We cannot deploy the Athena toolkit at VPAC
  because of the OSes they have on their clusters
• We do not have the human resources to compile,
  test and support Athena on our own whilst it is
  in development and releases are occurring so
  frequently
• We do have portable distribution of the
• Belle Analysis Software Framework (BASF)
• meta-scheduler developed by Dr. Lyle Winton we
  can still use the resources

7
Deployment of an LCG Infrastructure in Australia

• Planned / Potential Deployments
• ac3, SAPAC, APAC National Facility
• Expected Concerns
• Grid tools on cluster compute nodes and
  connectivity to the outside world
• support issues limited computing and human
  resources
• National Facility
• highly specialised not a stock standard beowulf
• highly customised PBS openPBS further developed
  in-house
• strict security considerations

8
Deployment of an LCG Infrastructure in Australia

• Advanced Research Computing, The University of
  Melbourne
• Nordugrid deployment
• Participated in ATLAS Data Challenges 1 and 2
• Expertise in Grid prior experience with CERN /
  LCG
• Have provided a cluster for use as a prototype
  Tier 2
• Service Challenge 4 / ATLAS Data Challenge 3
• allow us to develop the expertise required to
  support a Grid site

9
Deployment of an LCG Infrastructure in Australia

• Beyond the Hype
• Human resources
• I am the systems and site administrator,
  security contact
• This is not my main wage earning occupation!
• Computing resources
• Have had to use Xen Virtual Machines because of a
  lack of physical computing resources
• small site 26 cpu, 2Tb disk
• Minimum commitment to join the Grid
• 3 computers (CE, SE, MON)
• ATLAS Data Challenge requirement VO-BOX
• make that 4 computers!

10
Deployment of an LCG Infrastructure in Australia

• Summary and Conclusions
• Distributed Management
• LCG Australia Site administrator and support
• Systems Managers at each Site
• Site Policy Very Important!
• The PAC's are not members of LCG / EGEE
• do not necessarily agree with LCG / EGEE security
  agreements may change with time
• Other people's resources shared facilities
• Good working relationship essential trust!

11
How-To Setup the LCG Grid Middleware A
beginner's perspective

• LCG Technical Workshop KEK, Japan, November 17
  18, 2005
• Installation and Configuration of an LCG Grid
• Get to know your middleware
• Beginner's introduction to the Grid
• Range of Experience
• very experienced Manage a Tier 2
• no experience Grid huh?

12
How-To Setup the LCG Grid Middleware A
beginner's perspective

• Format of the workshop
• Prepare the node
• Is APT installed?
• Set up APT
• Update the system to Sci Linux 3.0.5
• Configure NTP
• Download the installer and prepare for
  installation
• Configure site info files
• Install and configure the middleware component
• Common tasks certificates, preparing for
  installation
• Time Required One full day
• Testing does it work?
• Time Required One full day

13
How-To Setup the LCG Grid Middleware A
beginner's perspective

• YAIM is a great installer
• experience shows us that the combination of a
  maturing middleware and intelligent installation
  tool results in the correct installation and
  configuration of the components most of the time
• Generally, errors are a result of connectivity
  between components
• Side job post doc hears about Grid and wants
  to give it a go
• post doc is not a network administrator
• tests the Grid and finds problems
• thinks the middleware is buggy
• doesn't realise that port X is blocked at the
  firewall
• gives up

14
How-To Setup the LCG Grid Middleware A
beginner's perspective

• Information
• Lots of it and scattered all over the internet
• No good general introduction and guide
• experience helps but how do you gain experience
  if you don't know where to start?
• As a new administrator it's not immediately
  obvious
• what is needed to get started (minimum number of
  computers)
• can LCG facilities be used? (R-GMA registry and
  schema server)
• when you do get started, how do you go about
  joining the Grid if you've never heard of a ROC?

15
How-To Setup the LCG Grid Middleware A
beginner's perspective

• What did we learn?
• Grid does not just work
• But - it can
• With the help of
• experienced administrators,
• good documentation,
• a lesson on debugging network connectivity
  issues,
• good general understanding of The Grid,
• services and ports,
• firewalls...

16
The End...
Thankyou http//epp.ph.unimelb.edu.au/EPP mlaros
a_at_physics.unimelb.edu.au