High Performance Computing at EPCC

1
High Performance Computing at EPCC Alan D
Simpson Technical Director
Telephone [phone]
Fax [phone]
Email a.simpson_at_epcc.ed.ac.uk
http//www.epcc.ed.ac.uk/
2
Overview

• Background
• HPC Facilities at EPCC
• HPCx
• Current Status
• HPCx and the Grid
• Training and Research in HPC
• Summary

3
EPCC

• Founded in 1990 as a focus for the University of
  Edinburgh activities in HPC
• Mission
• to accelerate the effective exploitation of
  novel computing in industry, academia, and
  commerce
• One of leading HPC Centres in Europe
• 65 staff
• 40 applications consultants support staff
• Income 2.7M per annum 30 from Industry
• Academic and industrial clients from UK, Europe
  and beyond

4
Technology Transfer
5
Industrial Consultancy

• Provide project-based consultancy to industry and
  commerce
• Over 30 clients in 3 years
• Large enterprises...
• eg, UK Met Office, Sun, CG, AEA, Cisco
• ...to local SMEs
• eg, Weidlinger, Quadstone, Jardine
• 40 of technical staff
• Funded by direct contracts with business, local
  government and European Commission

6
Industrial Clients
7
European Programmes

• Collaborative research
• HPC-Europa EPCC coordinates
  pan-European visitor programme
• DEISA connecting national centres across Europe
• IST (industrial) projects
• EUTIST-IMV co-ordination of 80 machine vision
  organisations
• Gridstart co-ordination of all EU Grid
  development projects

8
HPC Facilities at EPCC

• 1982 ICL DAPs
• 1986 Meiko T800 CS (400 processors)
• 1988 AMT DAP608

• 1990 Meiko i860 CS (64 processors)
• 1991 TMC CM-200 (16K processors)
• 1992 Meiko i860 CS (16 processors)

• 1994 Cray T3D (512 processors)
• Cray Y-MP
• 1995 Meiko CS-2

• 1997 Cray T3E (344 processors)
• 1997 Hitachi SR2201
• 2000 Sun UltraSPARC III Cluster

• 2002 Sun E15000 (54 processors)
• 2002 IBM p690 Cluster (1280 processors)
• 2004 QCDOC

9
UoE HPC Service

• Funded by 400K JREI grant
• awarded to EPCC in 1998
• freely available to local researchers
• Service based on Sun SMP clusters
• familiar software and easy porting
• recently upgraded to Sunfire E15K
• large memory and CPU with a single system image
• EPCC is a Sun Centre of Excellence in HPC and
  Grid Computing

10
QCDOC

• QCDOC is a collaborative project
  to develop a special-purpose
  computer for QCD
• involving EPCC, Physics,
  Columbia University, IBM,
• QCD Quantum ChromoDynamics
• key part of Standard Model of particle physics
• has very extreme computing requirements
• Price-performance is critical
• may be cheaper to design special purpose machines
  for particular problems
• only pay for what you use
• put extra effort into what is important to you

11
QCDOC

• Each node is small and
  consists of a single
  specially designed chip
  plus some
  memory
• very large numbers of
  nodes are possible
• Equivalent general purpose
  machine would be huge and expensive
• Difficulty of chip design reduced by including
  components (eg, CPU) from IBM design library
• 10TF machine to be installed at EPCC in 2004

12
HPCx Overview

• UKs major HPC facility, funded by EPSRC
• 53M/6 year contact awarded to UoE HPCX Ltd
• wholly-owned subsidiary of University of
  Edinburgh
• work subcontracted to CCLRC (DL), EPCC and IBM
• Largest academic supercomputer in Europe
• doubling in performance every 2 years

13
HPCx Objectives

• Capability computing for world-leading science
• Capability computing jobs which use a
  significant fraction of the resource, eg, at
  least 512 CPUs
• Maximise benefits to the UKs computational
  science and engineering community
• IBM technology roadmap
• 12/02 40x32-way Regatta H frames Colony
  Switch
• initially 9 on Top 500 list
• 07/04 48x32-way Regatta H frames Federation
  switch
• 11/06 96x32-way Regatta H frames Federation
  switch
• Science support is key for effective use

14
Partnership

• EPCC and CCLRC
• are partners in C3ES (Consortium for Capability
  Computing and e-Science)
• providing science support and systems management
  for HPCx
• underpinned by MoU between UoE and CCLRC
• combines Europes foremost academic HPC,
  e-Science and technology transfer centres
• significant experience of
• operating national HPC services
• developing capability applications
• the strongest UK partnership ever to support
  scientific computing

15
Virtual Organisation
16
HPCx and the Grid

• Key responsibility for Software Engineering team
• HPCx is committed to support access via Grid
• currently provided through Globus 2
• Globus 3 support when appropriate
• HPCx is key part of UK collaboration with
  Extensible Teragrid Facility project in the US
• focus is exploiting unique features of Grid HPC
  systems for capability computing
• initial experiment planned for SC2003
• RealityGrid computational steering
• HPCx is major compute resource

17
HPCx Status

• HPCx builds on significant complementary
  experience at EPCC and DL
• Very successful start
• averaging gt75 utilisation
• with capability usage already up to 35
• Committed to e-Science and the Grid
• ETF experiment at SC2003
• HPCx is focussed on capability computing
• world-class service for world-class research

18
MSc in HPC

• 400K grant from UK research council
• runs for 5 years
• just started year 3
• One of a very few such courses in the world
• Each year an increasing number of students,
  especially overseas students

19
Training in HPC

• Courses include
• Fundamental Concepts of HPC
• Practical Software Development
• Message Passing Programming
• Shared Memory Programming
• Parallel Decomposition
• Applied Computer Science
• Object Oriented Programming for HPC
• Exploiting the Computational Grid
• Applied Numerical Algorithms
• Performance Optimisation
• Scientific Visualisation
• Remote runs at, eg, Cambridge, Daresbury,

20
HPC Research

• Java Grande Forum
• EPCC leads the benchmarking activity
• including parallel benchmarks and language
  comparisons
• have taught Java tutorials at Supercomputing
• OpenMP
• EPCC a full member of OpenMP
  Architecture Review Board
• OpenMP microbenchmarks
• tests quality of the compiler implementation
• becoming a de-facto standard

21
HPC Research

• JOMP
• an OpenMP-like standard for Java
• research implementation available for download
• Mixed Mode
• combined OpenMP MPI becoming popular
• topic of investigation at EPCC for over 3 years
• Single Sided MPI
• EPCC produced implementations for Cray, Sun,
• Optimised Libraries
• BLAS, FFTs, ScaLAPACK,

22
Summary

• EPCC is multidisciplinary and multi-funded
• ... supporting a large spectrum of activities ...
• and a critical mass of expertise
• Proven track record in Technology Transfer
• business-like approach benefits whole
  organisation
• New initiatives
• MSc in HPC
• European programmes
• Grid middleware
• HPCx
• EPCC has a unique breadth of expertise