General Talk about NeSC

1
National e-Science Core Programme Grid
Highlights BiGUM1 Meeting _at_ eSI 30th October
2001
2
Contents

• Welcome
• NeSC and e-Science Support
• Grid Definitions
• Grid Examples
• Grid Architectures

3
e-Science Programme
DG Research Councils
Grid TAG
E-Science Steering Committee
Director
Directors Management Role
Directors Awareness and Co-ordination Role
Generic Challenges EPSRC (15m), DTI (15m)
Academic Application Support Programme Research
Councils (74m), DTI (5m) PPARC (26m) BBSRC
(8m) MRC (8m) NERC (7m) ESRC (3m) EPSRC
(17m) CLRC (5m)
80m Collaborative projects
Industrial Collaboration (40m)
From Tony Hey 27 July 01
4
UK Grid Network
Edinburgh
Glasgow
Newcastle
DL
Belfast
Manchester
Cambridge
Oxford
Hinxton
RAL
Cardiff
London
Soton
From Tony Hey 27 July 01
5
Key Elements of UK Grid Development Plan

• Network of Grid Core Programme e-Science Centres
• Development of Generic Grid Middleware
• Grid IRC Grand Challenge Project
• Support for e-Science Testbeds
• International Involvement via GGF
• Grid Network Team

From Tony Hey 27 July 01
6
NeSCs context
Coordination
7
NeSC The Team

• Director
• Malcolm Atkinson (Universities of Glasgow
  Edinburgh)
• Deputy Director
• Arthur Trew (Director EPCC)
• Commercial Director
• Mark Parsons (EPCC)
• Regional Director
• Stuart Anderson (Edinburgh Informatics)
• Chairman
• Richard Kenway (Edinburgh Physics Astronomy)
• Initial Board Members
• Muffy Calder (Glasgow Computing Science)
• Tony Doyle (Glasgow Physics Astronomy)
• Centre Manager
• Anna Kenway

8
NeSCs Roles

• Stimulation of Grid e-Science Activity
• Users, developers, researchers
• Education, Training, Support
• Think Tank Research
• Coordination of Grid e-Science Activity
• Regional Centres, Task Forces, Pilots IRCs
• Technical and Managerial Fora
• Support for training, travel, participation
• Developing a High-Profile Institute
• Meetings
• Visiting Researchers
• Regional Support
• Portfolio of Industrial Research Projects

9
eSI Highlights

• Report given yesterday
• History
• 3 workshops week 1 DF1, GUM1 DBAG1
• HEC
• preGGF3 DF2
• October
• Steve Tuecke Globus tutorial (oversubscribed)
• 4-day workshop Getting Going with Globus (G3)
• Reports on DataGrid GridPP experience
• Biologist Grid Users Meeting 1 (BiGUM1)
• November
• GridPP
• Configuration management
• December
• AstroGrid

10
eSI Highlights cont.

• 2002 2003
• January
• Steve Tuecke 4 day Globus Developers Workshop
• February
• UKOLN
• March
• Protein folding Workshop 14th to 17th IBM
  sponsor
• May
• Mind and Brain Workshop
• 22nd to 26th July GGF5 HPDC 11 EICC
• August Research Festival
• 4 juxtaposed 1-week in-depth workshops
• Topics under consideration
• Dependability and Security for the Grid
• Metadata and the Grid
• Provenance, Annotation and Archiving
• The Knowledge Grid
• Programming Models for the Grid
• 14th to 16th April 2003 Dependability

11
Motivation for IPG

• Large-scale science and engineering are done
  through the interaction of people, heterogeneous
  computing resources, information systems, and
  instruments, all of which are geographically and
  organizationally dispersed.
• The overall motivation for Grids is to
  facilitate the routine interactions of these
  resources in order to support large-scale science
  and engineering.

From Bill Johnston 27 July 01
12
Why Grids?

• A biochemist exploits 10,000 computers to screen
  100,000 compounds in an hour
• 1,000 physicists worldwide pool resources for
  petaop analyses of petabytes of data
• Civil engineers collaborate to design, execute,
  analyze shake table experiments
• Climate scientists visualize, annotate, analyze
  terabyte simulation datasets
• An emergency response team couples real time
  data, weather model, population data

From Steve Tuecke 12 Oct. 01
13
Why Grids? (contd.)

• A multidisciplinary analysis in aerospace couples
  code and data in four companies
• A home user invokes architectural design
  functions at an application service provider
• An application service provider purchases cycles
  from compute cycle providers
• Scientists working for a multinational soap
  company design a new product
• A community group pools members PCs to analyze
  alternative designs for a local road

From Steve Tuecke 12 Oct. 01
14
The Grid Problem

• Flexible, secure, coordinated resource sharing
  among dynamic collections of individuals,
  institutions, and resource
• From The Anatomy of the Grid Enabling Scalable
  Virtual Organizations
• Enable communities (virtual organizations) to
  share geographically distributed resources as
  they pursue common goals -- assuming the absence
  of
• central location,
• central control,
• omniscience,
• existing trust relationships.

From Steve Tuecke 12 Oct. 01
15
Elements of the Problem

• Resource sharing
• Computers, storage, sensors, networks,
• Sharing always conditional issues of trust,
  policy, negotiation, payment,
• Coordinated problem solving
• Beyond client-server distributed data analysis,
  computation, collaboration,
• Dynamic, multi-institutional virtual
  organisations
• Community overlays on classic org structures
• Large or small, static or dynamic

From Steve Tuecke 12 Oct. 01
16
Why Now?

• Moores law improvements in computing produce
  highly functional endsystems
• The Internet and burgeoning wired and wireless
  provide universal connectivity
• Changing modes of working and problem solving
  emphasize teamwork, computation
• Network exponentials produce dramatic changes in
  geometry and geography

From Steve Tuecke 12 Oct. 01
17
Network Exponentials

• Network vs. computer performance
• Computer speed doubles every 18 months
• Network speed doubles every 9 months
• Difference order of magnitude per 5 years
• 1986 to 2000
• Computers x 500
• Networks x 340,000
• 2001 to 2010
• Computers x 60
• Networks x 4000

Moores Law vs. storage improvements vs. optical
improvements. Graph from Scientific American
(Jan-2001) by Cleo Vilett, source Vined Khoslan,
Kleiner, Caufield and Perkins.
From Steve Tuecke 12 Oct. 01
18
Broader Context

• Grid Computing has much in common with major
  industrial thrusts
• Business-to-business, Peer-to-peer, Application
  Service Providers, Storage Service Providers,
  Distributed Computing, Internet Computing
• Sharing issues not adequately addressed by
  existing technologies
• Complicated requirements run program X at site
  Y subject to community policy P, providing access
  to data at Z according to policy Q
• High performance unique demands of advanced
  high-performance systems

From Steve Tuecke 12 Oct. 01
19
The Globus ProjectMaking Grid computing a
reality

• Close collaboration with real Grid projects in
  science and industry
• Development and promotion of standard Grid
  protocols to enable interoperability and shared
  infrastructure
• Development and promotion of standard Grid
  software APIs and SDKs to enable portability and
  code sharing
• The Globus Toolkit Open source, reference
  software base for building grid infrastructure
  and applications
• Global Grid Forum Development of standard
  protocols and APIs for Grid computing

From Steve Tuecke 12 Oct. 01
20
Online Access to Scientific Instruments
Advanced Photon Source
wide-area dissemination
desktop VR clients with shared controls
real-time collection
archival storage
tomographic reconstruction
DOE X-ray grand challenge ANL, USC/ISI, NIST,
U.Chicago
From Steve Tuecke 12 Oct. 01
21
Supernova Cosmology Requires Complex,Widely
Distributed Workflow Management
22
Mathematicians Solve NUG30

• Looking for the solution to the NUG30 quadratic
  assignment problem
• An informal collaboration of mathematicians and
  computer scientists
• Condor-G delivered 3.46E8 CPU seconds in 7 days
  (peak 1009 processors) in U.S. and Italy (8 sites)

14,5,28,24,1,3,16,15, 10,9,21,2,4,29,25,22, 13,26,
17,30,6,20,19, 8,18,7,27,12,11,23
MetaNEOS Argonne, Iowa, Northwestern, Wisconsin
From Miron Livny 7 Aug. 01
23
Network for EarthquakeEngineering Simulation

• NEESgrid national infrastructure to couple
  earthquake engineers with experimental
  facilities, databases, computers, each other
• On-demand access to experiments, data streams,
  computing, archives, collaboration

NEESgrid Argonne, Michigan, NCSA, UIUC, USC
From Steve Tuecke 12 Oct. 01
24
Home ComputersEvaluate AIDS Drugs

• Community
• 1000s of home computer users
• Philanthropic computing vendor (Entropia)
• Research group (Scripps)
• Common goal advance AIDS research

From Steve Tuecke 12 Oct. 01
25
Layered Grid Architecture(By Analogy to Internet
Architecture)
From Steve Tuecke 12 Oct. 01
26
Architecture of a Grid
Discipline Specific Portals andScientific
Workflow Management Systems
Applications Simulations, Data Analysis,
etc. Toolkits Visualization, Data
Publication/Subscription, etc.
Grid Common Services Standardized Services and
Resources Interfaces
Collaboration and Remote Instrument Services
Grid Information Service
UniformResourceAccess
Co-Scheduling
Network Cache
Authentication Authorization
Security Services
Communication Services
Global Queuing
Global EventServices
Data Cataloguing
Uniform Data Access
Fault Management
Monitoring
Brokering
Auditing
Globus services
clusters
Distributed Resources
national supercomputer facilities
tertiary storage
national user facilities
Condor pools
networkcaches
High-speed Networks and Communications Services
27
Architecture of a Grid upper layers

• Knowledge based query
• Tools to implement the human interfaces, e.g.
  SciRun, ECCE, WebFlow, .....
• Mechanisms to express, organize, and manage the
  workflow of problem ????solutions
  (frameworks)
• Access control

Problem Solving Environments
Applications and Supporting Tools
Grid enabled libraries (security, communication
services, data access, global event management,
etc.)
Application Development and Execution Support
Grid Common Services
From Steve Tuecke 12 Oct. 01
Distributed Resources
28
Three Layer GRID Abstraction
From Tony Hey 12 Sep. 01
29
Data, Information and Knowledge

• Data
• Uninterpreted bits and bytes
• Information
• Data equipped with meaning
• Knowledge
• Information applied to achieve a goal, solve a
  problem or enact a decision

From Tony Hey 12 Sep. 01
30
Biological Grid Users

• Are they different?
• Do they have different collaborations?
• Do they have different data?
• Do they have different computations?
• Do they have the same shared instruments?
• Can they be supported using the same
• Infrastructure
• Architecture
• Policies?