Session 1 Second Part

1
Session 1 Second Part Day 1 Monday 10th July
Malcolm Atkinson
2
Distributed Systems Introduction, Context
Challenges
3
Introduction to Grids

• History of Distributed Computing
• Challenges of Distributed Computing
• Driving Forces Motivation for Grids
• Collaboration
• Data Deluge
• Computational Roadmap
• A history of Grids

4
What is e-Science?

• Goal to enable better research in all
  disciplines
• Method Develop collaboration supported by
  advanced distributed computation
• to generate, curate and analyse rich data
  resources
• From experiments, observations and simulations
• Quality management, preservation and reliable
  evidence
• to develop and explore models and simulations
• Computation and data at all scales
• Trustworthy, economic, timely and relevant
  results
• to enable dynamic distributed collaboration
• Facilitating collaboration with information and
  resource sharing
• Security, trust, reliability, accountability,
  manageability and agility

Our challenge is to develop an integrated
approach to all three
5
Distributed Systems History
ARPA net
1960
1970
1980
1990
2000
6
Distributed Systems to Grids
1960
1970
1980
1990
2000
7
Grids History Overview
8
A Grid Computing Timeline
US Grid Forum forms at SC 98
Grid Forums merge, form GGF
European AP Grid Forums
I-Way SuperComputing 95
OGSA-WG formed
Physiology paper
Anatomy paper
GGF EGAform OGF
OGSA v1.0
1995 96 97 98 99 2000 01 02 03 04 05 2006
Source Hiro Kishimoto GGF17 Keynote May 2006
9
What is a Grid?

• A grid is a system consisting of
• Distributed but connected resources and
• Software and/or hardware that provides and
  manages logically seamless access to those
  resources to meet desired objectives

Handheld
Supercomputer
Server
Data Center
Cluster
Workstation
Source Hiro Kishimoto GGF17 Keynote May 2006
10
Grid Related Paradigms

• Cluster
• Tightly coupled
• Homogeneous
• Cooperative working

• Distributed Computing
• Loosely coupled
• Heterogeneous
• Single Administration

• Grid Computing
• Large scale
• Cross-organizational
• Geographical distribution
• Distributed Management

Source Hiro Kishimoto GGF17 Keynote May 2006
11
How Are Grids Used?
High-performance computing
Collaborative design
E-Business
High-energy physics
Financial modeling
Life sciences
Data center automation
E-Science
Collaborative data-sharing
Drug discovery
Source Hiro Kishimoto GGF17 Keynote May 2006
12
Grids In Use E-Science Examples

• Data sharing and integration
• Life sciences, sharing standard data-sets,
  combining collaborative data-sets
• Medical informatics, integrating hospital
  information systems for better care and better
  science
• Sciences, high-energy physics

• Simulation-based science and engineering
• Earthquake simulation

• Capability computing
• Life sciences, molecular modeling, tomography
• Engineering, materials science
• Sciences, astronomy, physics

• High-throughput, capacity computing for
• Life sciences BLAST, CHARMM, drug screening
• Engineering aircraft design, materials,
  biomedical
• Sciences high-energy physics, economic modeling

Source Hiro Kishimoto GGF17 Keynote May 2006
13
Views of Grids
14
Grids integrating providing homogeneity

• (Potentially) Generic Industry Supported
• Mechanism for combining many heterogeneous
  resources so they can be used remotely
• Data resources
• Computation resources
• Instruments
• Research processes procedures
• Restricting choices as to how it may be done
• Harder for provider to make localised decisions
• Deployment can be challenging
• Providing more homogeneity through virtualisation
• Should be easier to compose services
• More opportunity to amortise costs
• A component of e-Infrastructure

Deliberately choosing consistent interfaces,
protocols management controls across a set of
compatible services
15
Grids as a Foundation for Solutions
Much to be done by developers of applications
services and by resource providers

• The grid per se doesnt provide
• Supported e-Science methods
• Supported data information resources
• Computations
• Convenient access
• Grids help providers of these
• International national secure e-Infrastructure
• Standards for interoperation
• Standard APIs to promote re-use
• But Research Support must be built
• What is needed?
• Who should do it?

16
Grids as a Foundation for Solutions
Much to be done by developers of applications
services and by resource providers

• Therefore must support many categories of user
• Application Service Developers
• Deployers Operations teams
• Higher-level abstraction builders gateway
  providers

• The grid per se doesnt provide
• Supported e-Science methods
• Supported data information resources
• Computations
• Convenient access
• Grids help providers of these
• International national secure e-Infrastructure
• Standards for interoperation
• Standard APIs to promote re-use
• But Research Support must be built
• What is needed?
• Who should do it?

17
Motives for Grids
18
Why use / build Grids?

• Research Arguments
• Enables new ways of working
• New distributed collaborative research
• Unprecedented scale and resources
• Economic Arguments
• Reduced system management costs
• Shared resources ? better utilisation
• Pooled resources ? increased capacity
• Load sharing utility computing
• Cheaper disaster recovery

19
Why use / build Grids?

• Computer Science Arguments
• New attempt at an old hard problem
• Frustrating ignorance existing results
• New scale, new dynamics, new scope
• Engineering Arguments
• Enable autonomous organisations to
• Write complementary software components
• Set up run use complementary services
• Share operational responsibility
• General consistent environment forAbstraction,
  Automation, Optimisation Tools
• Generally available code mobility

20
Why use / build Grids?

• Political Management Arguments
• Stimulate innovation
• Promote intra-organisation collaboration
• Promote inter-enterprise collaboration

21
Key drivers for Grids
22
Drivers for Grids

• Collaboration
• Expertise is distributed
• Necessary to achieve critical mass of effort
• Necessary to raise sufficient resources
• Computational Power
• Rapid growth in number of processors
• Powered by Moores law device roadmap
• Challenge to transform models to exploit this
• Deluge of Data
• Growth in scale Number and Size of resources
• Growth in complexity
• Policy drives greater availability

23
Collaboration is key
24
Biomedical Research Informatics Delivered by Grid
Enabled Services
Portal
http//www.brc.dcs.gla.ac.uk/projects/bridges/
25
eDiaMoND Screening for Breast Cancer
1 Trust ? Many Trusts Collaborative Working Audit
capability Epidemiology

• Other Modalities
• MRI
• PET
• Ultrasound

Better access to Case information And digital
tools
Supplement Mentoring With access to
digital Training cases and sharing Of information
across clinics
Provided by eDiamond project Prof. Sir Mike
Brady et al.
26
climateprediction.net and GENIE

• Largest climate model ensemble
• gt45,000 users, gt1,000,000 model years

Response of Atlantic circulation to freshwater
forcing
10K
2K
27
Integrative Biology

• Tackling two Grand Challenge research questions
• What causes heart disease?
• How does a cancer form and grow?
• Together these diseases cause 61 of all UK
  deaths

• Will build a powerful, fault-tolerant Grid
• infrastructure for biomedical science
• Enabling biomedical researchers to use
  distributed resources such as high-performance
  computers, databases and visualisation tools to
  develop complex models of how these killer
  diseases develop.

28
IB Partners
29
Foundations of Collaboration

• Strong commitment by individuals
• To work together
• To take on communication challenges
• Mutual respect mutual trust
• Distributed technology
• To support information interchange
• To support resource sharing
• To support data integration
• To support trust building
• Sufficient time
• Common goals
• Complementary knowledge, skills data

Can we predictwhen it will work? Can we
findremedies when itdoesnt?
30
Grid Collaboration Questions

• Without collaboration little is achievable
• But must collaboration precede successful grid
  applications?
• Or will persistently and pervasively available
  grids stimulate collaborations?
• If we deliver support for collaborative teams,
  will we also support the individual researcher?
• Can we use grids to democratise computation?

31
CARMEN - Scales of Integration
Understanding the brain may be the greatest
informatics challenge of the 21st century
32
CARMEN Consortium
Leadership Infrastructure
Colin Ingram
Paul Watson
Leslie Smith
Jim Austin
33
CARMEN Consortium
International Partners
Shiro Usui(RIKEN Brain Science Institute) Lead
for the Japan Node of the International
Neuroinformatics Coordinating Facility
34
CARMEN Consortium
Commercial Partners
- applications in the pharmaceutical sector
- interfacing of data acquisition software
- application of infrastructure
- commercialisation of tools
35
The Challenge
Toshiba 04
Device diversification 90nm HP, LOP, LSTP 45nm
UTB SOI 32nm Double gate
Ischia, Italy - 9-21 July 2006
36
University Partners
Advanced Processor Technologies Group
(APTGUM) Device Modelling Group
(DMGUG) Electronic Systems Design Group
(ESDGUS) Intelligent Systems Group
(ISGUY) National e-Science Centre
(NeSC) Microsystems Technology Group
(MSTGUG) Mixed-Mode Design Group in IMNS
(MMDGUE) e-Science NorthWest Centre (eSNW)
Ischia, Italy - 9-21 July 2006
37
Industrial Partners
Global EDS vendor and world TCAD leader 600
licences of grid implementation, model
implementation
UK fabless design company and world
microprocessor leader Core IP, simulation tools,
staff time
UK fabless design company and world mixed mode
leader Additional PhD studentship for mixed mode
design
Global semiconductor player with strong UK
presence Access to technology, device data,
processing
Global semiconductor player with strong UK
presence Access to technology, device data,
processing
Global semiconductor player with UK presence CASE
studentship, interconnects
Trade association of the microelectronics
industry in the UK Recruiting new industrial
partners and dissemination
38
Summary After the Break
39
Grids in context

• Part of a long-term drive for distributed
  computing
• A new and ambitious form
• Search for trade-offs multiple uses
• Leads to many varieties
• Multiple stake holders
• Many good reasons for building using grids
• Question
• Will we have many grids or a consistent general
  purpose foundation grid?
• What are the minimum standards across the grids
• Collaboration is a key driver enabler

40
Minimum Grid Functionalities

• Supports distributed computation
• Data and computation
• Over a variety of
• hardware components (servers, data stores, )
• Software components (services resource managers,
  computation and data services)
• With regularity that can be exploited
• By applications
• By other middleware tools
• By providers and operations
• It will normally have security mechanisms
• To develop and sustain trust regimes

41
After the break

• Distributed system challenges
• Service Oriented Architectures
• Examples of the basic architecture

42
Questions Comments