Internet%202%20and%20the%20Grid

1
Internet 2 and the Grid

• The Future of Computing for
• Big Science at UNM

Timothy L. Thomas UNM Dept of Physics and
Astronomy
2
Acknowledgements Thanks to

• Susan Atlas
• Bob Ballance
• Chris Jordan
• Terry Loring
• Brian Smith
• Art St. George
• Lou Sullo
• Maggie Werner-Washburne
• Susan Wilson
• Terry Yates
• for useful discussions.

3
Outline Deconstructing the Title

• Internet 2
• The Grid
• Computing
• Big Science
• at UNM
• The Future

4
Grokking The Grid

• Grok v. To perceive a subject so deeply that
  one no longer knows it, but rather understands it
  on a fundamental level. Coined by Robert
  Heinlein in his 1961 novel, Stranger in a Strange
  Land.

(Quotes from a colleague of mine) Feb 2002
This grid stuff is garbage. Dec 2002 Hey,
these grid visionaries are serious!
5
So what is a Grid?
6
(No Transcript)
7
(No Transcript)
8
(No Transcript)
9

• Ensemble of distributed resources acting
• together to solve a problem
• The Grid is about collaboration, about
• people working together.
• Linking people, computing resources, and sensors
  / instruments
• Idea is decades old, but enabling technologies
  are recent.
• Capacity distributed throughout an
  infrastructure
• Aspects of Grid computing
• Pervasive
• Consistent
• Dependable
• Inexpensive

10

• History of Grids a topic of its own
• Corp. for Natl. Research Initiatives Study
  Development of societal structures coupled to
• the development of infrastructure
• Railroads
• Telegraphs and telephones
• Electric power
• Banking
• The Internet
• Dynamic the emergence of pooling and
  caching drives the evolution of middleware.
• Why Chicago?

11

• Virtual Organizations (VOs)
• Security implications
• Ian Fosters Three Requirements
• VOs that span multiple administrative domains
• Participant services based on open standards
• Delivery of serious Quality of Service

12

• Open Standards
• The technical basis for The Grid.
• Its all about standards. CJ
• Global Grid Forum a open standards body
• OGSA / OGSI
• Web services and WSDL
• Globus toolkit version 3 hopes to be
• analogous to TCP/IP.

13

• Quality of Service (QoS)
• A Grid should exceed the sum of its parts,
• not be bottlenecked by one of them.
• The Dream Capability, as well as Capacity
• The Grid as a way to solve Really Hard Probems
• Marc Ingbur problems you shouldnt be able to
  do
• Foster/Kesselman theres 105 out there
• Capability computing is a form of QoS !

14
Grid Metaphores

• A Grid allows infrastructures and computations
  that can adapt and evolve.
• As opposed to a just a "Cyber Infrastructure, I
  think more in terms of a "Cyber Ecosystem !
• In any case, new way of organizing human
  enterprise
• in the 21st centuryboth commercial and
  scientific.

15
Kinds of Grids

• The Grid will not emerge all at once it
• will grow and evolve like the internet did
• It will form as an intergrid

16
Kinds of Grids

• Foster and Kesselman identify several categories
  of Grids,
• by way of examples
• Government a National Grid
• relatively small numbers of users
• strategic computing reserves
• national collaboratory
• An HMO
• A private Grid
• Life-critical applications
• Specialized instrumentation e.g., CAT scanners
• Privacy issues

17

• Material science collaboratory
• Smaller-scale, open scientific research
• Very dynamic membership
• Diverse specialized instrumentation
• Synchrotron light sources
• Advanced microscopes
• Computational market economy
• An open public Grid
• Very large, very dynamic
• More loosely-couple computational resources

18
Examples existing Grids

• ( maybe see Hype)
• SETI_at_home
• FightAIDS_at_Home
• Folding_at_Home
• Genome_at_Home
• United Devices Corp w/ Intel Oxford - cancer
  research project
• U. of Wisc.-centered Condor Pools
• NASA Information Power Grid
• National Earthquake Engineering Simulations
  (NEES) Grid

19
High Energy Physics Grids

• GriPhyN (NSF)
• CS research focusing on virtual data, request
  planning
• Virtual Data Toolkit Delivery vehicle for
  GriPhyN products
• iVDGL International Virtual Data Grid
  Laboratory (NSF)
• A testbed for large-scale deployment and
  validation
• Particle Physics Data Grid (DOE)
• Grid-enabling six High-Energy/Nuclear Physics
  experiments
• EU Data Grid (EDG) Applications areas
• Particle physics
• Earth and planetary sciences "Earth
  Observation
• Biology
• GLUE Grid Laboratory Uniform Environment
• Link from US grids to EDG grids

20
ltltlt Grid Hype gtgtgt

• (Grids Grease or Glue?)

21

• Example The pejorization of Cluster
• Speaking loosly, Cray 1 was a cluster!
• DEC (? Windows NT)
• Beowulf clusters
• Sometimes cluster means nothing at all!
• Sometimes grid just means a
• distributed cluster not really a Grid.
• ? Keep in mind Fosters three criteria!

22
Big Science

• (The future of most science)

23
Enterprise Science more and more scientific
communities are finding themselves organizing
into what are effectively enterprises. Grid
computing as the foundation of enterprise activity
reveals a symmetry with industry and commerce a
symmetry that will benefit both.
24
(No Transcript)
25
(No Transcript)
26
Natural Grid Applications

• High-energy elementary particle and Nuclear
  Physics (HENP)
• Distributed image processing
• Astronomy
• Biological/biomedical research e.g., pathology
• Earth and Planetary Sciences
• Military applications e.g., space surveillance
• Engineering simulations ? NEES Grid
• Distributed event simulations
• Military applications e.g., SF Express
• Medicine distributed, immersive patient
  simulations ? Project Touch
• Biology complete cell simulations

27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
(No Transcript)
31
Natural Grid Applications

• High-energy elementary particle and nuclear
  physics (HE(N)P)
• Distributed image processing
• Astronomy
• Biological/biomedical research e.g., pathology
• Earth and Planetary Sciences
• Military applications e.g., space surveillance
• Engineering simulations ? NEES Grid
• Distributed event simulations
• Military applications e.g., SF Express
• Medicine distributed, immersive patient
  simulations ? Project Touch
• Biology complete cell simulations

32
(No Transcript)
33
Natural Grid Applications

• High-energy elementary particle and nuclear
  physics (HE(N)P)
• Distributed image processing
• Astronomy
• Biological/biomedical research e.g., pathology
• Earth and Planetary Sciences
• Military applications e.g., space surveillance
• Engineering simulations ? NEES Grid
• Distributed event simulations
• Military applications e.g., SF Express
• Medicine distributed, immersive patient
  simulations ? Project Touch
• Biology complete cell simulations

34
(No Transcript)
35
Grid Applications (continued)

• Data mining and knowledge management
• Economics and Finance
• Bioinformatics
• Genomics and proteomics
• Pathology, virology

36
And of course
37
Applications not mentioned

• Some very important applications with significant
  on-campus programs
• computational fluid dynamics
• biological physics analyses e.g., phylogenetics
• weather forecasting
• are not mentioned because they are
  super-computer-bound computations, (possibly)
  not effectively distributable on a Grid.
• HOWEVER Dissemination of their data and
  knowledge products clearly demands Grid
  technologies data bases, remote visulatization,
  etc
• ? No field will be un-touched by The Grid.

38
At the University of New Mexico

• CERIA ? A perfect application on-campus Grid
  first, then the world
• MSB
• Sevilleta LTER
• NEON National Ecological Observatory Network
• much more!
• BIRN (Biomedical Informatics Research Network)
• UNM Center for Signal Transduction ? complete
  cell simulations
• Earth and Planetary Sciences
• HENP PHENIX and ATLAS experiment grids //
  accelerator simulations
• The Deep Lens Survey National Virtual
  Observatory (NVO) LOFAR
• The UNLV connection nuclear waste remediation
  studies
• The NCSA Alliance and Access Grid relationships
• and more!

39
Grid Application Processing requirementsQuestion
s to ask

• How much raw computing (CPU) horsepower?
• What kind of computing (e.g., massively parallel
  or not)
• What are the data base parameters
• number and complexity
• geographic and administrative distribution
• data acquisition, population, and access rates
• ? Network QoS requirements
• total active size
• backup requirements
• Other QoS requirements
• Vendor-supplied or in-house service?

40
Processing requirements

• Two examples
• Example 1 High-energy Nuclear Physics
• 10s of petabytes of data per year
• 10s of teraflops of distributed CPU power
• Comparable to todays largest supercomputers

41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
CC-F
50 (300) TB/yr
CC-J
10 TB/yr
BNL
analysis, simulation, and reconstruction of
simulated data
analysis and reconstruction
5 TB/yr
10 TB/yr
WI
UNM simulation farm
off-site analysis
51
And Future Projects will be EVEN BIGGER!
52
Processing requirements

• But this is just the beginning!
• Example 2 data mining in the biosciences

53
(No Transcript)
54
Biological Databases Complex interdependencies

• Domino-effect in data publishing
• Efficiently keep many versions

GERD
TRRD
EpoDB
BEAD
BEAD
Swissprot
Swissprot
GAIA
EMBL
GenBank
Transfac
Transfac
DDBJ
(Yong Zhao, University of Chicago)
55
Data Mining Example
56
The Future
57
and the role of Internet 2.

• It is clear that advanced networking will play a
  critical role in the development of an intergrid
  and its eventual evolution into The Grid
• Broadband capacity
• Advanced networking protocols
• Well-defined, finely graded, clearly-costed high
  Qualities of Service

58
(No Transcript)
59
(No Transcript)
60
(No Transcript)
61
Connectivity of the web one can pass from any
node of IN through SCC to any node of OUT.
Hanging off IN and OUT are TENDRILS containing
nodes that are reachable from portions of IN, or
that can reach portions of OUT, without passage
through SCC. It is possible for a TENDRIL
hanging off from IN to be hooked into a TENDRIL
leading into OUT, forming a TUBE -- a passage
from a portion of IN to a portion of OUT without
touching SCC.
62
(No Transcript)
63
In other wordsbarely predictable

• But no doubt inevitable, disruptive,
  transformative,
• and very exciting!