Chapter 4 Grid Computing

1
Chapter 4Grid Computing

• Phil FeibishIT 3653
• From From P2P to Web Services and Grids
• Peers in a Client/Server World
• Taylor, Ian J. Springer, 2005
• and Wikipedia Grid Computing
• http//en.wikipedia.org/wiki/Grid_computing

2
Definition Wikipedia

• Grid computing is an emerging computing model
  that provides the ability to perform higher
  throughput computing by taking advantage of many
  networked computers to model a virtual computer
  architecture that is able to distribute process
  execution across a parallel infrastructure.

3
Definition, continued

• Grids use the resources of many separate
  computers connected by a network (usually the
  Internet) to solve large-scale computation
  problems.
• Used for "big science"

4
Definition, continued

• Grids provide the ability to perform computations
  on large data sets, by breaking them down into
  many smaller ones, or provide the ability to
  perform many more computations at once than would
  be possible on a single computer, by modeling a
  parallel division of labor between processes.

5
4.1 The Grid Dream

• The Grid takes its name from an analogy to the
  electrical power grid.
• The grid dream is to allow users to tap into
  resources off the Internet as easily as
  electrical power can be drawn from a wall socket.
• Must be pervasive and provide security and
  accountancy for transparent access. These
  features will be provided by the Grid middleware.

6
Grid Dream, continued

• Grid accountancy has not been realized.
• However, a number of users and scientists have
  devoted resources to form prototypes for research
  and development.
• Many different types some evolving, some
  private, some public, some regional, some global,
  some specific, and some generic (p.58).

7
Design Goals (Wikipedia)

• Grid computing has the design goal of solving
  problems too big for any single supercomputer,
  whilst retaining the flexibility to work on
  multiple smaller problems.
• Thus Grid computing provides a multi-user
  environment.

8
Grids vs. Clusters (Wikipedia)

• Grid computing is often confused with cluster
  computing.
• The key difference is that a cluster is a single
  set of nodes sitting in one location, while a
  Grid is composed of many clusters and other kinds
  of resources (e.g. networks, storage facilities)

9
Broadband through electrical grid

• As an aside, some companies are starting to offer
  broadband Internet access through the standard
  electric power grid.
• Data is transmitted by special modems using about
  10w of power over existing electrical lines.
• Tests show that symmetrical speeds of about 1
  Mbps can be achieved.
• This is not exactly Grid computing (know why!),
  but it may be a first step in a general
  implementation.

10
4.2 Social Perspective

• Simply solving technological problems of the Grid
  is not sufficient.
• We must also address the social aspects of how
  to engage researchers, educators, businesses, and
  consumers in using the Grid as part of their
  everyday work. (Taylor, p58).
• the social good of any new infrastructure has
  wide social implications, i.e. if those
  investing in this new technology do not see an
  economic return then the progress will slow down
  and consequently, so will its widespread
  availability. (Taylor, p58)

11
Social perspective

• Therefore, the acceptance of the Grid is highly
  dependent on social acceptance and industrial
  success if the cost-volume relations are to break
  even.
• How?
• Gain large industrial backing
• Conforming to international standards
• e.g. US National Science Foundation has
  committed 53 million to TeraGrid (p59)
• Convergence of Web Services and Grid into OGSA
  (Open Grid Service Architecture) and WSRF (Web
  Services Resource Framework)

12
History of the GridFirst Generation

• Early 1990s shift if the emphasis on wide-area
  distributed computing
• 2 representative, yet diverse experiments are
  described by Taylor which describe 1st generation
  attempts to provide an infrastructure for access
  to computational resources by high performance
  applications
• FAFNER
• I-WAY

13
FAFNER

• Factoring via Network-Enabled Recursion
• An experiment to factor RSA130 using the NFS
  factoring method.
• Used a Web interface. Contributors could take
  the form and use it to invoke CGI scripts to
  perform the factoring.
• FAFNER is basically a collection of Perl scripts,
  HTML pages and documentation, so processing was
  server-side.
• The FAFNER software didnt factor the RSA130,
  rather, it provided interactive registration,
  task assignment and solution database services to
  clients that perform the actual work.

14
I-WAY

• Started as a project to link various
  super-computing centers and to provide the
  infrastructure for a metacomputing environment
  for high computational scientific applications.
• Forerunner to the Globus toolkit.
• Connected 17 sites across North American based on
  ATM network technology.
• Used for a diverse set of applications including
  large-scale scientific simulations, collaborative
  engineering, etc.

15
History of the GridSecond Generation

• Goal of 2nd Generation provide a common
  infrastructure for Grid applications through the
  development of the Globus toolkit and Legion.
• Legion an integrated OS for Grids or
  meta-systems. Its focus is to give the user
  the impression that he is using a global virtual
  computer, which transparently handles all the
  complexity involved with having such a
  distributed system (61)
• Also included widespread adoption of distributed
  object systems including Jini and CORBA (Common
  Object Request Broker Architecture). CORBA
  defines an object-oriented model for accessing
  distributed objects.

16
History of the GridSecond Generation

• 2nd generation also added such things as Grid
  resource brokers and schedulers, transparent
  checkpointing, and migration of jobs across the
  network. (Taylor, p62).
• Checkpointing saving a jobs state to disk so
  it can be resumed at a later state, either
  locally or remotely by migrating it to another
  machine.

17
History of the GridThird Generation

• Basic Interoperability to enable large-scale
  distributed computation and sharing of resources.
• Flexible assembly of Grid resources by exposing
  the functionality through standard resources with
  agreed interpretation.
• Keywords
• Service-oriented model
• Metadata
• Distributed collaboration
• Virtual organizations / OGSA (Open Grid Service
  Architecture)

18
4.4 Grid Computing Architecture

• Def (Taylor) Grid computing is flexible,
  secure, coordinated resource sharing among
  dynamic collections of individuals, institutions,
  and resources.
• Emphasis on flexible and dynamic environment that
  can be used to discover and interoperate with
  distributed resources.
• Key notion Virtual Organizations

19
Virtual Organizations

• VOs provide a highly controlled environment to
  allow each resource provider to specify exactly
  what she wants to share, who is allowed to share
  it and the conditions whereby this sharing
  occurs.
• The set of individuals and/or institutions that
  provides such sharing rules is collectively known
  as a virtual organization.

20
Middleware

• In Grid computing, the transparent access to
  distributed resources is achieved through the use
  of middleware, such as Globus.
• The middleware toolkit (Globus) allows users to
  discover the existence of resources, make
  reservations for their use, and gain access to
  them.
• Access to the resources is achieved through
  Internet technologies such as FTP (or enhanced
  GridFTP). Routing is handled through TCP/IP.

21
VOs and sharing of resources

• VOs are dynamically accessible from a Grid
  application
• Applications are capable of spanning multiple
  different VOs.
• VOs can span multiple organizations.
• The pooling of resources at multiple sites it a
  key element in aggregating functionality that
  exposes new services to the community that could
  not have been previously achieved.
• VO must be flexible and allow mechanisms for
  users to express policies, establish identities,
  etc.

22
Different kinds of VOs

• Resource Providers
• ASPs, Storage service providers, CPU cycle
  providers, etc.
• Product design
• Industrial consortium, etc.
• Crisis management
• Data Intense Applications
• E.g. DataGrid

23
4.5 Criteria for being a Grid

• Resources are not subject to centralized control.
• Uses standard, open, general-purpose protocols
  and interfaces.
• Delivers non-trivial QoS. (Qualities of
  Service).
• Note Review this material from Wikipedia
  article.
• This is a very important slide.

24
4.6 Types of Grid

• Computational Grid
• Distributed set of resources that are dedicated
  to aggregate computational capacity
• Appropriate for large scientific searches through
  networks with little node-to-node communication
• Data Grid
• Distributed set of resources set up for
  processing and transferring large amounts of
  data. (petabytes or more?)

25
Types of Grids

• Service Grid
• A collection of distributed resources that
  provides a service that cannot possible be
  achieved through one single computer.
• For example, combining databases from two
  distinct VOs

26
4.7 The Globus Toolkit

• Middleware which enables applications to handle
  distributed, heterogeneous computing resources as
  a single virtual machine.
• A U.S. multi-institutional research effort that
  seeks to enable the construction of computational
  grids and contains a core set of services that
  aims to provide solutions for the Grid
  infrastructure.

27
4.7 The Globus Toolkit

• Consists of 4 layers (Taylor, p71)
• Layer 4 Applications
• The applications themselves hundreds exist
  (appendix A)
• Layer 3 Application Toolkits
• Portals, remote steering, data grids, remote
  visualization,
• a set of capabilities that are specific to an
  application or a set of applications
• Layer 2 Grid APIs
• Globus security, resource management,
  information services, data management, etc.
• Layer 1 Grid Fabric
• Computer storage, sensors, network and their
  associated services

28
4.7.1 Globus tools

• Security
• Provide authentication, delegation, authorization
• Information Services
• Provide information about Grid services.
• Resource station, configuration information
• Data Management
• Accessing and Managing Data
• GridFTP, Data Replication, GASS (Global Access to
  Secondary Storage)
• Resource Management
• Allocate Resources provided by the Grid
• Main component is the Gatekeeper, which
  communicates with the Job manager.
• See diagram, p76.

29
Questions?