Components of Grid Development

1
Overview

• Components of Grid Development
• Middleware
• Resource Integration
• Connectivity
• Acceptance of Grid in the Community
• Sustainability

2
National frame

• Two pillars of D-Grid concept
• Communities
• DGI
• Problem Concept of Grid is still not widely
  percieved as the way of integrating all resources
  into a collaborative framework for scientific
  work
• Example discussion about german Supercomputer

3
Components of Grid Development

• Middleware
• Complementary development and extention of
  Middleware stacks from DGI and Communities
• Starting point is (mostly) Globus 4.x
• Open Source based and oriented
• Integrative towards community specific tailored
  middleware (based on globus) (HEP) und
  specialized middleware (Unicore)

4
Components of Grid Development

• Resource integration
• 1 Level National integration
• Unified Certification Policies
• Building of an efficient certification chain
• for users
• for services
• Service Certification is still not sufficient
• for hardware (Hosts) it is sufficient
• for Services the Certification (Webservices,
  Compute-Services etc) no unified system
• different approaches not integrated (Shibboleth)
• VO Management still not available on national
  level

5
Components of Grid Development

• Resource Integration
• Level 1 National Integration
• Obstacles
• Resource-providers in scientific community face
  legal ond procedural obstacles which hamper the
  grid way provding resources with Vos, especially
  the SCC
• Community-Resources are more flexible in that
  aspect
• In the long run we need a change of these
  procedures and substitute them with grid-based
  usage policies (establishing new comittees for
  eg. allocating resources to a science project)

6
Components of Grid Development

• Resource Integration
• Level 2 Grid-Middleware
• Accounting and Billing procedures are still not
  available
• Reasons
• different community procedures and requirements
• software for these procedures still in concept
  phase
• resource integration into a functional grid only
  now comes into being
• Job monitoring and quality assessments for QoS
  agreements for resource usage is one of the
  crucial steps in the next year, as a base for
  even raising the question of resource allocation
  procedures toward grid ways
• BMBF sponsored considerable hardware resources,
  which provide a great opportunity for progress,
  since there are no legal (and local) restrictions
  imposed

7
Components of Grid Development

• Connectivity
• The Grid is hungry for bandwidth
• Useability of grid integrated resources is very
  limited without availability of sufficient
  network connectivity
• Requirement for scientific communities is
• Centers of Grid development are the nodes of DGI
  amd Communities
• HEP-Community already has a network concept for
  the LHC-Experiment
• at least connect all BMBF hardware nodes with
  10GBit/s
• (Example BaWü for science institutions a
  countrywide Gbit network without fees)

8
Components of Grid Development

• Connectivity
• For QoS of grid services with respect to network
  connectivity no middleware stack has any
  component to offer
• Without incorporating QoS for network
  connectivity accounting/billing is only partially
  possible

9
Components of Grid Development

• Improving of acceptance of the Grid is one of the
  most important tastks of the Community grids
• Getting scientific communites to use the grid
  infrastructure for their day to day work is one
  of the most important contibutions towards the
  sustainability
• Grid integration of huge projects is a pilot
  towards this goal
• Example LOFAR
• Building Institute resource grids and VOs,
  establishing resource sharing on that level is
  one of the roads
• Getting more associated partners into Community
  projects by offering support for middleware
  installation to get them started
• Collaboration on international level with other
  projects within the communities
• Example IVOA / GAVO
• Example GAIA
• Example Ukraine Astrophysics Grid

10
Resource Integration Next steps

• Get the resources in a useable state
• Deploy our VOMRS on all resources, have all local
  RA use the tool
• Ensure a regular update of the gridmaps on all
  grid resources
• Run a daily health-checker on each resource
  thats in MDS to report the running services, the
  state of the local gridmaps and if possible, the
  gridusers and their jobs
• Start an issue tracking module for the resource
  providers to report on problems with the grid
  machines, establish a procedure for resolving
  conflicts with gridjobs and gridusers
• Draft a set of policies for grid resource usage
• Basic policy all AstroGrid-D grid resources are
  available to all our VO members
• Exceptions are ok, but only for specialized
  resources (eg dont try to run a taskfarming job
  on a cluster, or a program without gravitational
  interaction on a cluster with GRAPE boards)
• Local users run with a higher priority than grid
  users

11
Resource Integration Next steps

• Special for clusters
• Incorporate more resources from AstroGrid-D,
  especially the pending clusters
• Start to define default libraries and
  environments for the clusters (mpirun etc),
  document standards for the different local
  queue-managers and policies
• Provide above information via commandline
  facilities as well as hardware info etc,
  incorporate queue status information
• Special for workstations
• Provide owners (main user) with a means to
  suspend grid jobs for some time and bar starting
  new ones if this state is signaled
• Get this suspended status communicated to the
  information service
• Add additional security related monitoring of
  globus ports wrt security and attacks

12
Resource Integration Next steps

• BMBF Hardware for our Nodes
• For ZAH and Potsdam already agreed
• Installation of same OS (Scientific Linux 4.3 or
  4.4)
• Source installation of Globus 4.03
• Start an admin group for these resources across
  institute borders
• Ensure the participation of the community already
  in the planning stages of the integration of all
  middleware stacks to ensure a forward solution

13
Nachhaltigkeitsmodelle in Science und Business

• Nachhaltigkeitsmodelle Motivation
• Ziele und Erfolgskriterien
• Science Erzielung von wissenschaftlichen
  Erkenntnissen
• durch effiziente Nutzung von Resourcen, Know-How
  und Personal
• Erfolgskriterien
• Wissenschaftlicher Output, Publikationen,
  Grundlagen für neue Technologien
• Business Erzielung von Gewinn
• durch effiziente Nutzung von Resourcen, Know-How
  und Personal
• Erfolgskriterien
• wirtschaftlicher Erfolg, Gewinn

14
Nachhaltigkeitsmodelle in Science und Business

• Nachhaltigkeitsmodelle Möglichkeiten der
  Zusammenarbeit
• Motivationen für die Entwicklung von
  grundlegenden Verfahren und Technologien
• Science neue Verfahren werden entwickelt, um in
  einem bestimmten Gebiet die Erkenntnismöglichkeite
  n zu erweitern
• Business neue Verfahren werden entwickelt, um
  Konkurrenzvorteile und höhere Gewinnchancen zu
  erlangen
• Nachhaltigkeit bedeutet in beiden Kontexten
• langfristige Planung der Entwicklungen
• Science Grossprojekte haben eine Planungs und
  Laufzeit von Jahrzehnten,
• wesentlich bestimmt von Science-Output
• Business eine längerfristige Entwicklung ist
  wesentlich von der Kapitalumschlagszeit / ROI
  und anderen wirtschaftlichen Gesichtspunkten
  bestimmt (Zeiträume von lt5 Jahren)

15
Nachhaltigkeitsmodelle in Science und Business

• Nachhaltigkeitsmodelle Möglichkeiten der
  Zusammenarbeit
• Beispiel für eine mögliche Zusammenarbeit im
  Grid-Kontext
• Wissenstransfer durch Zusammenarbeit mit
  Hardware-Lieferanten bei der Installation von
  Default-D-Grid-Middleware auf kleinen und
  mittleren Clustern für Industrie und Wissenschaft
• Entwicklung von Methoden zur Bewertung von QoS
  des Grids
• Entwicklung von Montioring/Accounting im
  Grid-Kontext

16
(No Transcript)