Open Science Grid Governance

1

• Open Science Grid Governance

Open Science Grid Workshop Harvard University
Cambridge, Mass. Sep. 9-10, 2004
Governance Parallel Session Summary
2
Governance

• ? What are
• The Charter
• The organization (bodies, groups)
• Definitions and By-laws
• Agreements
• Formal processes
• Needed to make OSG successful technically,
  scientifically, funding-wise and in achieving its
  broader goals
• Goal 1 - to have just sufficient top down
  coordination to ensure success
• Goal 2 to establish a decentralised economic
  model as a means for resource (people and
  hardware) sharing and allocation (as opposed to a
  central resource allocation mechanism)
• Goal 3 to ensure we are able to make a
  significant step with OSG (not just a lowest
  common denominator effort) opportunity for
  global impact, high impact e.g. in education and
  outreach, if done well
• Goal 4 - to adopt a model that the funding
  agencies will understand and be comfortable with
• (Note some of these goals may be conflicting and
  pulling us in different directions a compromise
  position is necessary)

3
Governance Parallel Group methodology

• Consider governance-related issues that need to
  be decided. By this we mean
• Issues that would be decided and clarified if we
  had written governance documents
• Issues that illustrate that formal bodies, rules,
  definitions, processes, etc. are necessary
• Consider what documents (or fragments of
  documents) should be written to address these
  issues and to clarify Governance, i.e. lay out
• The Charter
• The Organization
• Definitions and By-laws
• Agreements
• Formal processes
• In sub-groups work on each of the documents
  needed

4
Governance-related issues

• Does OSG itself obtain or own resources?
• If so it must have some management body
• Even if OSG doesnt have any resources it may
  have to act as if it has some
• Also implies some type of management body
• Who can be a member of OSG? Institutions? VOs?
• Who can propose to provide something or make a
  proposal for funding to add something to OSG?
• How does OSG interact coherently with other Grids
  and funding agencies?
• How do we ensure that there are responsible
  parties in OSG and that issues and interactions
  get resolved in a timely way?
• Make it obvious who to go to for what issue
• How does OSG optimize coherence and technical
  decisions?
• Are members of OSG in US only or international?
  What partnerships does OSG have formally?

5
Governance-related issues (2)

• Who decides who can use OSG resources each
  site?
• How does a stakeholder get assurance that
  investment in OSG activities will be a solid
  investment and decisions will stand?
• How does a stakeholder know the longevity of
  services and infrastructure.
• VOs MOU with their service provider to deal with
  this?
• Who can deploy new services? When can sites
  upgrade?
• How do you ask a member to leave?
• Is commercial use of OSG resources OK?
• e.g. in collaboration with company?
• How do we Enforce rules and validate status of
  members?

6
Governance-related issues (3)

• What is the minimum buy-in to bring resources and
  services?
• Who decides what else you need to bring to the
  the table to join
• MOUs?
• Are there any MOUs involving OSG?
• Who decides when a new version is used or when a
  goal is to be met?
• Any fees to be a member?
• Are there any OSG staff?
• Is it staffed entirely with Voluntary
  contributions?
• Are there some core shared services/activities
  that are OSG only, not provided by members?

7
Documents or document fragments needed

• OSG Charter document
• What is OSG
• What can OSG commit to (if anything)
• How does it work as an economic marketplace
• What is the framework for sharing
• Resource Allocation model
• The organization (bodies, groups)
• Players in OSG and their roles
• Stakeholders, Consumers, Providers, VOs,
  Institutions, Sites, Facilities, etc.
• Roles of a VO and of a VOFacility pair
• Special relationship and responsibilities of a VO
• Relationship diagram between all the players and
  OSG
• List of management bodies, boards, steering
  groups, decision groups
• Which are they and what does each do
• How are they formed, who are the leaders
• Elections and voting mechanisms
• Definitions and By-laws
• Definition of a member and the mechanisms to join
  OSG
• Definition of a Partnership with other Grids

8
Documents or document fragments needed (2)

• Agreements
• Site charter and agreement that you sign as an
  institution/member.
• Description of minimal set of services that must
  be running or other features to be an OSG
  participant
• MOUs if they exist? Other agreements or sets of
  rules
• Formal processes descriptions
• How membership actions are taken
• Selection of various board members
• Approval processes e.g. for submitting proposal
  for OSG,
• Process for forming a self-organized Technical
  Group
• Process for deciding needs and requesting funding
  for OSG as an entity

9
Governance-related issues

• Does OSG itself obtain or own resources?
• If so it must have some management body (1), (2d)
• Even if OSG doesnt have any resources it may
  have to act as it it has some (1), (2d)
• Also implies some type of management body
• Who can be a member of OSG? Institutions? VOs?
  (4a), (2a), (2c)
• Who can propose to provide something or make a
  proposal for funding to add something to OSG?
  (2a), (2c), (1), (2d), (3a)
• How does OSG interact coherently with other Grids
  and funding agencies? (2d), (3b)
• How do we ensure that there are responsible
  parties in OSG and that issues and interactions
  get resolved in a timely way? (2d), (5a)
• Make it obvious who to go to for what issue
• How does OSG optimize coherence and technical
  decisions? (2d)
• Are members of OSG in US only or international?
  What partnerships does OSG have formally? (1),
  (3a)

10
Governance-related issues (2)

• Who decides who can use OSG resources each
  site? (5a)
• How does a stakeholder get assurance that
  investment in OSG activities will be a solid
  investment and decisions will stand? (2b)
• How does a stakeholder know the longevity of
  services and infrastructure.
• VOs MOU with their service provider to deal with
  this? (2d)
• Who can deploy new services? When can sites
  upgrade? (4a)
• How do you ask a member to leave? (3a), (4a)
• Is commercial use of OSG resources OK?
• e.g. in collaboration with company? (3a)
• How do we Enforce rules and validate status of
  members? (2d)

11
Governance-related issues (3)

• What is the minimum buy-in to bring resources and
  services?
• Who decides what else you need to bring to the
  the table to join (2d)
• MOUs?
• Are there any MOUs involving OSG? (4a), (4b)
• Who decides when a new version is used or when a
  goal is to be met? (1). (2d)
• Any fees to be a member? (1), (2d)
• Are there any OSG staff?
• Is it staffed entirely with Voluntary
  contributions?
• Are there some core shared services/activities
  that are OSG only, not provided by members? (1),
  (2d)

12
Subgroups worked on

• Group 1 - Doc 1 - Charter
• (I. Gaines, P. Rosen, L. Price, M. Tuts)
• Group 2 - Docs 2a,2b,2c The
  members/stakeholders and their relationships (and
  related parts of Doc 5 formal process)
• (R. Mount, L. Bauerdick, J. Shank, J. McNabb)
• Group 3 - Doc 2d Organization (and related
  parts of Doc 5)
• (F. Wuerthwein, B. Gibbard, V. White)
• Group 4 - Docs 3a, 3b 4a, 4b -- Definitions
  and site or member agreements and other MOUs if
  needed
• (P. Avery, J. Ibarra, A. Blatecky)

13
Group 1 - Charter of OSG

• Increasing scale and complexity of 21st century
  science has led to larger and more global
  collaborations involving massive data sets. This
  requires efficient utilization of widely
  distributed computational resources and effective
  global communication. Among the sciences that can
  immediately benefit from global computing grids
  are high energy physics, nuclear physics,
  astrophysics, computational biology, and more
  ....
• The vision of the Open Science Grid Consortium
  (OSG) is one of a persistent production national
  grid infrastructure for large scale US science
  the Open Science Grid.
• The OSG will provide a set of goals and an
  overall infrastructure within which the Grid
  resources of the different members can be
  operated coherently and compatibly.
• The OSG is a consortium of scientific
  collaborations, scientific computing centers and
  existing and new grid research and deployment
  projects, involving both computational and
  application scientists, working together to
  provide and support the set of facilities,
  services and infrastructure needed.

14
Charter (continued)

• OSG will require a dedicated staff and other
  resources (which could be paid for by member
  contributions or eventually by direct funding).
• A structure of management and coordination bodies
  will oversee and coordinate the work of OSG.
• The OSG will be open to all sciences that have a
  need for distributed large scale computing and
  data management, and can bring resources to be
  federated.
• A fully functional and production quality grid
  will be built and operated in the US, extending
  internationally to create large scale global
  grids for LHC science, Run II, Babar, RHIC,
  Astrophysics, Computational Biology and others.
  (???)
• OSG will create opportunities for educators and
  students to participate in building and
  exploiting this grid infrastructure and
  opportunities for developing and training a
  scientific and technical workforce. It has
  potential to transform the integration of
  education and research at all levels.

15
Charter (continued)

• The Open Science Grid will ensure that the U.S.
  plays a leading role in defining and operating
  the global grid infrastructure needed for
  large-scale collaborative and international
  scientific research. The Open Science Grid will
  provide a set of services that can be enriched as
  new science areas choose to join and federate
  their resources. For the first time combined
  computing resources at several national labs and
  at dozens of universities will effectively become
  a single national computing infrastructure for
  science, the Open Science Grid.

16
Open Science Grid Partnerships
17
Group 2 - The members/stakeholders/players and
their relationships

• Users (aka Virtual Organizations)The science
  done by users is the goal of OSG
• Resource ProvidersHardware and/or effort
• Collaborating Projects and GridsPPDG, GriPhyN,
  iVDGL, SRM, TeraGrid, EGEE,
• Funding AgenciesDOE, NSF, NIH, NASA,
• Workers
• OSG must have mechanisms to interact directly and
  effectively with all constituencies
• Constituencies 1,2 and 3 have massive overlap,
  but this is NOT a reason to assume that one can
  be represented by another
• Potential members of 1,2,3 and 4 must also have
  well-defined paths to interact with OSG

18
OSG Constituencies clarifications?

• Workers are members of either a VO, a Resource
  Provider or a Collaborating Project
• Not explicitly addressed in governance documents
• All associated people/workers need access to
  information e.g. mail list for all participants
  and representation

19
Membership of the OSG Consortium Board (or OSG
Council?)

• Resource providers, or groups of resource
  providers, bringing more than a threshold
  annualized value of effort and/or hardware will
  be entitled to representation on the OSG Council
• Groups of users (Virtual Organizations) using or
  controlling more than a threshold annualized
  value of OSG resources will be entitled to
  representation on the OSG Council
• Collaborating Projects may be invited to be
  represented on the Collaboration Council by the
  existing Council members.

20
Group 3 - Organization

• Started from Suggested Organization shown by Paul
  in the morning and evolved a bit
• Basically two types of groups/boards
• Self organized and technical
• Formal management and representation boards

21
Possible OSG Collaborative Framework
Advisory Committee
Universities,Labs
Service Providers
Oversight Group VOs?
Technical Groups 0n (small)
Sites
Collaboration Board Institutions?
Researchers
VOs
Joint committees (0N small)
Research Grid Projects
Enterprise
Participants provide resources, management,
project steering groups
22
New suggested OSG Governance bodies
Advisory Committee
Technical Groups 0n (small)
Universities,Labs
Service Providers
Executive Board (8-15 representatives Chair,
Officers)
Sites
Researchers
VOs
Research Grid Projects
OSG Council (all members above a certain
threshold Chair, officers)
Core OSG Staff (few FTE, manager)
Enterprise
Participants provide resources, management,
project steering groups
23
Roles of bodies

• Executive Board
• Resolves conflicts among activities whenever it
  threatens the OSG as a whole
• Proposes Core Staff needs and gets them
  satisfied by one of
• Contributions from stakeholders
• Sponsoring proposal from subset of stakeholders
• Getting OSG-owned funding and staff from funding
  agency
• Appoints Core Staff Manager(s)
• Consists of a smallish group of major
  stakeholders and/or people with strong commitment
  to understand all of OSG
• Membership changes over time elected, mechanism
  defined in by-laws
• Recognition of Technical Groups
• Recognition, oversight and coordination of
  Activities
• Works to find/allocate resources for TGs and
  Activities

24
Roles of bodies

• OSG Council (or Consortium Board? )
• Analagous to Experiment Collaborations
  Institutional Board
• Makes membership decisions
• Approves changes to by-laws
• Approves changes to the charter
• Approves the Acceptable Use Policy and other
  policies and rules
• Approves/sets up procedures for selection of
  Executive Board

25
Roles of Bodies

• Core OSG Staff
• Provides work needed for coherence and process
  management
• May provide some operational work not contributed
  by stakeholders
• May provide some leadership for the desired
  engineered and managed aspects of OSG
• May provide validation, testing?
• May be funded by stakeholder contributions
• May be funded through direct funding to OSG
  Consortium

26
Group 4 Definitions and Agreements(very much a
work in progress still)

• Definition of a Member and mechanisms to join
• Joining should not be easy
• Avery emphasized its very labor intensive
• Managing expectations
• Provides Resources
• What are valid resources?
• testing groups
• development groups
• training education groups
• not a clear algorithm negotiated people helping
  are resources labor
• mechanism need a clear algorithm
• Provides explicit description of resources
• Agrees to play by OSG rules
• Should there be classes of members? Associates?
• Full decision making. in committees (1)
• General (2)
• observer status (3)
• VO manages resources of its sites
• Operations team has authority to shut down VO or
  single site of that VO.

27
OSG membership issues?

• Special role of the VO in membership in OSG?
• Resource providers always become members through
  association with one or more VOs
• Membership of an institution may be through its
  VO (if it is say part of a group of resource
  providers for that VO)

28
Definitions and Agreements (cont)

• Partnership agreements with other Grids and
  Regions
• Management decision/operations group at OSG to be
  the partner
• need more information from a technical
  perspective
• If attempting to peer with another Grid, some
  body within OSG would make such an agreement with
  support from members,
• There are technical issues of how jobs would run.
  Need more information.

29
Definitions and Agreements (cont)

• Site charter and agreement that you sign as an
  institution/member.
• There needs to be an agreement that VO signs
• When VO joins, should provide a prospectus.
  conscious membership. Criterion for membership
  should be flexible. VO has capacity and
  experience to represent its member sites
• VO is the fundamental membership entity that
  provides information about member sites.
• does the ops team turn on site by site? Or does
  the VO turn up its own sites?
• there needs to be a scalable validation
  mechanism? Too much for Ops team?
• - there needs to be a validation team running
  tests. Does VO run test or

30
Status and Proposal for next steps

• Obviously still a lot of work to do
• Propose that subset of this group ( volunteers)
  continue the activity of writing the various
  governance documents new TG to do this?
• Circulate documents widely for comment (15 day
  comment period)
• Propose we immediately adopt the Charter
• Circulate to all on mailing list to make
  suggestions and improvements
• Propose we appoint an Interim Executive Board
  representing current major players in Grid3
  places for future
• US-CMS, US-ATLAS, Ligo, GriPhyN, iVDGL, PPDG,
  Fermilab, BNL, SLAC, NERSC, 2 or 3 reps from
  non-Physics sciences VO and/or resource providers
• Propose these players also seed the OSG Council
  by proposing Council members from their
  VO/Institution.
• Propose formal membership process for OSG and for
  seat on OSG Council be carried out