NASAs Earth Science Data Systems Standards Endorsement Process

1
NASAs Earth Science Data Systems Standards
Endorsement Process

• July 03, 2006
• Richard Ullman
• Ming-Hsiang Tsou
• Co-chairs

2
Motivation

• One initiative after another has stressed the
  need for interoperability standards.
• Many standards initiatives, both formal and grass
  roots have put forward specifications or
  demonstrated various ways to enable access to
  data.
• NASA, or NASA funded projects are often in the
  forefront of these activities.
• However, NASA participation in a standards
  development activity does not imply that NASA
  projects endorse the results of that activity.
• Need a way to identify the standards that work
  in the context of NASAs research and
  applications data systems under the operational
  loads that NASA typically experience.

3
Standards Role in Achieving NASAs Earth Science
Data Systems Needs.

• Future Data Systems Features (January 2004)
• Selection and management will emphasize
  flexibility and accountability over
  centralization.
• More distributed geographically, functionally and
  managerially.
• Ability to add new data system components,
  independently developed and independently managed
  without unduly perturbing existing systems
• Responsiveness to defined communities Innovation
  to serve new community needs encouraged.
• Services to broad community.
• Diversity in implementation will be encouraged -
  with coordination at the interfaces.

4
The Standards Process Group (SPG)

• History Starting in January 2004, NASA
  instituted a set of internal working groups to
  develop ongoing recommendations for evolution of
  Earth Science Data Systems development and
  management within NASA. One of these Data
  Systems Working Groups is called the Standards
  Process Group.
• Goal Facilitate broader use of data standards
  that have proven implementation and operational
  benefit to NASA Earth science.
• This is a new strategy for standards at NASA
• Grass-roots rather than top-down.
• Only after practices have been shown to (1) have
  demonstrated implementation and (2) benefit to
  operation will they be endorsed for preferential
  use.

5
Insights from SEEDS Analysis

• Interoperability does not require homogeneous
  systems, but rather coordination at the
  interfaces.
• Management can judge success based upon program
  goals rather than dictate solutions.
• example degree of interoperability rather than
  use of particular data format.
• Communities of practice have solutions.
• Published practices that demonstrate benefit can
  grow
• successful practice in specific community
• broader community adoption
• community-recognized standards

6
The Request For Comment Process

• Modeled after example of Internet IETF RFC.
• Tailored for responsiveness to NASA.
• Proposed standards are documented as
  specifications acording to SPG guidelines and
  submitted by practitioners within the NASA
  community.
• The Standards Process Group forms a Technical
  Working Group (TWG) to coordinate evaluation.
• What does implementation of this specification
  mean in the context of NASA Earth Science Data
  Systems?
• What constitutes successful operational
  experience?
• The community is invited by means of email
  announcement to comment on the specification and
  particularly to address questions formulated by
  the TWG.
• The TWG also identifies key stakeholders that are
  likely to have particular experience with the
  technology and solicits their opinion.
• The TWG reports to the SPG and the SPG makes
  recommendations for final status of the RFC.

7
The Three Step Standards Process

• Initial Screening
• Initial review of the RFC
• Provide RFC submission support
• Form TWG set schedule

RFC
Community
Core
Proposed Standard
Community
Core
Review of Implementation
Stakeholders
Evaluate Implementations
TWG
Evaluate Implementations and Community Response
SPG
Recommendation
Draft Standard
Community
Core
Review of Operation
Stakeholders
Evaluate Implementations
TWG
Evaluate Implementations and Community Response
SPG
Standard
Recommendation
Community
Core
8
Responsibilities

• Community Leader
• Identify someone in their community who will
  document standard according to SPG guidelines.
• Work with the community to get an extended review
  of the proposed standard.
• SPG
• Assign RFC editor to advise on RFC document.
• Publish and publicize RFC
• Assign TWG, technical working group to organize
  community review and evaluate responses.
• Recommend action to NASA HQ.

9
Kinds of Practices Suitable for SPG

• Any data system practice that increases
  interoperability or interuse of data within a
  community or among communities.
• Standard - Documents Operational Use
• Tech Note - Builds community awareness
  sometimes a precursor to a standard
• Examples
• Describe science content (e.g. Content standard
  for a level-2 precipitation product, surface
  reflectance product content)
• Describe interface (e.g. Data Access Protocol,
  Web Map Server)
• Describe metadata (e.g. DIF, ECHO)
• Describe File Format (e.g. HDF, GeoTIFF)
• Best Practices (e.g. File naming conventions,
  data management procedures)

10
Successful RFCs will have

• At least two implementers.
• Demonstrated operational benefit.
• Strong community leadership to support and use
  standard
• Leadership in generating the RFC.
• Community willing/able to review
• Potential for impact and spillover to other
  communities

11
Benefit of publishing through the SPG

• Benefit to NASA data systems of community
  endorsement
• NASA Earth science data management can rely on
  standards to achieve highest priority
  interoperability.
• Science investigators are assured that standards
  contribute to science success in their
  discipline.
• Benefit to community/project that proposes RFC
• Encourage consensus within the community.
• Grows use of common practices among related
  activities.
• Wider discipline community learns from successful
  practice.
• Lowers barriers to entry and use of NASA data by
  external discipline communities within NASA and
  outside NASA.
• Accelerate evolution of practices through
  better communication.
• From successful practice in specific community
• To broader community adoption
• To community-recognized standards

12
Words from OpENDAP Group

• Excerpt from James Gallaghers (OpENDAP Group)
• A small group cannot develop a high quality
  specification without external review. The SPG
  provides a critical service because the process
  is too expensive for such a group.
• By choosing to vet and publish standards for
  established technologies, the SPG standards carry
  extra weight. It is likely they will (continue
  to) be implemented and adopted.
• Having a high quality document benefits
  development teams (reducing costs due to errors,
  miscommunication, et cetera).
• The nature of the standards increase the
  likelihood of more independent implementation,
  which will strengthen the individual
  implementations and lead to high quality
  (demand-based) improvements.
• Standards are notorious on several levels this
  process has gone very smoothly.
• The OPeNDAP Board or Directors singled this
  activity out as one of the most important for the
  past year. They felt that the benefits were well
  worth the (low) costs.

13
RFCs Status

• RFC-004 Data Access Protocol (DAP) 2.0 The SPG
  reviewed community use of DAP (a.k.a. OPeNDAP)
  and found that the protocol has multiple
  interoperable implementations and has
  demonstrated operational effectiveness. The SPG
  has recommended that NASA endorse DAP as a
  community standard.
• RFC-005 Web Map Service Implementation
  Specification (WMS) v1.3 The WMS is an Open
  Geospatial Consortium standard. This RFC makes a
  case for WMS use by NASA data systems. Comments
  received on implementation indicated that NASA
  data systems did not have sufficient experience
  with Version 1.3 implementation to make a
  recommendation for WMS 1.3 to be considered a
  NASA community standard. The RFC was therefore
  published as a Technical note
• RFC-006 WMS V1.1.1 NASA stakeholders have have
  good experience with v1.1.1 implementation of
  WMS. The RFC is now out for comment on use of
  WMS 1.1.1 in an operational environment.
• RFC-007 HDF 5.0 The underlying library and
  format of the EOS Aura standard products. This
  RFC is now out for comment on implementation
  experience.
• RFC-008 HDF5-EOS The particular profile of HDF5
  used by Aura standard products. This RFC is now
  out for comment on implementation experience.

14
In work Candidate RFCs Status

• GeoTIFF Members of the geoTIFF community have
  expressed a desire to use the NASA process for
  publication of a standard geoTIFF specification.
  Author identified but progress slow because of
  lack of funding.
• Aura DSWG data product standards The data
  systems working group for the Aura mission
  specification for mission standard products.
  Publication of this as an RFC will enhance public
  and scientific use of these products. Author
  identified, but presently stalled.
• ECHO NASAs Earth science data metadata
  clearinghouse has a series of APIs for data
  providers, client developers and service
  providers. Publication of these APIs as a
  specification is proposed as technical notes. Not
  a present priority of ECHO development.

15
Potental RFCs

• To be discussed at the July 2006 meeting
• ESMF (Earth Systems Modeling Framework)
• SWEET (Semantic Web for Earth and Environmental
  Terminology)
• Marine Geoscience Data Management System
• International Institute for Climate and Society
• netCDF (network Common Data Format)

16
Earth Science DSWG Standards Process

• Facilitate Data Systems Interoperability and
  Data Interuse.
• Benefit to NASA data systems of community
  endorsement
• NASA Earth science data management can rely on
  standards to achieve highest priority
  interoperability.
• Science investigators assured that standards
  contribute to success in their discipline.
• Benefit to community/project that proposes RFC
• Encourage consensus within the community.
• Grows use of common practices among related
  activities.
• Wider discipline community learns from successful
  practice.
• Lowers barriers to entry and use of NASA data by
  external discipline communities within NASA and
  outside NASA.
• Accelerate adoption of practices through better
  communication.
• From successful practice in specific community
• To broader community adoption
• To community-recognized standards

• The Process
• Modeled after example of Internet IETF RFC.
• Tailored for responsiveness to NASA.
• Proposed standards are documented as
  specifications acording to SPG guidelines and
  submitted by practitioners within the NASA
  community.
• The Standards Process Group forms a Technical
  Working Group (TWG) to coordinate evaluation.
• What does implementation of this specification
  mean in the context of NASA Earth Science Data
  Systems?
• What constitutes successful operational
  experience?
• The community is invited by means of email
  announcement to comment on the specification and
  particularly to address questions formulated by
  the TWG.
• The TWG also identifies key stakeholders that are
  likely to have particular experience with the
  technology and solicits their opinion.
• The TWG reports to the SPG and the SPG makes
  recommendations for final status of the RFC.

• Status of Requests for Comments
• July 03, 2006
• RFC-004OPeNDAP recommended for endorsement -
  2005
• RFC-005 WMS 1.4 NOT recommended for endorsement
• RFC-006 WMS1.1.1 Open for comment on
  operational experience.
• RFC-007 HDF5 Open for comment on implementation
  experience.
• RFC-008 HDF5-EOS Open for comment on
  implementation experience.
• geoTIFF Interest from community, waiting for
  RFC
• Aura DSWG Interest from community, no one
  available for RFC
• ECHO APIs Some interest from project, not
  priority.
• ESMF, SWEET, netCDF In conversation with
  communities.