Metrics Planning and Reporting MPAR WG

1

Metrics Planning and Reporting (MPAR)
WG Overview H. K. (Rama) Ramapriyan NASA/GSFC Pau
l Davis University of Maryland Co-Chairs,
MPARWG 6th Earth Science Data Systems Working
Group Joint Working Group Meeting College Park,
MD October 23-25, 2007
2
MPAR Working Group

• Mission Statement for the WG
• Review and recommend program-level performance
  metrics and collection tools that measure how
  well each data activity supports the NASA Science
  Mission Directorates Earth science, application
  and education programs
• Membership in WG
• WG membership open to NASA data and service
  provider community (REASoN, ACCESS and MeaSUREs
  projects, DAACs, SIPSs, etc.)
• We are open to suggestions for participation by
  others
• Scope of Work
• WG provides on-going MPAR review, evaluation,
  recommendations and metrics evolution for the
  NASA ES data and service provider community
• WG recommends additions, deletions or
  modifications to set of metrics. Recommendations
  may be approved, modified or rejected by NASA. If
  approved, NASA Science Mission Directorate funded
  Earth science data and service providers will
  have to make recommended changes in their
  reporting

3
ESE MPAR WG Rules of Operation, Continued

• MPAR WG Internal Processes
• Process to adopt recommendation
• Majority vote of MPAR WG members to adopt
  proposed recommendation as a WG draft
• One MPAR WG member appointed shepherd
• 30 day period of ESE activity review for WG draft
  (not all ESE activities are represented by MPAR
  WG members) coordinated by shepherd
• Shepherd assembles comments, drafts revisions to
  recommendation per activity feedback, presents
  summary of feedback and draft revisions to full
  WG
• WG considers revisions and need for beta test
• Majority vote of MPAR WG members to adopt revised
  WG draft
• Shepherd coordinates Impact Analysis, Rationale,
  Justification
• Two thirds vote of responding MPAR WG members to
  adopt final recommendation package and send to HQ
  / ESE.

4
Players and Roles (1 of 2)

• MPARWG - Review and recommend program-level
  performance metrics and collection tools
• H. (Rama) Ramapriyan (GSFC) and Paul Davis (UMD)
  Co-Chairs set agenda, conduct business,
  gather and forward recommendations to Kathy
  Fontaine for transmission to HQ
• Kathy Fontaine (GSFC) overall coordination of
  all ESDSWG activities, reporting to HQ
• Frank Lindsay (HQ) Review recommendations,
  advice Martha Maiden, coordinate with HQ Program
  Managers, Program Scientists, Study Managers
• Martha Maiden (HQ) Approve, disapprove or
  modify recommendations
• MPARWG members participate (vigorously) in
  discussions preceding recommendations
• Greg Hunolt, Bud Booth (SGT) support co-chairs
  in conducting business (e.g., keep minutes of
  meetings/telecons, shepherd recommendations
  through MPARWG voting process and prepare for
  approval)

5
Players and Roles (2 of 2)

• Metrics Reporting
• H. (Rama) Ramapriyan (GSFC) Lead Ensure
  collection of metrics from all activities
  (REASoN/ACCESS), development of summary reports
  required by HQ
• Paul Davis Saurabh Channan (UMD) Develop and
  maintain web-based metrics collection tool
• Kathy Fontaine (GSFC) Liaison with HQ for all
  ESDSWG activities
• Frank Lindsay (HQ) Coordinate with HQ Program
  Managers, Program Scientists, Study Managers
• Martha Maiden (HQ) Set policy, establish
  reporting requirements, liaison with OMB to
  provide overall summary reports of all NASA Earth
  science data system activities
• REASoN/ACCESS Projects Enter reports using
  metrics tools
• Greg Hunolt (SGT) Support Rama in contacting
  projects, resolving any technical issues,
  ensuring consistency, generating summary reports,
  and reporting status
• Jody Garner (ADNET) Maintain operational
  web-site to collect metrics provide technical
  help with web-site to reporting projects

6
FY 2007 Status (1 of 2)

• MPARWG Recommendation 2 developed in October
  2005 ESDSWG Workshop and approved by HQ in
  September 2006
• Implementation of changes to web site were
  completed
• Reporting in FY 07 used new baseline
• Eliminated one metric (timeliness of product
  delivery) that is no longer useful
• Included experimental service metrics as well
  as project-defined/project-unique metrics
• Metrics Collection and Reporting Tool
• Many improvements were discussed during November
  2006 MPARWG breakout sessions
• Held telecon focused on issues with impact
  metrics reporting and obtained suggestions for
  improvement
• Recommendations were prioritized into Phase 1
  through 3 phase 1 improvements have been
  completed
• Will be demonstrated Wednesday afternoon

7
FY 2007 Status (2 of 2)

• Improve collection
• Continued procedures established during the last
  two years
• Operational site at GSFC
• Monthly reminders
• Review of reports, follow-up and assistance as
  needed by Greg Hunolt
• 36 of 38 REASoN projects reporting regularly 12
  of 16 ACCESS projects have reported at least once
• Significant increase in submission of Impact
  Metrics
• 41 impact metrics from 13 REASoN projects (12
  from 4 projects at end of FY 06)
• 3 impact metrics from 2 ACCESS projects
• Rama regularly (almost) forwards one impact
  metrics each week to HQ

8
Joint WG Meeting (Wednesday) and MPARWG Breakouts

• Joint WG meeting
• Recent Challenges in Metrics
• Breakout sessions
• See agenda next page

9
MPARWG Breakout Sessions
10
BACK-UP
11
MPARWG Membership as of October 2, 2007