9th%20Annual%20Performance%20Report%20Scorecard%20%20AGENDA%20Welcome%20%20%20%20%20%20%20Hon.%20Maurice%20McTigue%20Study%20Results%20%20%20%20%20%20%20Dr.%20Jerry%20Ellig%20Henry%20Wray%20%20%20Remarks%20%20%20%20%20%20%20Hon.%20Jim%20Cooper%20AwardsHon.%20Jim%20Cooper

1
9th Annual Performance Report Scorecard
AGENDA Welcome Hon.
Maurice McTigue Study Results
Dr. Jerry Ellig Henry Wray Remarks
Hon. Jim Cooper Awards Hon.
Jim Cooper Hon. Maurice
McTigue QA Panel of AuthorsFull text
www.Mercatus.org/Scorecard
2
Project Design

• Team of 3 experts with experience in govt. and/or
  performance management evaluates reports
• 12 criteria based on GPRA requirements
• Evaluate reports from 24 CFO Act agencies
• Each evaluation reviewed by a member of advisory
  panel
• Entire report reviewed by entire advisory panel

3
A caveat

• This Scorecard evaluates only the quality of
  agency reports, not the quality of the results
  they produced for the public. Actual agency
  performance may or may not be correlated with
  report rankings in this Scorecard.

4
How we score the reports 1-5 rating scale
5 Sets a standard for best practice
4 Shows innovation and creativity
3 Satisfactory
2 Partially complete
1 Fails to meet expectations

• 3 Categories
• Transparency
• Public Benefits
• Leadership
• 4 criteria in each category
• Criteria tightened each year to reflect previous
  years best practices
• Total score can range from 12 to 60

5
Fiscal 2007 pilot format

• Financial report (November)
• Performance information published with
  congressional budget justification (Feb. 1)
• Highlights document (Feb. 1)

6
What did the research team examine?

• Pilot format
• Required highlights document
• Other materials clearly identified by the
  highlights document (eg, Performance Reports,
  Financial Reports)
• Traditional format
• Performance and Accountability Report
• Optional highlights document (for effect on
  readability criterion)

7
Final Four Become Top Three
FY 2007 Rank FY 2007 Score FY 2006 Rank FY 2006 Score
Transportation 1 55 1 53
Labor 2 53 2 51
Veterans 3 51 2 51
8
Big Movers
FY 2007 Rank FY 2007 Score FY 2006 Rank FY 2006 Score
HHS 5 37 24 25
DHS 5 37 21 30
State 18 31 4 50
USAID 14 32 5 42
Defense 24 17 16 32
9
Lower overall scores
10
Substantial room for improvement
11
More of budget covered by satisfactory disclosure
12
Anybody can do it!
13
Pilots vs. PARs, fiscal 2007
PAR average score 37.33
Pilot average score 30.00
Difference 7.33
Difference 24
14
Pilot vs. PAR scores, fiscal 2006-07
PAR Pilot
2006 37.73 34.11
2007 37.33 30.00
Change -0.4 -4.11
Change -1 -12
15
The Pilot and Public Disclosure

• Performance information released in February
  rather than November.
• Once released, the information was harder to find
  and use.
• Little additional information that was not
  available last November.
• Well-done highlights documents (mainly those done
  voluntarily by non-pilot agencies) add
  significant value for lay readers.

16
Availability of Performance Information

• Highlights for all 9 agencies (2 not timely
  posted on line).
• Highlights, performance report, and financial
  report found for only 2 of 9 pilot agencies.
• 3 of 9 performance reports useable.
• 3 could not be found by due date
• 3 embedded in budget justifications

17
Observations

• Highlights and links to other documents must
  improve substantially in order for the public to
  get any value from performance reporting under
  the pilot format.
• Highlights should be improved and expanded
  whether or not the pilot continues.
• Scorecard offers Top Ten suggestions for
  improving the highlights documents.

18
For more information

• Mercatus Scorecard web page
• www.mercatus.org/scorecard