Results Based Accountability Basics A One Day Presentation

1
Results Based Accountability BasicsA One Day
Presentation

• Standard Training Slides Sponsored by the
  Ministry of Social Development

2
Results Based Accountability
The Fiscal Policy Studies Institute Santa Fe, New
Mexico
Websites raguide.orgresultsaccountability.com
Book - DVD Orderssheapita.co.nz amazon.comresult
sleadership.org
3
How could RBA add value to you?
4
SIMPLE STEPS COMMON SENSE PLAIN LANGUAGE MINIMUM
PAPER TALK TO ACTION!
5
Results Based Accountabilityis made up of two
parts
Population Accountabilityabout the wellbeing
ofWHOLE POPULATIONS
For Communities Cities Districts
Countries E.g. All Rangatahi/Youth in Te Tai
Tokerau, All Migrants in Nelson
Performance Accountabilityabout the wellbeing
ofCLIENT GROUPS/CUSTOMERS
For Teams - Providers Programmes - Agencies
Service Systems E.g. Clients of Services,
Collectives, Ministries or the Health System
6
The Language TrapToo many terms. Too few
definitions. Too little discipline
Lewis Carroll Center for Language Disorders
Core qualitative strategic objectives
Your made up jargon here
Measurable urgent systemic indicators
7
Results Based Accountability
COMMON LANGUAGE COMMON SENSE COMMON GROUND
8
Definitions

• RESULT / OUTCOME
• A condition of wellbeing for children, adults,
  families or communities
• All Tamariki in Hamilton are Born Healthy, Safe
  Roads, Nurturing Whanau/Families, A Prosperous
  Economy
• INDICATOR / BENCHMARK
• A measure which helps quantify the achievement of
  a result.
• Rate of low-birth weight babies, Rate of road
  crashes, Rate of child abuse and neglect,
  Unemployment rate
• PERFORMANCE MEASURE
• A measure of whether a programme, agency or
  service system is working. Three types
• How much did we do?
• How well did we do it?
• Is anyone better off? Client Results /
  Outcomes

Population
Performance
9
Translation Guide/Rosetta StoneNot the Language
Police
Ideas
Group 1
Group 2
Group 3 etc.
1. A condition of well-being for children,
adults, families communities 2. 3. etc.
RESULT
OUTCOME
GOAL
10
From Ends to Means
From Talk to Action
RESULT / OUTCOME
ENDS
Population
INDICATOR / BENCHMARK
PERFORMANCEMEASURE
MEANS
Performance
11
Results Indicators Performance Measures in
Maori, Fijian, Tuvaluan
12
Result, Indicator, Strategy or Performance
Measure?
Result Indicator Perf Measure Result Strategy Resu
lt Indicator Perf Measure

1. A Safe Community
2. Percentage of Total Recorded Offences
3. Average Police response time
4. An Educated Workforce
5. Installing street lights to make people feel safe
6. People have living wage jobs and income
7. of people with living wage jobs and income
8. of participants in job training programme who
   get living wage jobs

13
Key RBA concepts

• 2 key types of accountability and language
  discipline
• Population accountability - results / outcomes
  and indicators
• Performance accountability - performance measures
  
• 3 types of performance measures
• How much did we do?
• How well did we do it?
• Is anyone better off?
• 7 questions from ends to means
• baselines and turning the curve to make life
  better for our families / whanau, children /
  tamariki, and communities.

14
Population Accountability

• For whole populations in a geographic area

Mark Friedman (author)www.resultsaccountability.c
omwww.raguide.org
15
The 7 Population Accountability Questions

1. What are the quality of life conditions we want
   for the children, adults and families who live in
   our community? (Population Results)
2. What would these conditions look like if we could
   see them? (Experience)
3. How can we measure these conditions? (Population
   Indicators)
4. How are we doing on the most important of these
   measures? (Baseline Data and Story)
5. Who are the partners that have a role to play in
   doing better? (Partners)
6. What works to do better including no- cost and
   low-cost ideas? (What works)
7. What do we propose to do? (Action Plan)

16
16
17
Christchurch City Community Outcomes

• A safe city
• A city of inclusive and diverse communities
• A city of people who value and protect the
  natural environment
• A well governed city
• A prosperous city
• A healthy city
• A city of recreation, fun and creativity
• A city of lifelong learning
• A city that is attractive and well designed

18
Kotahitanga Whanau Ora Collective
positive statements - positive focus

• All whaanau in Te Puuaha ki Manuka(Greater South
  Auckland) are ...
• Mana Ora Healthy and safe
• Mana Motuhake Economically secure
• Mana Tangata Culturally confident
• Mana Rangatiratanga Knowledgeable and skilled
• Mana Whanau Connected, engaged and entrepreneurs

Acknowledgement Kotahitanga Collective Members
Turuki Healthcare Trust, Papakura Marae, Huakina
Development Trust and Te Kaha O Te Rangatahi
Trust, South Auckland, New Zealand.
19
Aranui Community Trust
Acknowledgement Aranui Community Trust Inc
Society (www.actis.org.nz)
20
Implementing RBA Aranui Result areas and
indicators
Result Area 1
A community that is spiritually and socially
strong Indicator of police callouts for family
violence
Result Area 2

• A community full of knowledge and learning
• Indicator
• of students with NCEA level 1 _at_year 11
• primary school students performing at national
  average for literacy and numeracy

21
Result Area 3

• A great physical environment
• Indicator
• Expenditure on repairs and maintenance to city
  property in the Aranui Burwood Pegasus area
• of R M that is due to damage

Result Area 4

• People who know and fit in Aranui
• Indicator
• of people who offer to participate in local
  events

22
Result Area 5

• A community that is healthy
• Indicator
• of Aranui residents presenting at Accident and
  Emergency with no trauma needs/ concerns
• of total acute inpatient admissions

23
Results for Children, Families and CommunitiesA
Working List

• Healthy Births
• Healthy Children and Adults
• Children Ready for School
• Children Succeeding in School
• Young People Staying Out of Trouble
• Stable Families
• Families with Adequate Income
• Safe and Supportive Communities

23
24
Georgia Policy Council for Children and
Families RESULTS

• Healthy children
• Children ready for school
• Children succeeding in school
• Strong families
• Self-sufficient families

24
25
Tip for Drafting Population Accountability Results

• All ______ in ______ are __________

Families
Tauranga
Economically Secure
Insert your Population
Insert your Geographic area
Insert your Condition of Wellbeing
25
26
Examples of Means not Ends
1. COLLABORATION 2. SYSTEMS REFORM 3. SERVICE
INTEGRATION 4. DEVOLUTION 5. FUNDING POOLS
26
27
Leaking Roof(Results thinking in everyday life)
Experience
Not OK
Inches of Water
Measure
Turning the Curve
Story behind the baseline (causes)
Partners
What Works
Action Plan 2
Action Plan
27
28
3 criteria for choosing Indicators
Communication Power
Does the indicator communicate to a broad range
of audiences?
Proxy Power
Does the indicator say something of central
importance about the result?
Does the indicator bring along the data HERD?
Data Power
Quality data available on a timely basis.
28
29
Choosing IndicatorsWorksheet
Safe Community
Outcome or Result_______________________
ProxyPower
DataPower
CommunicationPower
Candidate Indicators
Measure 1 Measure 2 Measure 3 Measure 4 Measure
5 Measure 6 Measure 7 Measure 8
H M L
H M L
H M L
H
H
H
H
L
H
DataDevelopmentAgenda
30
Three Part Indicator List for each Result
Part 1 Primary Indicators

• 3 to 5 Headline Indicators
• What this result means to the community
• Meets the Public Square Test

Part 2 Secondary Indicators

• Everything else thats any good (Nothing is
  wasted.)
• Used later in the Story behind the Curve

Part 3 Data Development Agenda

• New data
• Data in need of repair (quality, timeliness etc.)

30
31
What do we mean by a baseline?
OK?
Point to Point
Turning the Curve
Forecast
History
Baselines have two parts history and forecast
31
32
Indicator Reports
Neighbourhood
Country
City

KruidenbuurtTilburg, Netherlands
New Zealand
Portsmouth, UK
33
33
34
Key RBA concepts

• 2 key types of accountability
• Population results / outcomes and indicators
• Performance performance measures
• 3 types of performance measures
• How Much Did We Do?
• How Well Did We Do It?
• Is Anyone Better Off?
• 7 questions that take you from talk to action
• From Ends to Means Change our thinking and
  approach - from what we do, to what we achieve

35
quick exercise

• Population Accountability

36
Tip for Drafting Population Accountability Results

• All ______ in ______ are __________

How would you experience this outcome? What
would be different?
Insert your Geographic area
How would you measure success? What Indicator
would you use?
Insert your Population
Insert your Condition of Wellbeing
37
Performance Accountability

• For clients of programmes, agencies, teams and
  service systems

Mark Friedman (author)www.resultsaccountability.c
omwww.raguide.org
38
Results Based Accountabilityis made up of two
parts
Population Accountabilityabout the wellbeing
ofWHOLE POPULATIONS
For Communities Cities Districts
Countries E.g. All Rangatahi/Youth in Te Tai
Tokerau, All Migrants in Nelson
Performance Accountabilityabout the wellbeing
ofCLIENT GROUPS/CUSTOMERS
For Teams - Providers Programmes - Agencies
Service Systems E.g. Clients of Services,
Collectives, Ministries or the Health System
39
The 7 Performance Accountability Questions

• Who are our clients? (Client Group/Customers)
• How can we measure if our clients are better off?
  (Client/Customer Result / Outcome)
• How can we measure if we are delivering services
  well? (Quality Measures)
• How are we doing on the most important of these
  measures? (Baseline Data and Story)
• Who are the partners that have a role to play in
  doing better? (Partners)
• 6. What works to do better including no-cost and
  low cost ideas? (Common sense ideas research
  where available)
• What do we propose to do? (Action Plan)

40
Performance AccountabilityGetting from talk to
action
Client Group/Customers
41
All performance measures that have ever
existed for any programme in the history of the
universe involve answering two sets
of interlocking questions.
42
Performance Measures
Quantity
Quality
HowMuch did we do? ( )
HowWell did we do it? ()
43
Performance Measures
Effort How hard did we try?
Effect Anyone better off?
44
Performance Measures

HowMuch
HowWell
Effort
Effect
45
Performance Measures
Quantity
Quality
How much service did we deliver?
How welldid we deliver it?
Effect Effort
Output Input
How much change / effect did we produce?
What quality of change / effect did we produce?
46
Social Services Example
Quantity
Quality
How much did we do?
How well did we do it?
Effect Effort
Is anyone better off?
of clients who move off a working age benefit
and into employment (at 6 months and at 12 months)
47
Education example
Quantity
Quality
How much did we do?
How well did we do it?
Student-teacherratio
Number ofstudents
Effect Effort
Is anyone better off?
Number ofgraduates
Percent ofgraduates
48
Drug/Alcohol Treatment Programme
Quantity
Quality
How much did we do?
How well did we do it?
Percent ofstaff withtraining/certification
Number ofpersonstreated
Effect Effort
Is anyone better off?
Number of clientsoff of alcohol drugs - at
exit - 12 months after exit
Percent of clientsoff of alcohol drugs -
at exit - 12 months after exit
49
Education example 2
Quantity
Quality
How much did we do?
How well did we do it?
Student-teacherratio
Number ofstudents
Is anyone better off?
Effect Effort
Percent of secondary school students
whograduate on timeand enter Uni oremployment
after graduation
50
Primary Care Practice
Quantity
Quality
How much did we do?
How well did we do it?
Percent ofchildren who did not attend
Number ofchildren aged 0-2 enrolled
Effect Effort
Is anyone better off?
51
(No Transcript)
52
Not all performance measures are created equal
Quantity
Quality
How much did we do?
How well did we do it?
LeastImportant
Least
AlsoVery Important
Effect Effort
Is anyone better off?
MostImportant
Most
53
The matter of control
Quality
Quantity
How much did we do?
How well did we do it?
MostControl
Effect Effort
Is anyone better off?
LeastControl
PARTNERSHIPS
54
Separating the Wheat from the ChaffTypes of
performance measures found in each quadrant
How much did we do?
How well did we do it?
Clients/customers served
Activities (by type of activity)
Is anyone better off?
55
Choosing Headline Measures and the Data
Development Agenda
Quantity
Quality
How much did we do?
How well did we do it?
3 DDA
Measure 1 ----------------------------
Measure 2 ----------------------------
Measure 3 ----------------------------
Measure 4 ----------------------------
Measure 5 ----------------------------
Measure 6 ----------------------------
Measure 7 ----------------------------
Measure 8 ----------------------------
Measure 9 -----------------------------
Measure 10 ---------------------------
Measure 11 ---------------------------
Measure 12 ---------------------------
Measure 13 ---------------------------
Measure 14 ---------------------------
2 Headline
Effect
Effort
Is anyone better off?
2 DDA
Measure 15 ----------------------------
Measure 16 ----------------------------
Measure 17 ----------------------------
Measure 18 ----------------------------
Measure 19 ----------------------------
Measure 20 ----------------------------
Measure 21 ----------------------------
Measure 15 ----------------------------
Measure 16 ----------------------------
Measure 17 ----------------------------
Measure 18 ----------------------------
Measure 19 ----------------------------
Measure 20 ----------------------------
Measure 21 ----------------------------
3 Headline
1 Headline
1 DDA
56
The matter of use

1. The first purpose of performance measurement is
   to improve performance.

2. Avoid the performance measurement equals
punishment trap. ? Create a healthy
organisational environment ? Start small ?
Build bottom-up and top-down simultaneously
57
RBA categories account for all performance
measures(in the history of the universe)
TQM
Efficiency, Admin overhead, Unit costStaffing
ratios, Staff turnoverStaff morale, Access,
Waiting time, Waiting lists, Worker safety
Efficiency
Cost
Quantity
Quality
Process Input
Effort
Client Satisfaction(quality service delivery
client benefit)
Product Output Impact
Effect
Cost / Benefit ratioReturn on investment Client
results / outcomes
Benefit value
EffectivenessValue addedProductivity
Effectiveness
58
RBA categories account for all performance
measures(in the history of the universe)
TQM
Efficiency, Admin overhead, Unit costStaffing
ratios, Staff turnoverStaff morale, Access,
Waiting time, Waiting lists, Worker safety
Cost
Quantity
Quality
Process Input
Effort
Client Satisfaction(quality service delivery
client benefit)
Product Output Impact
Effect
Cost / Benefit ratioReturn on investment Client
results / outcomes
Benefit value
EffectivenessValue addedProductivity
59
RBA categories account for all performance
measures(in the history of the universe)
TQM
Efficiency, Admin overhead, Unit costStaffing
ratios, Staff turnoverStaff morale, Access,
Waiting time, Waiting lists, Worker safety
Cost
Quantity
Quality
Process Input
Effort

Client Satisfaction(quality service delivery
client benefit)
Product Output Impact
Effect
Cost / Benefit ratioReturn on investment Client
results / outcomes
Benefit value
EffectivenessValue addedProductivity
60
Not all performance measures are created equal
TQM
Efficiency, Admin overhead, Unit costStaffing
ratios, Staff turnoverStaff morale, Access,
Waiting time, Waiting lists, Worker safety
Cost
Quantity
Quality
Process Input
Effort
Customer Satisfaction(quality service delivery
customer benefit)
Product Output Impact
Effect
Cost / Benefit ratioReturn on investment Client
results / outcomes
Benefit value
EffectivenessValue addedProductivity
61
Comparing performance
62
Comparing Performance
1. To Ourselves First Can we do better than
our own history?
2. To Others When it is a fair
apples/apples comparison.
3. To Standards When we know what
good performance is.
63
Comparing Performance
1. To Ourselves First Can we do better than
our own history?
2. To Others When it is a fair
apples/apples comparison.
3. To Standards When we know what
good performance is.
64
Comparing Performance
1. To Ourselves First Can we do better than
our own history?
2. To Others When it is a fair
apples/apples comparison.
3. To Standards When we know what
good performance is.
65
The matter of standards
1. Quality of Effort Standards aresometimes WELL
ESTABLISHED
? Childcare staffing ratios? Application
processing time? Handicap accessibility? Child
abuse response time
Quantity
Effort
BUT
Effect
2. Quality of Effect Standards are almost always
EXPERIMENTAL
? Hospital recovery rates ? Employment placement
and retention rates? Recidivism rates
AND
3. Both require aLEVEL PLAYING FIELD and an
ESTABLISHED RECORDof what good performance is.
66
Advanced Baseline Display
Create targets only when they areFAIR USEFUL
Goal (line)
Target or Standard
Avoid publicly declaringtargets by year if
possible.
Your Baseline
InsteadCount anything better than baseline as
progress.
Comparison Baseline
67
Key RBA concepts

• 2 key types of accountability
• Population results / outcomes and indicators
• Performance performance measures
• 3 types of performance measures
• How Much Did We Do?
• How Well Did We Do It?
• Is Anyone Better Off?
• 7 questions that take you from talk to action
• From Ends to Means Change our thinking and
  approach - from what we do, to what we achieve

68
quick exercise

• Performance Accountability

69
Performance Measures for my (insert the name of
your Programme or Service here)
How much did we do?
How well did we do it?
Who are your clients?
Clients/customers served
What would you put in here?
client satisfaction with xxx
Is anyone better off?
Choose one
70
How Population Performance Accountabilities
Fit Together
71
THE LINKAGE Between POPULATION and PERFORMANCE
POPULATION RESULT
Contributionrelationship
Mentoring Programme for Young Offenders
Alignmentof measures
meeting weekly with mentor
young people on programme
Appropriateresponsibility
reoffending
reoffending
CLIENT RESULTS/OUTCOMES
72
Population Accountability
Result to which you contribute to most directly.
Indicators
Story
Every timeyou make apresentation, use
atwo-partapproach
Partners
What would it take?
Your Role as part of a larger strategy.
Your Role
Performance Accountability
Programme
Performance measures
Story
Partners
Action plan to get better
73
Population Accountability
Result to which you contribute to most directly.
Indicators
Story
Every timeyou make apresentation, use
atwo-partapproach
Partners
What would it take?
Your Role within the larger strategy.
Your Role
Performance Accountability
Programme
Performance measures
Story
Partners
Action plan to get better
74
Division 1
Programme 1
75
(No Transcript)
76
(No Transcript)
77
Different kinds of progress
1. Data a. Population indicators Reporting
on curves turned increase or decrease of the
graphed data (e.g. the baseline).
b. Performance measures Client
group progress and improved service delivery
How much did we do? How well did we do it? Is
anyone better off? E.g. Skills/Knowledge,
Attitude/Opinion, Behaviour Change,
Circumstance Change
2. Accomplishments Other positive activities
accomplished, not included above.
3. Stories Real stories that sit behind the
statistics that show how individuals are better
off e.g. case studies, vignettes, social media
clips.
78
Whats next?A Basic Action Plan for Results
Based Accountability
TRACK 1 POPULATION ACCOUNTABILITY

• Establish results
• Establish indicators, baselines and charts on the
  wall
• Create a result card
• Set tables (action groups) to turn curves

TRACK 2 PERFORMANCE ACCOUNTABILITY

• Performance measures, and charts on the wall for
  programmes, agencies and service systems
• Use 7 Questions manager by manager, and programme
  by programme, in management, budgeting and
  strategic planning

79
IN CLOSING
80
(No Transcript)
81
Kia ora / thank you!
82
Exercises
Fiscal Policy Studies Institute Santa Fe, New
Mexicowww.resultsaccountability.comwww.raguide.o
rg
83
3 Options

1. Turn the Curve Population Accountability
2. Turn the Curve Performance Accountability
3. Develop a Performance Measure Framework for a
   programme or service (the 4 Quadrants)

84
Turn the Curve Exercise Population Results
5 min Starting Points - timekeeper and
reporter - geographic area - two hats (yours
plus partners)
10 min Baseline - pick a result and a curve to
turn - forecast OK or not OK?
15 min Story behind the baseline -
causes/forces at work - information research
agenda part 1 - causes
Two pointers to action
15 min What works? (What would it take?) -
what could work to do better? - each partners
contribution - no-cost / low-cost ideas -
information research agenda part 2 what works
10 min Report convert notes to one page
85
4. --------- Off the Wall
ONE PAGE Turn the Curve Report Population
Result _______________
Indicator(Lay Definition)
IndicatorBaseline
Story behind the baseline
---------------------------
--------------------------- (List as many as
needed)
Partners ---------------------------
--------------------------- (List as many
as needed)
Three Best Ideas What Works 1.
--------------------------- 2.
--------------------------- 3.
---------No-cost / low-cost
SharpEdges
4. --------- Off the Wall
86
Turn the Curve Exercise Programme Performance
5 min Starting Points - timekeeper and
reporter - identify a Programme to work on -
two hats (yours plus partners)
10 min Performance measure baseline - choose 1
measure to work on from the lower right
quadrant - forecast OK or not OK?
15 min Story behind the baseline -
causes/forces at work - information research
agenda part 1 - causes
Two pointers to action
15 min What works? (What would it take?) -
what could work to do better? - each partners
contribution - no-cost / low-cost ideas -
information research agenda part 2 what works
10 min Report Convert notes to one page
87
4. --------- Off the Wall
ONE PAGE Turn the Curve Report Performance
Programme _______________
Performance Measure (Lay definition)
PerformanceMeasureBaseline
Story behind the baseline
---------------------------
--------------------------- (List as many as
needed)
Partners ---------------------------
--------------------------- (List as many
as needed)
Three Best Ideas What Works 1.
--------------------------- 2.
--------------------------- 3.
---------No-cost / low-cost
SharpEdges
4. --------- Off the Wall
88
Develop a Performance Measure Quadrant

1. Decide on a service or programme that you would
   like to create a set of performance measures for
2. Draw the four quadrants on a flip chart
3. Brainstorm with your group the range of measures
   in each quadrant
4. Work with the group to identify which measures
   you currently have data for
5. Circle these and create the list of your most
   vital few (e.g. 3-5 most important measures
   from your perspective)
6. Put these on to a clean piece of flip chart paper
   and you now have your performance measures for
   your service or programme.

89
Other helpful slides
90
The first step in performance accountability is
toDRAW A FENCEAround something that has
ORGANISATIONAL OR FUNCTIONAL IDENTITY
91
What Kind of PERFORMANCE MEASURE?

• ? of people served
• ? participants who got jobs
• ? staff turnover rate
• ? participants who got jobs
• ? of children reading at grade level
• ? cost per unit of service
• ? applications processed
• ? patients who fully recover

Upper Left
Lower Right
Upper Right
Lower Left
Lower Right
Upper Right
Upper Left
Lower Right
92
Bridging across all professions and partners
1. A condition of well-being for children,
adults, families and communities 2. A
measure that helps quantify the achievement of
a (result) 3. A measure of how well a
programme agency or service system is
working 4. A measure of the quantity of effort,
how much service was delivered 5. A measure
of the quality of effort, how well the service
was delivered, how well the functions were
performed 6. A measure of the quantity and
quality of effect on (customer's) lives.
7. A visual display of the history and
forecast(s) for a measure 8. Doing better
than the baseline forecast.
RESULT
INDICATOR
PERF. MEASURE
HOW MUCH DID WE DO?
HOW WELL DID WE DO IT?
IS ANYONE BETTER OFF?
BASELINE
TURNING THE CURVE
93
The cost of bad results
The costs of remediating problems after they occur
300 billion
InvestmentTrack
Cost
Invest in prevention to reduce or avoid out-year
costs.
94
The business case for investment in prevention
United States 1970 to 2010
2008
95
All data have two incarnations
University Graduation Rate
enrolled 01 June who graduate 15 June
enrolled 30 Sept who graduate 15 June
enrolled Bachelors who graduate at x date
96
Select 3 to 5 Performance Measuresat each level
of the organization
Be disciplined about whats most important. Dont
get distracted.
3 - 5
3 - 5
3 - 5
3 - 5
?
Get over it!
Pick the 3 5 most important of the 9 15
measures or create composites.
3 - 5
3 - 5
3 - 5
97
(No Transcript)
98
(No Transcript)
99
Logic Model