Contribution Analysis: An introduction

1
Contribution Analysis An introduction

• Anita Morrison
• Scottish Government

2
(No Transcript)
3
Evaluation in Scottish Government

• A need to revisit our practice in context of
  performance management and outcomes-based
  approach shift balance from small scale,
  process evaluation
• Building a more strategic approach to
  evaluation planning and delivery - multiple
  policies / interventions to deliver outcomes
• Ensure the right kind of evidence is available
  answering questions of effectiveness,
  contribution, for Spending Review 09 and beyond

4
Key analytical questions

• What progress are we making towards the Purpose,
  Purpose targets and National Outcomes?
• What contribution have our policies and
  programmes made?
• How cost-effective are our policies and
  programmes?
• What are the lessons for future policy
  intervention /changes in resource allocation?
• Contribution Analysis

5
Contribution analysis

• Holy grail of attributionBuilds a reasonably
  credible case about the difference the program is
  making. (John Mayne)
• Good contribution analysis is, in effect,
  theory-based evaluation
• Use of logic models and results chains
  strengthens programme planning and design
• Clearly sets out assumptions and external factors
• Also strengthens approach to research /
  analytical planning
• Developing in a couple of ways in Scotland
  contribution of organisations contribution of
  policies and programmes

6
Alcohol Partner Contributions to Outcomes
Improved mental wellbeingReduced inequalities in
healthy life expectancyReduced inequalities in
alcohol-related deaths and hospital admissions
BehaviourReduced alcohol consumption levelsLess
drunkenness less drink-driving
EnvironmentsPhysical Reduced exposure to
alcohol-related hazardsEconomic Reduced
availability/affordability of alcohol Social
Drunkenness less attractive sensible drinking
the norm
Service uptake engagement
Understanding risks, attitudes to drinking
Hazardous and harmful drinkers
General public - targeted
Preventive services
Sensible drinking messages
Media campaigns
Brief advice
Scottish Govt
NHS
7
(No Transcript)
8
A Basic Results Chain
Program (Results) Chain of Events (Theory of
Action)
Key Questions
WHY?
Indirect Influence
7. End results
7. What is our impact on ends?
6. Practice and behavior change
6. Do we influence behavioural change?
WHAT?
5. Knowledge, attitude, skill and / or
aspirations changes
5. What do people learn? Do we address their
needs?
Direct Influence
4. Are clients satisfied? How do people learn
about us?
4. Reactions
WHO?
3. Who do we reach? Who uses / participates?
3. Engagement / involvement
Control
2. What do we offer? How do we deliver?
2. Activities and outputs
HOW?
1. Inputs
1. How much does our program cost? (, HR etc)
Source Adapted from Claude Bennett 1979. Taken
from Michael Quinn Patton, Utilization-Focused
Evaluation The New Century Text, Thousand Oaks,
California, 1997, p 235.
8
steve.montague_at_pmn.net
9
Assembling and critiquing the story

• Multiple sources of evidence to accommodate can
  include
• Population based statistics
• Survey data
• Needs assessments
• Process and outcome evaluations
• Economic analysis and evaluation
• Systematic reviews and other evidence synthesis
• International policy and analytical work
• Baseline and forward look data /evidence
  gaps

10
How do you actually do this in practice?

• To date few live examples of contribution
  analysis applied in the UK Health Scotland led
  on this
• Scottish Government early work outcome 15, GES
  population target, interest on outcome 8, outcome
  12
• Sessions to develop and discuss logic model (mix
  of analysts and policy leads)
• Baseline narrative and performance story so far
• Analytical planning what data and analysis will
  we need in the future?

11
Some tentative conclusions

• Outcomes focus is an opportunity to undertake
  longer-term, more strategic analytical work
• Needs input of policy, practice and analytical
  professionals
• Contribution analysis not a panacea but a tool
  to help think and discuss policy and service
  development differently
• Does help you to plan any research and analysis
  required in assessing progress