Nothing as Practical as a Good Theory

1
Nothing as Practical as a Good Theory

• RAY PAWSON
• r.d.pawson_at_leeds.ac.uk

2
The Talks Tasks

• Some new roles for theory in evidence based
  policy
• Systematic Review
• Knowledge Transfer
• Dealing with Complexity

3
Programmes are theories
Dishy-David-Beckham-theory Interviewer But do
you think the fact that these good-looking blokes
are footballers has any effect on girls' attitude
to playing football? Girl No, I think it has
more effect on them watching football, well not
the football - the guys (general laughter and
agreement).
4
Rival approaches to systematic review
Meta-analysis
Narrative reviews
Hybrid reviews
THEORYDRIVEN REVIEWS
www.evidencenetwork.org
5
Megans Law - Programme Theory
STEP ONE Problem Identification Identify
high-risk released sex offenders and create valid
and reliable registers
STEP TWO Public disclosure Issue bulletins,
press releases, call meeting to identify released
offenders to their community
STEP THREE Sanction Instigation Community joins
with police and probation to increase
surveillance of suspicious behaviour
STEP FOUR Offender response Community actions
shame offenders and decrease opportunity of
further offence
6
Evidence fragment one could the law have made a
difference? - a prospective simulation
6 could potentially respond to community
notification
36 previous offence
12 stranger predatory offences
136 serious sex offences
6 offender from out of state
24 known to victim
100 no previous offence
Petrosino Petrosino
7
Evidence fragment two did the law effect
recidivism? A matched trial
Pre-intervention sample sex
recidivism 22
Post-intervention sample sex
recidivism 19
Pre-intervention sample arrest
slow
Post- intervention sample arrest
significantly quicker
Schram Milloy
8
Evidence fragment three how did practitioners
respond? Office talk
The Law is an unfunded mandate
Special Bulletin Notification added more work to
already over-worked agents
There is more pressure to baby sit with SBN
cases simply because they are SBN cases
Zevitz and Farkas
9
Megans Law - an emerging theory

• Opportunity for and effectiveness of surveillance
  by the community is low.
• Increased surveillance and control by law
  enforcement - though this might lead to detection
  rather than deterrence.

10
On the shoulders of giants?

• Are evaluations lessons transferable?
• Can we recycle evaluation findings?
• But would it work in my patch?
• Replication and the search for enduring empirical
  generalisation
• OR
• Comparison and the confederation of of diverse
  findings under theory

11
THE GENERIC TOOLS OF GOVERNMENT ACTION

• Rather than focusing on individual programs, as
  is now done, or even collections of programs
  grouped according to major purpose as is
  frequently proposed, the suggestion here is that
  we should concentrate on the generic tools of
  government action that come to be used, in
  varying combinations in particular public
  programs.

Lester M Salamon 1981
12
Carrot theory - a Cooks Tour

• How Incentives Fare
• New York Tenements Early 20th C
• USA - Erewon State University Campus Late 20th C
• Breakfast Room Domesticville, Canada Late 20th C

13
Earmarking death money

• A significant proportion of charitable poor
  relief, intended for basic nutrition and child
  welfare, was paid over as death money to be
  used for the extravagant funerals of family
  members. The neighbours would talk if their
  wasnt a fine layout.

V Zelizer
14
Earmarking blood money

• I kind of considered it like getting twenty
  bucks from grandmother. Its 20 free dollars.
  Your going to go blow it on something. I never
  really needed it essentially but it was always
  useful.

L. Anderson et al
15
Earmarking survey response incentives

• Charities - NO!
• Lotteries - NO!
• Cash - YES!

2
5
10
The response incentive lies in the middle ground
between more subtle concepts of helping behaviour
and naked economic self interest
K. Warriner et al
16
Carrot Theory Refined.

• Payment is the measure - but incentives are the
  intended mechanism
• But subjects act on the measure rather than the
  intended mechanism
• The targeting of incentives will always be
  distorted via earmarking
• Earmarking can be benign or a blight in policy
  terms
• Earmarking lies between egoism and altruism
• Earmarking practices will vary according to
  recipient and context
• Policy makers need to consider what are the
  potential earmarking practices associated with a
  new incentive, and whether they will support or
  distort the intended outcome

17
COMPLEXITY New Deal for Communities - The
Implementation Chain
Policy Architects ? ? Practitioners
? ? Subjects
Social exclusion ?? Social mobilisation ??
Organisational ?? Programme
18
Dealing with Complexity

• Dont try to evaluate everything - assume some
  programme theories work
• Use suites of research - both live evaluations
  and retrospective reviews
• Remember to cull evidence from other policy
  domains
• Use horizontal (theories-of-change) cuts and
  follow implementation sequences
• Use vertical (realist) cuts and test programme
  mechanisms in different contexts
• THE BOTTOM LINE - IT IS IMPOSSIBLE TO EVALUATE
  AND CONTROL EVERY CONSTITUENT PROCESS

19
CALL-TO-ARMS

• GO
• FORTH
• AND
• THEORISE