Government Social Research Unit

1
Government Social Research Unit
Umbrella Conference 2005 Manchester
How Research Supports Evidence-Based Policy
Philip Davies PhD Government Social Research
Unit Cabinet Office London SW1A 2WH
www.gsr.gov.uk
2
Outline

• What is Evidence Based Policy?
• Problems with using evidence for policy
• Different types of evidence
• Importance of harnessing research evidence
• Factors other than evidence
• Importance of Librarians and Information
  Specialists

3
What is Evidence-Based Policy?

• Evidence-based policy helps people make
  well-informed decisions about policies,
  programmes and projects by putting the best
  available evidence from research at the heart of
  policy development and implementation
  
  
• (Davies, P.T., 1999)

4
Evidence-Based Policy
Opinion-Based Policy
Evidence-Based Policy
Increasing Pressure (Time)
Adapted from Muir Gray 1997
5
Problems with Finding and Using Evidence

• Sheer amount and flow of information/research
• Variable quality of research outputs

6
Problems with Social Scientific Knowledge

• Not all research is of sufficient quality

• Unclear objectives
• Poor research design
• Methodological weaknesses
• Inadequate data reporting
• Selective use of data
• Unsupported conclusions

• Uncertainty of scientific knowledge
• Different status of different fields of knowledge

7
Evidence-Based Policy
Opinion-Based Policy
Evidence-Based Policy
Increasing Pressure (Time)
Adapted from Muir Gray 1997
8
Problems with Finding and Using Evidence

• Sheer amount and flow of information/research
• Variable quality of research outputsNeed to
  separate the wheat from the chaff
• Problems of publication bias
• Need for the balance of evidence
• Limitations of single studies

9
Limitations Of Single Studies

• Single studies can misrepresent the balance of
  research evidence
• Illuminate only one part of a policy/practice
  issue
• Sample-specific
• Time-specific
• Context-specific
• Often of poor methodological quality
• Consequently, biased

10
Systematic Reviews

• attempt to discover the consistencies and
  account for the variability in similar-appearing
  studies
• seeking generalisations also involves seeking
  the limits and modifiers of generalisations
• identify the contextual-specificity of available
  research and evidence
• (Cooper and Hedges, 19944).

11
Different Types of Evidence for Policy

• Systematic Reviews
• Rapid/Interim Evidence Assessments
• Single Studies
• Pilots and Case Studies
• Experts Evidence (GOBSATT)
• Internet Evidence

12
Types of Research Evidence
Experimental Quasi-Experimental Counterfactual
Experimental Quasi-Experimental
Qualitative Theories of Change
Social Ethics Public Consultation
Surveys Admin Data Comparative Qualitative
Implementation Evidence
Ethical Evidence
Descriptive Analytical Evidence
Cost-Benefit Cost-Effectiveness Cost-Utility Econo
metrics
Impact Evidence
Surveys Qualitative
Economic and Econometric Evidence
Attitudinal Evidence
Statistical Modelling
Linear and Logistic Regression
13
The Contributions of Librarians and Information
Specialists

• Undertake mediated systematic searching for
  evidence
• Support and guide researchers searching
• Provide training on searching and retrieval
• Help with the critical appraisal of research
  evidence
• Help separate the wheat from the chaff
• Manage the stock and flow of research knowledge
• Managed delivery of documents
• Keep abreast of knowledge management tools/systems

14
Factors Influencing Policy Making
Experience Expertise
Pragmatics Contingencies
Judgement
Evidence
Lobbyists Pressure Groups
Resources
Habits Tradition
Values
15
What is Evidence-Based Policy?

• Evidence-based Policy means integrating
  experience, expertise and judgement with the best
  available external evidence from systematic
  research.
• 
  
• (Davies, P.T., 1999)

16
Take Home Messages

• Policy making uses many types of evidence and
  research
• Evidence-based policy involves research
  utilisation
• Not all research is of equal value
• Critical appraisal of research evidence is
  crucial
• Need to differentiate research evidence on
  quality
• Systematic reviews are essential for harnessing
  evidence
• Knowledge management systems are also essential
• Librarians and information specialists are
  invaluable
• There are factors other than evidence and research

17
References
Cooper, H. and Hedges, L.V. (eds), 1994 The
Handbook of Research Synthesis, New York, Russell
Sage Foundation.   Davies, P.T. 1999 What is
Evidence-Based Education?, British Journal of
Educational Studies, 47, 2, 108-121.   Gray,
J.A.M. 1997 Evidence-Based Healthcare How to
Make Health Policy and Management Decisions, New
York, Edinburgh, London, Churchill
Livingstone.    
18
Contact
phil.davies_at_cabinet-office.x.gsi.gov.uk Government
Social Research Unit Cabinet Office Admiralty
Arch The Mall London SW1A 2WH England Tel 44
(0)20 7276 1862 Fax [phone]
http//www.policyhub.gov.uk