Systematic knowledge reviews in social care:

1

• Systematic knowledge reviews in social care
• SCIE approach
• Wendy Hardyman
• Research analyst
• Professor Mike Fisher
• Director of Research and Reviews
• wendy.hardyman_at_scie.org.uk
• mike.fisher_at_scie.org.uk
• www.scie.org.uk

2
overview of presentation

• Knowledge production industry in social care
• Challenges to the production of knowledge
• Towards a wider definition of knowledge
• Debates about knowledge synthesis
• Methodological developments
• SCIE guidance and systematic knowledge reviews

3
industry capacity

• Health 423m (NHS levy), 805m other sources
• 2bn pharmaceutical investment
• National RD priorities framework
• National research register
• 6350 RD staff, 205 RD units
• Social care est. 40-50m from various sources
• No national priorities framework
• No national research register
• 1350 RD staff, 77 RD units

4
capacity
5
myths and systematic reviews

• Petticrew (2001) suggests 8 common myths about
  systematic reviews (SRs)
• SRs are the same as ordinary reviews only bigger
• SRs include only randomised controlled trials
• SRs require adoption of a biomedical model of
  health
• SRs are of no relevance to the real world
• SRs necessarily involve statistical synthesis
• SRs can be done without experienced
  information/library support
• SRs have to be done by experts
• SRs are a substitute for doing good quality
  studies

6
qualitative research and SRs

• Qualitative research as an adjunct to
  quantitative research or as research in its own
  right?
• Dixon-Woods et al (2001)
• Marginalised in SRs?
• Traditional hierarchies of evidence unhelpful?
• Call for methodological development- is this a
  new question?
• Difficulties finding the evidence
• Challenges synthesising data

7
critiques (1) positivism vs. realism

• Systematic reviews the brainchild of the evidence
  based policy movement (Pawson 2002)
• Assumptions based on a medical model of clinical
  interventions
• Questionable whether this approach can apply to
  social interventions
• Pawson (2002)suggests that medical treatments are
  tested using placebos and double blinding to see
  if they work regardless of human volition whereas
  social programmes in contrast only work through
  human intentionality
• Social programmes as theories- role of realist
  synthesis
• (Pawson 2002)

8
critiques(2)research production

• Challenges to material and social relations of
  research production (Oliver 1992, Shakespeare
  1996, Zarb 1992)
• the material relations of research production
  can be demonstrated as constraining the social
  relations of research production. This in turn
  has influenced how the objectives of the
  different projects were defined and
  operationalised, the kinds of questions we asked,
  and what happened to the products of the
  research-including the critical question of
  whether or not there were any practical benefits
  to disabled people
• (Zarb 1992 page 129)

9
critiques (3) models of research

• Calls for participative research leading to
  emancipatory research
• Democratic vs. consumerist models of
  participation
• Beresford (2002) number of key issues to address
• Validity of different knowledge standpoints and
  knowledge claims
• Ownership of knowledge and its interpretation
• Dominant hierarchies of credibility
• Nature of relationship between knowledge and
  direct experience
• Meaning of evidence-based and what counts as
  evidence

10
critiques (4)barriers to research utilisation

• Differing agendas
• Many influences on decision making
• Choice of research question, tools and outcomes
• Role of experiential knowledge
• Lack of organisational/management/political
  support
• Adapted from Percy-Smith (2003)

11
towards a definition of knowledge

• Calls for a wider definition of what counts as
  knowledge
• Challenges in synthesising of knowledge from
  diverse sources
• Rethinking of traditional hierarchies of
  evidence
• SCIE and Evidence Network commission

12
sources of knowledge
Policy
Research
Practitioner knowledge
User knowledge
A KNOWLEDGE BASE FOR SOCIAL CARE
Organisational knowledge
13
issues in ebsc reviews

• few outcome studies examples of research
  utilisation and communication reviews
• need for appropriate systematic review methods
• appropriate research questions appropriate data
  sources

14
methodological issues

• synthesis of quantitative and qualitative
  research
• synthesis of qualitative research

15
methodological developments

• MacDonald (2003)
• think piece on systematic reviews
• Popay and Roen (2003)
• Reviews of methodological literature
• Methodological research
• synthesis of qualitative evidence
• synthesis of qualitative and quantitative
  evidence
• Specific projects involving methodological
  developments

16
ebsc and systematic reviews goals

• a way of classifying and appraising the quality
  of knowledge
• guidance to SCIE commissionees
• guidance to the public and to professionals about
  knowledge they can trust

17
three stages

• requirement to implement a systematic approach
• interim guidance by July 2003
• to underpin 2003-04 commissions
• full statement by 1st quarter, followed by
  consultation
• to underpin 2004-05 programme

18
key features

• a wide range of knowledge
• may include new data collection
• clarifies policy and practice as well as
  evaluating interventions
• international, includes non-English language
  sources
• critical appraisal of knowledge
• framework rather than straitjacket

19
future challenges

• Information geared to decision points,
  decision deadlines and decision makers may be
  relevant in a surprisingly limited number of
  instances. When most people most of the time
  operate from a knowledge base that they have
  acquired informally and haphazardly over time,
  research and analysis have to become incorporated
  into that base if they are to become
  influential.
• (Weiss and Bucuvalas 1980-Social Research
  Decision Making cited in Pawson 2002)

20
references

• Beresford, P. (2002). "User involvement in
  research and evaluation liberation or
  regulation?" Social Policy and Society 1 95-105.
• Dixon-Woods, M., R. Fitzpatrick, et al. (2001).
  "Including qualitative research in systematic
  reviews opportunities and problems." Journal of
  Evaluation in Clinical Practice 7(2) 125-133.
• Macdonald, G. (2003). Using Systematic Reviews to
  Improve Social Care. London, Social Care
  Institute for Excellence.
• Oliver, M. (1992). "Changing the social relations
  of research production." Disability, Handicap and
  Society 7(2) 101-114.
• Pawson, R. (2002). "Evidence and policy and
  naming and shaming." Policy Studies 23(3)
  211-230.
• Petticrew, M. (2001). "Systematic reviews from
  astronomy to zoology myths and misconceptions."
  BMJ 322 98-101.
• Popay, J. and K. Roen (2003). Synthesis Of
  Evidence From Research Using Diverse Study
  Designs A Preliminary Review Of Methodological
  Work. London, Social Care Institute for
  Excellence.
• Shakespeare, T. (1996). "Rules of engagement
  doing disability research." Disability and
  Society 11(1) 115-119.
• Zarb, G. (1992). "On the road to Damascus first
  steps towards changing the relations of
  disability research production." Disability,
  Handicap and Society 7(2) 125-139.