Best available evidence on Research Capacity Development

1
Best available evidence on Research Capacity
Development

• Mr Andrew Booth, Reader in Evidence Based
  Information Practice, ScHARR,
• Ms Jo Cooke, Primary Social Care Lead, Trent
  RDSU,
• Dr Susan Nancarrow, Primary Care Research
  Coordinator, Trent RDSU,
• Ms Anna Wilkinson, Information Officer, ScHARR

2
ReCap Scoping review to identify the
evidence-base for Research Capacity development
in health and social care

• Aims
• To map literature on RCD interventions identify
  extent of research note gaps in evidence make
  recommendations for future research.
• Objectives
• To identify and characterise existing literature
  
• To develop comprehensive taxonomy of
  interventions and to identify theoretical models
  of RCD
• To classify interventions to promote RCD
• To identify which RCD interventions have been
  evaluated
• To identify and critique measures to assess
  effectiveness
• To develop recommendations and identify research
  agenda.

3
Scoping Review1

• a preliminary assessment of the potential size
  and scope of the available research literature2
• Ref 1 Arksey O'Malley (2005). Scoping Studies
  Towards a Methodological Framework. International
  Journal of Social Research Methodology
  8(1)19-32.
• Ref 2 EPPICentre. A scoping review of the
  evidence for incentive schemes.. London 2005.

4
Methodology

• Expert panel for interventions and taxonomy
• Comprehensive literature searches across RCD and
  specific interventions
• Production of evidence tables
• Supplementary search strategies
• Model construction

5
Comprehensive literature search

• Medline, Embase, CINAHL, PsycLit, British Nursing
  Index, Cochrane Library.
• Social sciences and related databases including
  ASSIA, Agricola, Caredata, Social Sciences
  Citation Index, ERIC, OCLC, ESRC.
• Catalogues of British Library, National Library
  for Medicine and COPAC combined UK universities
  catalogue.

Total 11,107 references 167 key documents 1.5
yield
6
Total abstracts identified and coded Phase 1
N 6992
QUOROM Flowchart
Rejected at Coding stage n 5008
Further abstracts screened Coded
1984 Phase 2 searches 4115
N 6099
Rejected at abstract stage n 5932
Studies included (167) Models
35 Interventions 132
7
Yield on Interventions (N 132)

• Training n 26
• Funding n 7
• Networks, Infrastructure Collaboration n 11
• Priority setting n 29
• Research facilitators
  n 9
• Mentorship
  n 34
• Leadership
  n 16
• Plus
• Models of RCD
  n 35

8
Supplementary search strategies

• Government policy and Professional position
  statements
• Specific named interventions citation searches
  and follow up of references
• Specific models phrase searches and citation
  searching (Google Scholar, Copernic)
• Training/skills needs analyses/surveys

9
Model construction

• Reading and analysis of existing models
• Classified as Structural, Developmental,
  Hierarchical etcetera
• Critiqued using a brief checklist
• Needs/Drivers
• Structures
• Interventions/Activities
• Temporal
• Outcomes
• Synthesis into a meta-model

10
Farmer (2002)
11
Cooke, (2005)
12
Final outcomes
Needs Drivers
Intermediate Outcomes
Interventions
Environmental
Organisational
Personal
Time
Needs Assessment
The Research Capacity Prism A Meta-Model
13
Seamless Handover!
14
RCD Interventions

• Training
• Funding
• Mentorship
• Leadership
• Facilitation
• Networks and Collaborations
• Priority setting

15
Priority setting

• Papers were selected where priorities were
  developed by consensus views of informed
  participants (known as interpretative assessment
  Lomas et al 2003).
• Relevant to RCD because its aim is to develop
  applied and relevant research, thereby having an
  impact on services- making more useful
  research.
• Present in some models of RCD

16
Techniques and methods

• Formal consensus methods ( NGT, Delphi)
• Discussion and forums
• Systematic literature analysis combined with
  expert opinion to identify gaps in knowledge
• Mixed methods- data collection of different
  sources and triangulation of findings
• Transparency poor in discussion and workshop
  approaches
• Developing literature about working with service
  users
• Action linked to training and support

17
Mentoring

• "a process whereby an experienced, highly
  regarded person (the mentor) guides another
  individual (the mentee) in the development and
  examination of their own ideas, learning and
  personal and professional development" SCOPME
  report (1998)
• Issues of reciprocity and the nature of shared
  personal development. Healy and Welchert (1990)
• Used to promote research and career development

18
Quality of the literature

• Retrospective studies
• Identify mentors as one of an number of actors
  within a research environment- more likely to be
  a characteristic of a population rather than an
  intervention
• Difficult to unpack causal relationships in the
  studies, may reflect the motivation of the
  mentee.
• Mainly focussed on medical staff and not other
  professional groups

19
ExampleSteiner et al, 2004

• Examined the association between mentorship and
  subsequent research productivity and career
  development in 139 primary care research fellows.
• Time conducting research (p .007), published
  more papers (p .003), were more likely to be the
  principal investigator on a grant (p .008), and
  more often provided research mentorship to others
  (72.5 versus 66.7 of those with unsustained
  mentorship p .008).

20
Training

• Wide ranging to include increasing skills and
  knowledge.
• Individuals and teams
• Incorporated into professional training, research
  training and short courses
• May be related to one particular area of practice
• May require completion of project.
• Locations varied academic, health care
  organisations, association with research
  programmes

21
Literature identified

• 207 papers identfied-26 evaluations studies
  identified
• Mainly retrospective surveys, two case control
  designs, and one cohort design
• 2 systematic reviews on critical appraisal
• Some had other RCD support (stipend/ mentors)

22
Training continued

• Systematic reviews showed changes in knowledge
  and attitudes
• Case control design on 1 year research
  methodology course. The course showed changes in
  productivity, activity and commitment, and
  application of research findings to practice
  (Adamsen et al 2003)
• Long term follow up of research course for cancer
  nurses- showed high levels of career progression-
  but cause or effect?
• Realistic issues around writing for publication
  workshop

23
Outcomes and RCD

• Focus of three general areas of change related to
  capacity development
• Productivity and improvement in desired function
• Learning, accommodation, and adaptation to
  change. This includes looking for differences in
  attitudes, values and beliefs
• Engagement, participation and empowerment

24
Issues that impact on outcomes

• Choice and relevance of outcome measurement
  reflects purpose of RCD
• Health gain and service improvement
• The pursuit of knowledge including professional
  knowledge, and to promote intellectual
  excellence.
• Economic gain and wealth.

25
Outcomes identified in the literature

• Academic productivity
• Research activity
• Return on investment
• Income generated
• Sustainability
• Implementation/ impact of research findings
• Personal Professional Development

• Skills and knowledge development
• Attitudes to research
• Career progression
• Job Satisfaction
• Equity
• Linkages and networking
• Infrastructure within organisation

26
Outcomes and interventions

• Most used outcome across all interventions
  include
• research productivity, increased knowledge and
  skills, and increased involvement in research/
  research activity.
• Training, funding and mentorship
• Career progression
• Mentorship and leadership
• Satisfaction and retention rates

27
Outcomes and interventions

• Networks in addition included
• network configuration
• return on investment
• improved profile
• enhanced access to research participants.
• The costs of collaborations/ networks poorly
  explored

28
Methodological issues

• Intermediate and long tem outcomes
• Process as outcomes or proxy outcome
• Different structural levels and outcome
  measurement
• Few examples explore outcomes and the inter-
  relationship between different structural levels
• Environmental and contextual factors influence
  the effectiveness
• Environmental and contextual factors may
  influence what is seen as a pertinent outcome

29
Conclusions

• RCD is often a series of joined-together, complex
  interventions, not a magic bullet
• Need for clarity around the purpose of RCD, its
  interventions, and outcomes.
• Need to explore indicators to show short term
  change that is likely to be sustainable
• Many of the activities defined as RCD may
  actually be indicators of capacity building,
  rather than the cause (eg research mentorship)

30
Conclusions (cont)

• RCD philosophies reflect the concepts of
  empowerment and participation, yet most commonly
  used outcome are those around productivity
• i.e. what gets measured is that which is easy to
  measure

31
Recommendations

• Use of a consistent framework or approach to RCD
  planning and evaluation
• Need for a range of outcomes and outcome
  indicators that reflect the purposes of Research
  Capacity
• Use of methodological approaches that reflect the
  underlying concepts of empowerment and
  participation (eg action research)

32
Recommendations (cont)

• Need to recognise the stages in RCD at different
  structural levels
• Greater need for comparative research and studies
  which measure change over a longer time frame

33
Questions?

• Andrew Booth
• A.Booth_at_sheffield.ac.uk
• Jo Cooke
• J.M.Cooke_at_sheffield.ac.uk
• Susan Nancarrow
• S.Nancarrow_at_sheffield.ac.uk
• Anna Wilkinson
• A.J.Wilkinson_at_sheffield.ac.uk