Systematic Review of training outcomes in the Alcohol and Other Drugs AOD Field

1
Systematic Review of training outcomes in the
Alcohol and Other Drugs (AOD) Field

• Jodie Shoobridge
• Drug and Alcohol Services South Australia (DASSA)
• with Darlene Addy, Kathy Larrigy, Sandra Manning,
  Judith Seabel

2
Introduction

• Topic
• A systematic review of Training Needs Analysis
  (TNA) in the AOD Field
• Purpose of review
• To inform the development of evidence-based
  methods for performing Training Needs Analysis in
  the AOD field
• Identify whether a conducting a TNA to inform
  training results in actual (sustained) practice
  change following training
• Identify, review and recommend psychometric tools
  that will assist trainers to perform TNA in the
  AOD field
• Anticipated outcomes
• Identification of relevant tools development of
  guidelines to assist trainers to conduct
  effective TNA that result in actual practice
  change demonstrate effectiveness of conducting
  TNA prior to training (versus not doing so)
• Development of skills in conducting systematic
  review / increased competence in literature
  search and identification.

3
And then??

• Reality hits!
• Work
• Team
• Time
• TTRQ!!
• ?TNRQ

4
TNRQ

• From a systematic review of the literature, what
  is the evidence that non-tertiary based training
  programs have an impact on changing behaviour of
  health professionals or AOD workers to improve
  their management of clients with AOD-related
  problems?

5

• Faced with a lack of career structure, a
  concentration on individuals (rather than the
  systemic approach to) provide AOD services, and
  lack of national coordination,
• Could it ever be that in 10 years time, 1990
  will mark the date when something was actually
  done to substantially change the face of training
  in substance use problems? Michael Farrell,
  1990

6
Strategies for which there is evidence of
behaviour change
7
Tools for conducting review

• NHMRC Guidelines
• EPOC checklist
• SARNeT website
• Kirkpatrick evaluation criteria

8
Levels of Evidence

• Kirkpatrick 1998,
• Reaction
• did it feel good? Course improvement
• Learning
• KSAs
• Change in Behaviour
• application of KSAOs
• Change in organisation
• impact on org effectiveness
• Change in achievement of ultimate goals
• imp. national health, inc service provision
  efficiency

• NHMRC, 1999
• I SR of all RCTs
• II SR of gt1RCT
• III-1 pseudo-RCT
• III-2 comparative studies with concurrent
  controls (case controlled, interrupted TS with
  CG)
• III-3 comparative with historical control, gt2
  arms, Inter. TS without parallel control
• IV case series, PT or PreT/PT

9
Search strategy

• Databases
• OVID (CINAHL, MEDLINE, PSYCHINFO), DRUG, AIMMAT,
  DARE, Cochrane, EBM Reviews, EMBASE
• Search terms
• AOD
• substance/s ab/mis/use/disorder/dependence,
  alcohol, tobacco, heroin, amphetamine, cocaine,
  drug/s addiction/dependence/rehabilitation/addicti
  on
• Intervention
• Skill/s/professional/staff development,continuing
  education/workplace training, learning need/s
  analysis/identification/assessment upgrading,
  upskilling, consultation, quality

10
Search strategy

• Target group eg.
• health/nursing/es/medical/GP/doctor/physician/medi
  cal staff/counsel/ling/clinical/industrial/organis
  ational/psychology/ists/youth/social
  work/paraprofessional/counsel/lor/continuing
  education
• Evaluation method
• Pre/post test training/design, evaluation,
  practice changes, client/trainee/satisfaction,
  happy/smile sheet
• Changes in
• Knowledge, skill/s, attitude, confident,
  counsel/ling, soft skills, role adequacy,
  behaviour change, effectiveness/efficacy,
  practice change, performance improvement,training/
  technology transfer, workforce development,
  treatment/psychotherapeutic outcomes

11
Search strategy

• Limited to
• English only
• 1990-2005
• Exclusions
• Peer/school education
• Pastors/religious instructors
• Refuge workers
• Psychiatrists
• Post/graduate/tertiary education or program
  evaluations
• Identify training from education

12
Findings

• Over 2500 possible references
• Over 200 potentially relevant
• Past reviews eg. Ask et al 1998,
• Identify relevant studies
• Descriptive, not systematic criteria of
  comparison
• Often report changes in KSAs as beneficial
  outcomes
• little evidence of actual practice change and
  other variables (organisational etc ) that may
  support both transfer and maintenance of skills.

13
Examples of assessed studies
14
Examples of assessed studies
15
Questions

• Relevance of outcome variables
• Relationship between traditional training,
  evaluation and outcome methods and philosophy
  change ie WFD approach
• Issues to consider individual factors
  (therapeutic alliance, preferences and KSAOs,
  core competencies within organisation ie is
  training properly targeted?
• Opportunities for Systemic approach to training
  ie requires massive cultural change in approach
  et Tober and Raistrick, 1990 .

16
Work in progress

• Complete study identification
• Second rater
• Paper write up
• Conference presentation

17

• Thanks to
• Karin Ried
• Rachel Waters
• Liz Farmer
• Margaret ONeill
• SARNet
• FUSA PHC RED Scheme
• Our Team!