Problem Based Learning Service Evaluation

1
Problem Based LearningService Evaluation

• A Psychotherapy Service
• (Please note this is one example, produced with
  permission, from one small group of trainees. It
  represents only the powerpoint show part of the
  group presentation which was held after only a
  few weeks of being on the course. Thus it does
  not include all the other information that was
  provided at the time of the group presentation.)

2
Overview of Presentation

• Overview of Service
• Background
• Development
• Context
• How the service works?
• Progress to date
• Tentative evaluation aims
• Framework
• Methodology
• Analysis
• Timeline

3
Overview of Service

• Background
• Service 4 years old
• Need arose at 2 levels
• National Directives talking therapies, service
  user demand
• Local service was fragmented, service user
  demand
• Chose overall provision of therapeutic practice
  with joint working, supervision, training and
  some individual work.
• This was to avoid a small medically led
  service long waiting lists, clinic based work
  etc

4
Overview of Service

• Development and Nature
• Head of Service
• Put together team of local therapists, with
  different backgrounds, from PCT
• Group individual therapy. Includes CBT,
  Psychodynamic, CAT, DBT, Family therapy, Sex
  therapy, Mindfulness
• Consultation, supervision and training offered to
  wider service

5
Overview of Service

• Context
• Tertiary service
• Referrals from secondary service, must have care
  co-ordinator
• Service assesses appropriateness of referrals and
  capacity to meet needs.
• To avoid waiting list consultation may be offered
  in place of direct work

6
Overview of Service
Psychotherapy Service
Primary Care Secondary Care Mental Health
Professionals (undertaking therapy)
Staff with day-to-day contact with Service Users
(NHS Voluntary Sector)
Individual Service Users
Therapeutic Groups
7
Progress to date

• Literature search service evaluations
  theoretical discussions
• Gathered relevant DoH documentation
• Liased with Occupational Psychologist (staff
  satisfaction service) Clinical Psychologist.
• Consultation with service users in progress
• Consulted with specialist psychotherapist on team
  and Head of Service

8
Tentative Evaluation Aims

• Following consultation with Head of Service
  possible research areas
• Assess the impact of psychotherapy service on
  wider mental health service
• Use Department of Health Paper markers of best
  practice in psychological therapies service as
  evaluation framework (Organising and Delivering
  Psychological Therapies, 2004).
• To avoid top-down evaluation we have arranged to
  consult service users regarding what they see as
  important to evaluate

9
Chosen framework

• We have chosen to explore the option of using
  the DoH paper as a framework, however we have not
  yet gained the views of service users therefore,
  these plans are subject to change.

10
Framework

• Evaluate against 10 desirable features of
  psychotherapy service
• Targeting those with greatest need
• Multi-disciplinary multi-professional
• Clear training strategy
• Single point of entry use of referral protocols
  and procedures
• Involves service users and carers
• Tiered psychological network
• Effective waiting list management
• Regular routine audit and research
• Clear and defined leadership structure
• Well integrated into mental health service as a
  whole.

11
Methodology

• Data will be collected from a variety of methods
• Quantitative, including CORE assessment data
  (routinely collected), ePEX electronic patient
  records questionnaires (staff, wider mental
  health workers and service users)
• Qualitative, including Focus group (wider mental
  health workers) interviews (as follow up to
  some questionnaires)

12
Analysis

• Tentative as design not finalised
• Descriptive statistics of ePEX data and some
  questionnaire data.
• Inferential statistics of some questionnaire data
  and CORE assessments
• Thematic content analysis of transcripts from
  focus groups and interviews

13
TimeLine

• November
• Consult with service users
• Assimilate feedback from presentation and views
  of service users
• Revise design
• December
• Liase with academic tutors statisticians to
  confirm validity of design and analysis
• Discuss revised design with service
• Contact LREC

14
Timeline

• January
• Develop/select quantitative measures
• Decide questions for focus groups and
  questionnaires
• February
• Ethics committee application
• March-April
• Write research proposal
• Commence data collection