A Review of Evaluation Studies of PBL across Disciplines, and the Implications for Our Future Practi

1
A Review of Evaluation Studies of PBL across
Disciplines, and the Implications for Our Future
Practice

• Andy Ginty
• Caroline Marcangelo

2
Presentation Outline

• Background
• Looking for evidence
• What did we find?
• Where now?

3
Background

• Evaluation studies can inform the community of
  practitioners of the effectiveness of PBL and
  thus support its implementation into new and
  diverse curricula
• Current existence of a considerable body of
  published literature that reports the evaluation
  of PBL within health-related programmes.
• We wished to explore the types of evaluation
  studies that have been used to measure the
  effectiveness of PBL, and appraise the methods
  used in these studies

4
Looking for Evidence

• Following EPPI (2006) guidelines to plan the
  review
• Setting the scope
• Gathering describing research in field
• Analysis synthesis methodology sound,
  appropriate and relevant to focus judgement of
  weight
• Writing making use of report
• Andrews Harlen (2006) used to inform our
  synthesis of the review
• Meta Analysis or Narrative Synthesis

5
Setting the Scope

• Parameters
• Published in English UK 2001-2006 health care
  community
• Keywords Problem-Based Learning (PBL)
• Evaluation, Adoption, Educational,
  Professional Competence
  Professional Skills
• Search of Databases
• EBSCO, Ingenta Select, Science Direct
• Blackwell Synergy

6
Table of first results
7
Refining the Search inclusions exclusions

• From initial results
• focus on PBL Evaluation
• excluded studies outside UK
• excluded indirect reference to pbl
• excluded non-evaluative focus
• excluded non-health-related courses
• Identified duplicate papers
• Compared reviewer notes

8
What have we found?

• 13 papers in the final review
• 8 Process Evaluation
• 4 Outcome Evaluation
• 2 Descriptive Studies
• some combined 2 types of evaluation
• Mainly concerned with student opinions
• Majority were questionnaire plus focus groups
  lacked consistency as an overall group of studies
• Mostly on one iteration of a module, with a
  minority on a whole course all single HEIs

9
Where now?

• Lessons from medicine ..
• Norman Schmidt (2000)
• Evidence of treating PBL as a single intervention
  and examining the usual cognitive and clinical
  outcomes will conclude minimal differences need
  a systematic research approach looking at all
  aspects
• Maudsley (2001)
• effort expended chasing elusive outcomes needs
  complementing with meaningful interpretation of
  process
• Richards (2003)
• CMO we should aim to evaluate at a lower level
  of explanation, taking into account the actual
  processes that cause it to work

10
Norman GR Schmidt GH (2000) Effectiveness of
problem-based learning curricula theory,
practice and paper darts. Medical Education
34721-728

• Variables important in PBL
• Amount of prior knowledge
• Quality of problems
• Tutor performance
• Group functioning
• Time spent on individual study
• Lead to
• Interest in subject matter
• Achievement

11
Opportunity for PBL SIG ?

• To coordinate a larger evaluation study
• Standardised questionnaire to measure across
  HEIs
• Standardise focus groups
• Longitudinal study following either student
  cohorts, or year of a programme
• More high quality evaluation studies focussing on
  the facilitator perspective

12
References

• Andrews A Harlen W (2006) Issues in
  synthesizing research in education. Educational
  Research 48,(3) 287-299
• EPPI-Centre (2006) EPPI-Centre Methods for
  conducting systematic reviews (Version 1.0).
  http//eppi.ioe.ac.uk/ accessed 12/11/06
• Maudsley G (2001) What issues are raised by
  evaluating problem-based undergraduate medical
  curricula? Making healthy connections across the
  literature. Journal of Evaluation in Clinical
  Practice 7,3, 311-324.
• Norman GR Schmidt GH (2000) Effectiveness of
  problem-based learning curricula theory,
  practice and paper darts. Medical Education
  34721-728
• Richards L (2003) Evaluation in medical
  education moving forward. Medical Education 37,
  1062 1063