Online Education in the ER

1
Online Education in the ER

• Nadim Lalani MD

2

3
Vanilla Sky

• Tom Cruise 2001
• Existential Mind warp
• Deals with cryogenics and the possibility of
  living a virtual life after death
• Blending of the technologic and biologic worlds ?
  plugged in
• ?Virtual technologic world to supplement real
  world EM medical education

4
Objectives

• Definition
• Background
• Literature Review
• Med Ed
• Resident Ed
• Professional Development
• What it might look like
• Future Directions

5
What is Online learning?

• Online Learning e-learning is digital
• Evolved from CD/computer labs
• Everyone does it!
• Performance Support for software e.g
• Web page e.g. Uptodate
• Self-paced Web-based CME
• Leader-led Distance Learning
• Blended or hybrid learning
• combines conventional with digital learning

6
Advantages of e-learning

• Rich environment
• Media-filled esp in EM
• transfer of difficult concepts
• Links to sources
• Convenient, efficient flexible
• Asynchronous
• Can be accessed from a distance
• Adult learning principles
• Self-paced and self-directed
• Flexible/ home access
• efficiency

7
Background Why bother?

• U of C Medical School current enrollment 130
  students ? goal 150
• Mandatory EM rotation / increasing competencies
• Resident numbers also increasing
• Result ? more learners in the ED
• Relative shortage of preceptors, relevant
  clinical encounters and curricular time
• Will be worse when our program expands ? usurp
  learning opportunities

8
Why bother

• Deficiency in learning encounters a
  performance gap
• future physicians do not have the adequate
  exposure to emergent problems.
• imperative we equip students, clerks
  residents with the skills and training.

9
why bother

• Increased digitalisation is a key strategic goal
  of the U of C
• Learners are unique with mulitdimensional
  learning styles.
• Adult learning principles
• Attract the best candidates
• Provide a method of training students and clerks
  at two different campuses
• Provide consistency in teaching

10
Why bother?

• Provide efficient means of knowledge transfer to
  residents
• Increasing number of competencies CANMEDs
• Better use of academic half-day.
• Provide more effective professional development
• Asynchronous ? dont have to be there
• Interactive discussion board
• Consistent, evidence-based standard of practice
• Increased self-efficacy

11
E-learning not a panacea

• there is more to training and education than
  e-learning
• Certain skills do not lend themselves to
  e-learning
• The key will be selecting the best delivery
  method.
• Cannot simply upload old material.
• Learner focused
• no one solution blended may work for
  residents.

12
Process Can it be done?

• Fail to prepare prepare to fail
• Need to address several key questions
• Purpose? Added value?
• What support and expertise exist?
• Ongoing upkeep?
• Stakeholders?
• Team?
• Instructional design/Pedagogy

13
Literature Review

• Same Search terms in PUBMED
• Bibliographies of relevant articles scanned
• Missing 1 Med Ed 1 CME both foreign language
  

14
Literature general Comments

• More Literature exists for Med Ed
• Pre 1990 Limited by lack of internal validity
• Few Randomised Controlled Studies
• Emerging Lit wrt Resident experience
• Despite lots of experience with online CME
• Little Literature mostly Descriptive

15
Literature General Comments

• Terminology inconsistent
• Interventions vary.
• ? prototypes of todays technology?
• Dont address some of the uniqueness internet
• Comparing apples to oranges

16
E Learning Med Ed

• Can E-learning be used to replace/augment
  Traditional Methods?

17

• Study Dartmouth Med School
• 328 Students randomised to
• Interactive Case-based study guide on Computer
• Case-Based Printed study guide
• Anemia and Cardiology Courses
• Outcomes
• Performance on higher order MCQ tests, exams
• Self-reported Efficiency
• media-rich ? images, blood smears and EKGs

18
Results
Time Spent
Cardiology Computer 4.4 0.6h
Cardiology Workbook 9.4 0.5h P 0.0001

Anemia Computer 5.5 0.5h
Anemia Workbook 8.0 0.5h P 0.001

• No Difference in Test Scores
• No difference on board exams
• The vehicle is an acceptable means of delivery

19
Limitations

• Self reporting of efficiency!
• Confounders other text books/practice
  exams/time-spent cramming
• Doesnt really tell us about dynamic
  problem-solving/ clinical judgment

Board Exams Anemia CV
Class CAL 81.5 6.5 80.0 6.7 NS
CAL 82.7 6.0 82.4 5.6 NS
20

• 179 Paeds Clerks in 2 sites Chicago
• Randomised to Lecture via
• Multimedia Text Book
• Lecture
• Printed Text
• No intervention
• Paeds airway diseases
• Outomes
• MCQ Test Score at end of rotation at 1 y
  later
• Only different in audio/video

21
Results
Score / 26 MMTB Printed Book Lecture Control
N 89 21 19 23 26
Initial Test 16.6 2.5 16.5 3.6 15.9 2.5 14.2 2.9

Final Test 15.3 2.8 15.3 2.9 14.6 2.3 14.5 3.4
22
Limitations

• 51 Attrition rate!
• Clerks at one site had mail-in repeat exam
• Confounders
• One hour instruction embedded in a 6 week
  clerkship

23

• 75 Med students Brisbane Australia
• Randomised after pretest to
• Computer Tutorial ? Focus on knowledge
• Computer tutorial ? create ideal patient for dx
  feedback every 10 cases
• Computer Tutorial ? both knowledge decision
  three different types of feedback after every 10
  cases
• Looking at diagnosing abdo pain 30 cases

24
Outcomes and Results

• Outcomes
• Attained knowledge
• Diagnostic accuracy
• Decision-making confidence self reported
• Results
• Students focusing on facts did not improve
  decision-making
• All feedback groups improved diagnostic accuracy
• Type of Feedback not important.
• Self reported confidence improved

25
Limitations

• Small study
• Very convoluted method ? ?reliability

26
E- Learning Med Ed

• Can E-learning be used to Teach Procedural Skills?

27

• 82 Medical Students Toronto Augusta
• Randomised to
• Computer Tutorial knot board
• Lecture with Feedback knot board
• Two-Handed Knot tying
• Tested right after filmed
• Outcomes
• Proportion Square/ Time to tie
• Knot Performance score blinded surgeons
• Student Questionnaire

28
Results

• NO difference in Cognitive portion
• Lower performance score in CAL group
• 89 Students would have preferred Lecture Session
• Lack of feedback cited as negative

29
Limitations

• Apples and Oranges!
• ? Not controlled for hands-on feedback
• Maybe CAL better if it described pitfalls /
  showed video of good and bad knots?
• Reliability of performance score not included

30

• 42 Clerks U of T
• Randomised to
• Computer Tutorial rich text, animations,
  interactive QA, no audio/video
• small group seminar also interactive
• Epistaxis Management
• Outcomes
• Short Answer written Test
• Practical Test 16 point performance scale

31
Results

• Poor Prior knowledge
• No difference in written scores
• Slightly better practical skills with CBL

32
Limitations

• Small numbers
• Examiners NOT blinded
• ?reliability of performance score not included
• Practical was on dummy
• ? transferability

33

• 69 Medical Students Wisconsin
• After pre-test Randomised to
• Didactic Session/QA
• Video-Tape Session
• Computer Tutorial
• Post Intervention
• MCQ test, Practical Skills test 2 blinded obs
• Repeat testing at one month
• no feedback . Instructor present

34

• Outcomes
• MCQ Test Scores
• Timed observation of skills
• Critical Skills evaluated via checklist
• Performance Quotient calculated

35
Results

• Higher initial mean correct / complete in CBT
  group plt0.01
• Significantly better PQ in CBT group at 1 month
  p lt 0.01

36
Results

• Didactic group better on immediate MCQ 63 vs
  49 for video/CBT p lt 0.01
• Difference in MCQ still there at 1 month

37
Results

• Bigger change in PQ with CBT at 1 month Plt 0.01

38
Limitations

• Small study
• Video vs CBT essentially the same intervention
• ? Why CBT would do better than Video
• ? Reliability of checklist and PQ?

39
What About the ED Experience?

• Can E-Learning be used for Emergency Medicine
  Rotations?

40

• 100 Clerks Mt Sinai Randomised by blocks
• EM rotation with access to EM Website
• Modules ACLS, Tox, Xrays, Pix, Paeds Cases
• EM Rotation without access
• Outomes
• Exam Scores
• Student Satisfaction

41
Results

• ONLY 28 intervention group used it.
• 72 Cited lack of time
• NON sig difference in exam score 72.8 vs 68.2
  p 0.058
• Non sig difference in satisfaction 77.5 vs 66
  p 0.23
• Baseline only 26 gt 1h /wk online cf 96 next
  class
• Baseline 65 wanted online component

42
Limitations

• lt 30 in intervention group ? didnt reach power.
• WAS ITT ? so results would have been ve with
  more participants
• Problems with randomising by block rotations
  given away on lottery CARMs
• Unmotivated learners?
• ?generalisable to clerks in 2008

43

• 23 Clerks U of T Sick Kids
• Volunteered for study, Randomised to
• Access to Web-based Modules
• No Access to Web-based modules
• ED Procedures lac, LP, splint
• Outcome
• Performance on MCQ Test

44
Results

• Statistically higher competence p 0.0001
• Cohens d Effect size r 0.79

45
Limitations

• Small sample size
• Volunteers EM /techno gung-ho
• Methodology
• Unclear when test was administered in relation to
  rotation
• ?randomised to learning vs no learning?
• Validity of MCQ vs Observed skills
• Transfer of knowledge?
• MCQ vs Observed skills

46

• 350 Urology Clerks 4 med schools US
• Randomised two-group crossover to
• Web-based Tutorials BPH,ED,PC,PSA
• No Access to Web Tutorials
• Served as the controls for the modules they
  didnt have access to online
• Outcomes
• Performance on test pre/post Cr .79
• Durability of learning/ Learning efficiency in
  SubG

47
Results
48
Results
49
Results

• Learning Efficiency 0.10 vs 0.03 plt0.001
• Test scores still improved without WBT 12 BPH,
  6 ED, 24 PC, 20 PSA
• Web-based alone had Cohens r 24.9!

50
Limitations

• Volunteers with unequal participation b/w sites
  93 HMS vs 52 BUSM
• High Drop out rate 210 /350 completed
• ? Generalisability of repeated measures
• ? Generalisability to EM

51
Summary E-Learning Med Ed

• Content can be delivered
• Appears to be transfer of learning of Cognitive
  skills Perhaps also Psychomotor skills
• Still need for experienced clinician feedback
• Increased student satisfaction
• Attempt to make instructional design identical
  validity undermines uniqueness of EL
• Always will be apples and oranges

52
Summary E-Learning EM

• Controlled Interventional Literature conflicts
• Learners
• Identify time constraints as a barrier in ER
• Want more visual aids Ekg, XR, photo WEB
• Effective strategies for ER teaching include
• Using Resources
• Going beyond patient care
• Improving the learning environment
• Adult principles/ learners may be driving force
  despite dearth of evidence

53
How Can we Augment Med Ed?
Currently Clerks Use LMS Osler Download PDFs of
Content/Cases Core Content Reviewed with Preceptor
54
We can do more

• Procedures/ Anatomy
• Pictures
• Xray / Ekg
• interactive ppt

55
E Learning for Residents
56

• 109 IM Residents US during clinics year
• Randomised Crossover to
• Access to Web-based Practice Guidelines
• On WebCT rich format with links to sources
• Printed Practice Guidelines
• Outcomes
• Format Preference
• Performance on Final test

57
Results

• Strong preference for Web Based material
• Men gt Women
• NO difference on test between groups
• Non-Significant reduction in time spent
• problems with WEbsite

58
Limitations

• Volunteer
• Significant dropout 145 eligible ? 109 enrolled ?
  51 completed all aspects
• ? generalisability

59

• 162 FM IM Residents
• Randomised to
• Web-based tutorial hyperlinks, graphics
• Printed material
• ACC/AHA Guidelines Management post AMI
• Outomes
• Test Score
• Efficiency
• Satisfaction

60
Results
61
Limitations

• Only enrolled 30 550 eligible
• Participants not blinded to hypothesis
• ? Generalisable
• Voluntary monetary honorarium

62

• 22 EM residents Staff U of T
• Noninferiority for U/S vascular access course
• Randomised after pretest to
• Web Tutorial practice
• 1h Lecture practice
• Outcomes 2 weeks later ..
• MCQ test
• 4 OSCE stations blinded obs used checklist
• Non-precepted

63
Results

• Same pass rate
• All web users logged on from home

64
Limitations

• Small study did have power though
• No controlling of practice session
• Stronger coaching the weaker?
• No interrater reliability for OSCE

65
Summary E-Learning Residents

• Learning can be delivered this way
• Weak evidence of non-inferiority for learning
  guidelines in other specialties
• Modest evidence for use prepping for a hands-on
  session relating to EM.
• Weak evidence for Psychomotor skill acquisition
• Consistent satisfaction with the method

66
How to Augment Resident Ed

• Website / Subscriptions / RemergS
• HPS / AHD

67
Better Use of AHD

• Longitudinal course on LMS?
• Already Licensed by U of C
• Used by Anesthesia

68
Interaction is key

• Learner-Sources, Learner-learner,
  Learner-Clinician

69
E Learning and CME
70

• 52 Physicians US randomised to
• Web-Based CME
• Normal Instruction
• Office Dermatologic procedures
• Outcomes
• Performance on test
• Satisfaction
• Self-reported performance change

71
Results

• More improvement in test score with WBL 13.2 vs
  9.6
• General satisfaction with WBL
• Increased self-reported competency
• Fair amount of interaction in asynchronous forums

72
Limitations

• Small study
• Self-reporting
• ? Reliability of MCQ test
• ? Direct observation of procedures

73

• 99 Physicians US
• Randomised to
• Web based CME for domestic violence
• Regular Instruction only did two surveys
• Outomes
• Self-Efficacy
• Externally validated Survey instrument CR 0.7
• Change in screening

74
Results

• Increase in self efficacy 18 intervention vs -
  0.6 control p 0.01
• Positive other endpoints in survey
• NO difference in DV screening

75
Limitations

• Honoraria to WEB participants
• Methods
• ?No learning to controls?
• generalisability

76

• 103 physicians US randomised to
• Web-based CME
• F-2-F small group CME session
• Cholesterol management
• Outcomes
• Performance on tests
• Chart audit 20 docs from each group 25 charts
• Satisfaction survey

77
Results

• More improvement with Online
• NO difference in Satifaction between groups
• Online spent 3.8 h / 3 sessions
• Chart review statistically signif small
  difference in guideline adherence but no
  difference in cholesterol screening

78
Limitations

• 170 eligible only 103 included
• ? Different groups? online better at baseline
• Live CME event happened before Online ?cross
  contamination
• Hard to reconcile change in behaviour when no
  change in screening
• ? generalisability

79

• 87 Physicians self-selected into
• Online CME 3 sessions ? 8h
• F-2-F CME one 8 h session
• Treatment of opioid dependence
• Outcomes
• Post intervention test
• Satisfaction

80
Results

• NO difference in improvement
• No difference in satisfaction scores

81
Limitations

• 30 attrition 87 entered ? data for 62
• NON-randomised
• ?reliability of MCQ test
• ?Generalisability

82
Summary E-Learning and CME

• Certainly can be used to disseminate info
• In keeping with adult learning principles
• Transfer of cognitive component
• ? Psychomotor
• NO change in practice patterns
• Subjective increased self-efficacy

83
Enhancing CME

• Combine LMS with narrated slides
• Provide facilitation for your topic

84

• Narrated
• Options for Quizzes

85
Future Directions

• International Emergency Medicine
• Disseminate info before experts hit the ground
• Collaboration with other programs
• Online Electives?
• EBM
• Toxicology
• Radiology

86
Thoughts?
87
Appendix How you get there
88
Discerning the Context

• Learning Context
• Power Dynamics
• PGME
• Regionalisation
• People dynamics
• Resistance change
• Relatively few champions of Med Ed

89
Supporting Cast

• Stakeholders
• University
• UME /PGME
• RMES
• Planning Committee
• Content experts
• Educational specialists/Instructional design
• IT graphic design/ media
• End-users

90
Identifying Program Ideas

• Core content
• Literature
• Current material
• Needs analysis
• Narrow focus
• Not redundant

91
Sorting and Prioritising

• Prioritise content
• Important?
• Feasible?

92
Developing Objectives

• Needs to be competency based
• Reflect goals of UME/ Colleges

93
Instructional Plan

• How are you going to deliver this?
• Need interaction
• Content, Instructors peers
• Dedicated ED computer
• WEBSITE
• LMS

94
Instructional Plan

• Instructors
• few innovators
• Clinicians Well suited many hats
• Need to train the trainers
• Learners
• Self-directed Computer savvy
• Clinical/ procedural Skills
• Blended approach

95
Instructional Plan

• Variety content
• Anatomy and radiology
• PowerPoint, Flash files, video streaming.
• e-case modules
• LMS ? online discussion
• Logisitics
• LMS powerful tool
• Free ones available ? STARS uses Moodle
• U of C has licenses tech Support
• Regionalisation