Title: Reactions of Surgical Program Directors to a WebBased Interactive Educational Program Focusing on Co
1Reactions of Surgical Program Directors to a
Web-Based Interactive Educational Program
Focusing on Cognitive Skills Donald A. Risucci,
PhD1, Patricia J. Numann, MD2, Richard Welling,
MD3, Marsha F. Pfingsten, MEd4, Patrice Gabler
Blair, MPH4, Ajit K. Sachdeva, MD4 1New York
Medical College, Valhalla NY 2SUNY Upstate
Medical University, Syracuse, NY 3Good Samaritan
Hospital, Cincinnati, OH 4American College of
Surgeons, Chicago, IL
2Quality Quantity of Teaching Learning
Opportunities
Traditional
Non-Traditional
3Quality Quantity of Teaching Learning
Opportunities
Traditional
Skills Labs
Work hour restrictions
Non-Traditional
4Quality Quantity of Teaching Learning
Opportunities
Traditional
Simulation Skills Labs
Work hour restrictions Patient Safety Concerns
Non-Traditional
5Quality Quantity of Teaching Learning
Opportunities
Traditional
Computer-Based Learning Simulation Skills Labs
Work hour restrictions Patient Safety
Concerns Diminished Reimbursement
Non-Traditional
6Quality Quantity of Teaching Learning
Opportunities
Traditional
Immersive technologies Computer-Based
Learning Simulation Skills Labs
Work hour restrictions Patient Safety
Concerns Diminished Reimbursement Increased
Specialization
Non-Traditional
7Quality Quantity of Teaching Learning
Opportunities
Traditional
Web 2.0, 3.0.. Immersive technologies Computer-Ba
sed Learning Simulation Skills Labs
Work hour restrictions Patient Safety
Concerns Diminished Reimbursement Increased
Specialization Advanced Technologies
Non-Traditional
8 Seasonal Variation in Surgical Outcomes as
Measured by the American College of
Surgeons-National Surgical Quality Improvement
Program (ACS-NSQIP) Ann Surg. 2007 September
246(3) 456465. Michael J. Englesbe, MD,
Shawn J. Pelletier, MD, John C. Magee, MD, Paul
Gauger, MD, Tracy Schifftner, MS, William G.
Henderson, PhD, Shukri F. Khuri, MD, and
Darrell A. Campbell, MD From the Department
of Surgery, University of Michigan Health System,
Ann Arbor, MI Department of Veterans Affairs,
VANational Surgical Quality Improvement Program,
Aurora, CO and National Surgical Quality
Improvement Program, West Roxbury, MA.
9(No Transcript)
10(No Transcript)
11 Fundamentals of Surgery Curriculum American
College of Surgeons Division of
Education www.facs.org/education/fundamentalsofsu
rgery.html
12Focus on Diagnosis
Diagnosis errors far outnumber medication errors
as a cause of claims lodged (26 v. 12 in one
study Sato, 2001) 32 v. 8 in another study
(Phillips et al 2004). Harris poll commissioned
by National Patient Safety Foundation one in six
people had personally experienced a medical error
related to misdiagnosis (Golodner 2004). Most
medical error studies find that 1030 percent
(range 0.656.8 percent) of errors are errors
in diagnosis. A recent review of 53 autopsy
studies found an average rate of 23.5 major
missed diagnoses (range 4.149.8 percent).
Disease-specific studies show that substantial
percentage of patients (range 2.1 61)
experienced missed or delayed diagnoses.
Schiff et al, 2005
13Fundamentals of Surgery Curriculum
- Case-based interactive curriculum
- Delivered on-line to PGY-1 surgical residents
- Each scenario presents a patient resident can
take a history, perform a physical
examination, order tests, review records, and
recommend therapeutic interventions. - Toolbar allows access to the patients virtual
chart, including appropriate reports, and
archives the information gathered during the
patient history, physical examination, and
diagnostic testing.
14Fundamentals of Surgery Curriculum
- Specific feedback for every action taken while
interviewing, examining, diagnosing and managing
the patient. - If a residents actions in caring for a virtual
patient result in dire consequences, the resident
is required to start over. - Upon completion of FSC, the resident will have
diagnosed and recommended treatment for scores of
patients
15Fundamentals of Surgery Curriculum
- Learning exercise, not a summative assessment
tool residents can explore various alternatives
and receive feedback without concern about being
formally evaluated. - When complete, FSC will be composed of 11
on-line modules with approximately 100
interactive case-based scenarios representing the
foundation of surgical cognitive content for
PGY-1 residents across all surgical specialties.
16A Scenario may open with a Learning Objective
Page, indicated by a Surgeon behind a Podium.
17This is a Reference Page, indicated by a Surgeon
with an Open Book.
18Scenario Authors provide a detailed description
of the patient. Images are created to exhibit
signs that the Resident is expected to notice and
to incorporate into the diagnostic process..
19Question
Possible Choices
Progress Meter
key
When the Resident opens a work space, a list of
choices is presented, a key to the grade that
each choice can receive, and a Progress Meter..
20When a selection is made, in this case a question
to be posed to the patient, the response appears
as illustrated above.
21In this case, a critical question was selected,
as indicated by the green dot. The progress
meter is partially full, indicating that
additional critical choices must be made before
the Resident can continue.
grade
Progress Meter
22After all critical choices have been made, the
Progress Meter registers 100, The Resident is
allowed to Continue to the next step in
evaluating the patient.
After all correct responses have been selected
23The Resident can review the patients responses
to the history by selecting the Patient History
and Image Tab from the Patient Section of The
Toolbar. The Toolbar represents the patients
Virtual Chart.
24The Resident can review the patients Vital signs
by selecting the Vital Signs Tab from the Patient
Section of The Toolbar.
25Reports and test results which are in the
patients chart at the time of the encounter are
made available in the Records Section and the
Tests Section of the Toolbar at the start of the
branch. If a report or test is ordered within a
branch, it is made available at the appropriate
time.
26Each selection receives a response that not only
indicates whether it is right or wrong, but
also provides feedback designed to enhance the
educational experience for the Resident.
27After the Resident has successfully completed the
scenario, a Summary is presented. If the
Learning Objective was not presented at the
beginning of the scenario, it is presented here.
28Key Learning Points that have been demonstrated
within the scenario are listed after the Summary,
to reinforce the learning
29A Resident may choose to review the Scenario at
this time, or to return to the Menu for another
scenarios. All scenarios, as well as all
reference material, tables, and charts, remain
available to each Resident for the entire year.
Feedback
The Fundamentals of Surgery Curriculum is an
educational program, not a test. For that
reason, and to encourage each Resident to
investigate each scenario completely, personal
feedback is provided to the Resident however
this is not tracked in any way.
30Current Study
- Evaluates Program Directors reactions to a
preview of a scenario from the Fundamentals of
Surgery Curriculum.
31Methods
- Program Directors were invited via the Listserv
to preview the Fundamentals of Surgery Curriculum
during the October 2007 ACS Clinical Congress. - Previews were scheduled for 31 Program Directors
on a first-come first-serve basis. - Each Program Director spent 1 hour engaged in
the scenario
32Methods
- After engaging in a scenario each Program
Director completed a questionnaire requesting - Age range
- Rating of comfort using computers
- Ratings of the scenarios utility in addressing 9
broadly defined educational goals (e.g., Provides
a Solid Foundation for Future Learning) - Ratings of 6 separate features of the scenario
(e.g., Ease of Use, Feasibility). - Informal debriefings were also conducted and
comments were collected.
33(No Transcript)
34(No Transcript)
35Program Directors with higher levels of comfort
using computers tended to perceive the modules as
easier to use (p lt 0.01)
36Ratings of Features
- Ease of use (logical flow and methods of
navigation) - Very difficult to use Extremely easy to use
Usefulness of multimedia components to present
material and enhance understanding Does not
enhance presentation Extremely useful
Quality of presentation (appealing color schemes,
format, and graphics) Very Poor Exceptional
37Ratings of Features
- Extent to which interactivity within the scenario
promotes - engagement in solving the problem presented
- Not at all engaging Made me want to solve
the problem
Extent to which immersion in this virtual
experience adequately recreates the environment
in which the scenario takes place Felt very
artificial Felt real
Feasibility of incorporating scenarios like this
one into the required PGY-1 curriculum Will not
be feasible Should be part of required
curriculum
38Ratings of FSC Utility in Addressing Educational
Goals
- Prepares residents to apply critical thinking
and judgment essential in clinical situations
during the PGY-1 year. - Improves residents patient care skills and
confidence during the PGY-1 year. - Delivers content consistent with current
practices and/or evidence (when available).
39Ratings of FSC Utility in Addressing Educational
Goals
- Provides a solid foundation for future learning.
- Challenges residents.
- Encourages self-assessment and reflection.
- Motivates residents to voluntarily complete
other scenarios. - Improves residents teamwork and
professionalism. - Increases resident satisfaction with the
educational process.
40Ratings of the Scenarios Utility for Addressing
9 Broadly Defined Educational Goals
41Comments Informal Debriefing
- Scenarios were fun and interesting.
- Felt as if they were diagnosing a real patient
very involved in scenario. - Several asked about the availability of other
modules. - Enables delivery of useful information
consistently and efficiently. - Helps to address educational objectives within
the 80 hour work. - Exposes residents to cases they may not actually
encounter.
42Comments Informal Debriefing
- Difficulties with the original physical
examination model were recorded and have led to
enhancements in that component of the scenarios. - Most Program Directors indicated interest in
participating in future content reviews, beta
testing, and research initiatives. - Some Program Directors indicated that they would
like to require residents to complete FSC prior
to entering their program.
43Evaluation Framework
- Reactions Perceptions of stakeholders users
- Learning Cognitive gains from engaging in the
curriculum - Behavior Changes in clinical performance
resulting from exposure to curriculum - Outcomes Effects on patient outcomes, quality
improvement
44Conclusions
- Program Directors generally perceived the
Fundamentals of Surgery Curriculum as a very
feasible and effective tool addressing important
educational needs of entering PGY-1 residents. - Positive reactions from Program Directors suggest
that the Fundamentals of Surgery Curriculum are
likely to be implemented and evaluated in a
significant number of programs.