Interactive Case-based Assessment Tool (iCAT) Bringing Together the

1
Interactive Case-based Assessment Tool(iCAT)
Bringing Together the What and Howof
Learning
Wendy Ham, Andrew Rosenfeld, Lukasz Babiarz,
Sandy Mong, Chris Dede, Susan Pasquale, Robert
Leffert,  and James Quattrochi Harvard
University, ICON Program for Multidisciplinary
Learning, Cambridge, MA University of
Massachusetts Medical School, Worcester,
MA Supported by the Harvard Provost Innovations
in Instructional Technology Award
2
Background

• We have developed a case simulation method,
  called ICON (Interactive Case-based Online
  Network)
• In ICON, students interact with case characters
  online
• Case characters (patient, physicians, case
  consultants, family members) are assumed by
  faculty members
• Events in ICON unfold in real time
• Students learn to
• Integrate the theory and practice of medicine
• Solve problems
• Navigate resources
• Make decisions
• Develop teamwork skills
• Be accountable to case characters and teammates
• How do we measure the competencies listed above?
  

3
Survey of Student Perception of ICON
Hypothesis 1 ICON facilitates a learning
process that is more engaging
Hypothesis 2 ICON increases peer contribution
in learning
Hypothesis 3 ICON increases active faculty
participation in student learning
Hypothesis 4 ICON decreases learning
dependence on faculty lecturer due to increased
contribution by peer and faculty mentor
4
Survey Method
Comparison of entrance and exit surveys from two
years of ICON implementation in the second-year
Human Nervous Systems and Behavior course at
Harvard Medical School
of Responses ( Response)
Total 40 (71) 29 (52)
5
Student Perception of ICON Survey Results
Hypothesis 1 ICON facilitates a learning
process that is more engaging hours spent
discussing tutorial cases outside of tutorial
session
P0.004
Without ICON With ICON
6
Student Perception of ICON Survey Results
Hypothesis 2 ICON increases peer contribution
in learning Mean ranking of peer relative to
faculty mentor and faculty lecturer in
contribution to learning
P0.02
Mean ranking
Without ICON With ICON
Min value 1 Max value3
7
Student Perception of ICON Survey Results
Hypothesis 3 ICON increases active faculty
participation in student learning Mean agreement
rating to the statement that faculty (as case
characters) actively participate in student
learning
P0.00003 1completely disagree 2somewhat
disagree 3somewhat agree 4completely agree
Agreement rating
Without ICON With ICON
Min value 1 Max value4
8
Student Perception of ICON Survey Results
Hypothesis 4 ICON decreases learning dependence
on faculty lecturer due to increased
contribution by peer and faculty mentor Mean
ranking of faculty lecturer relative to peer and
faculty mentor in contribution to learning
P0.009
Mean peer ranking
Mean ranking
9
OK so students think ICON

• is engaging
• increases peer contribution
• promotes active faculty participation
• decreases relative role of lectures

But how does ICON really add value in their
learning?
10
iCAT Interactive Case-based Assessment Tool

• Characterization of process, not only outcome
• Case-based
• Emphasis on problem solving in the context of
• Hypothesis formulation
• Diagnosis and treatment plan
• Risk management

11
A 42-year-old uninsured woman, who had not seen a
doctor in 20 years, presented to the emergency
department (ED) at 750 a.m., complaining of
chest pain and trouble breathing. She was
accompanied by her husband and her son, who
helped interpret because English was her second
language. The husband reported that his wife
had been experiencing chest pain since 1100 the
prior evening, which she treated with aspirin. In
the morning, she had sudden onset of severe chest
pain and fainted on her bed. The patient was
first seen by the ED attending and then by a
resident. Her initial vital signs were HR 107,
BP 146/99, RR29. Her chest pain was documented
as sudden onset, right-sided, sharp, under the
right breast, started while the patient was lying
in bed and worse with inspiration, movement, and
palpation. The patients medical history was
documented by the resident as fainting spells,
no family history of coronary artery disease or
clots, father suffered a stroke.
Toolbox
Background
HP
Test
Consult
Notebook
Incurred cost 0.00
Select a tool from the Toolbox
Time 000017
12
English as a second language
Click on a keyword for more information
Is there someone with a medical background who
can reliably interpret for the patient? Is
English the sons/husbands first language? Was
the history documented with the presence of an
interpreter other than family members? Was there
any ambiguity in taking the patients history?
A 42-year-old uninsured woman, who had not seen a
doctor in 20 years, presented to the emergency
department (ED) at 750 a.m., complaining of
chest pain and trouble breathing. She was
accompanied by her husband and her son, who
helped interpret because English was her second
language. The husband reported that his wife
had been experiencing chest pain since 1100 the
prior evening, which she treated with aspirin. In
the morning, she had sudden onset of severe chest
pain and fainted on her bed. The patient was
first seen by the ED attending and then by a
resident. Her initial vital signs were HR 107,
BP 146/99, RR29. Her chest pain was documented
as sudden onset, right-sided, sharp, under the
right breast, started while the patient was lying
in bed and worse with inspiration, movement, and
palpation. The patients medical history was
documented by the resident as fainting spells,
no family history of coronary artery disease or
clots, father suffered a stroke.
Toolbox
Background
HP
Test
Consult
Notebook
Incurred cost 0.00
Time 000135
13
English as a second language
Click on a question for more information
A reliable interpreter with a medical background
is not available.
Is there someone with a medical background who
can reliably interpret for the patient? Is
English the sons/husbands first language? Was
the history documented with the presence of an
interpreter other than family members? Was there
any ambiguity in taking the patients history?
Save in Notebook
A 42-year-old uninsured woman, who had not seen a
doctor in 20 years, presented to the emergency
department (ED) at 750 a.m., complaining of
chest pain and trouble breathing. She was
accompanied by her husband and her son, who
helped interpret because English was her second
language. The husband reported that his wife
had been experiencing chest pain since 1100 the
prior evening, which she treated with aspirin. In
the morning, she had sudden onset of severe chest
pain and fainted on her bed. The patient was
first seen by the ED attending and then by a
resident. Her initial vital signs were HR 107,
BP 146/99, RR29. Her chest pain was documented
as sudden onset, right-sided, sharp, under the
right breast, started while the patient was lying
in bed and worse with inspiration, movement, and
palpation. The patients medical history was
documented by the resident as fainting spells,
no family history of coronary artery disease or
clots, father suffered a stroke.
Toolbox
Background
HP
Test
Consult
Notebook
Incurred cost 0.00
Time 000209
14
Test
ECG
Chest X-Ray
ECG
Lab Tests 650 Blood gassesAcetoneArteritis
(ANA)CO2EKGGlucosePotassiumSodiumLiver
(SGOT)Liver (SGPT)StoolType cross blood
Studies/Tests 1,000 Angiogram Audiometry
Biopsy Bone Scan Cardiac Eval Cisternogram
CSF Studies CT EEG ECG EMG/NCS ENG Evoked
Potentials Genetic Tests Ischemic Exercise Test
MRA MRI Muscle Biochem Myelogram Neuropsych
Testing Pap Smear Pulmonary Function Schilling
Test SPECT Scan Tensilon Test Visual Fields
X-rays
Click to enlarge
Save in Notebook
Toolbox
Background
HP
Test
Consult
Notebook
Incurred cost 1,650.00
Time 000742
15
Test
ECG
Chest X-Ray
ECG
Lab Tests 650 Blood gassesAcetoneArteritis
(ANA)CO2EKGGlucosePotassiumSodiumLiver
(SGOT)Liver (SGPT)StoolType cross blood
Studies/Tests 1,000 Angiogram Audiometry
Biopsy Bone Scan Cardiac Eval Cisternogram
CSF Studies CT EEG ECG EMG/NCS ENG Evoked
Potentials Genetic Tests Ischemic Exercise Test
MRA MRI Muscle Biochem Myelogram Neuropsych
Testing Pap Smear Pulmonary Function Schilling
Test SPECT Scan Tensilon Test Visual Fields
X-rays
Click to enlarge
Save in Notebook
Toolbox
Background
HP
Also notice the timer and cost counter
Test
Consult
Notebook
Incurred cost 1,650.00
Time 000742
16
Notebook
Chest pain Shortness of breath Stroke in family
history Absence of trained interpreter Age Aspirin
Gender
Background
Final vital signs Initial vital signs
HP
ECG No cardiac enzymes results Elevated
glucose X-Rays Blood gases
Test
Absence of cardiology consult
Consult
17
Notebook
Key Student
Chest pain Shortness of breath Stroke in family
history Absence of trained interpreter Age Aspirin
Gender
Background
Why does the student not consider these high risk?
Final vital signs Initial vital signs
HP
ECG No cardiac enzymes results Elevated
glucose X-Rays Blood gases
Test
Absence of cardiology consult
Consult
18
Model of Students Rating Profile during Case
Progression
Profile of Students Rating during Case
Progression
Cumulative value of information accessed (Scale
1-5)
(High) 5
4
Student X
(Moderate) 3
2
What causes the profile to shift?
Baseline
(Low) 1
Time (seconds)
0
(Start)
(End)
19
iCAT beta
20
Instructional technology impacts the process of
learning, which impacts the outcome of learning
Design, Development, and Implementation of
Instructional Technology
Process of Learning
Outcome of Learning
Impact Inform
21
Impact is tightly coupled with information,
allowing progress in educational development to
be driven by hypothesis
Design, Development, and Implementation of
Instructional Technology
Process of Learning
Outcome of Learning
Impact Inform
22
Contact Information

• http//icon.fas.harvard.edu
• wendyham_at_gmail.com