Surgical Resident Peer Evaluations

1
Surgical Resident Peer Evaluations
What have We Learnt
2
Vijay K. Maker, M.D., F.A.C.S.,
University of Illinois, Metropolitan
Group Hospitals Residency in General Surgery,
Chicago, IL Michael B. Donnelly, Ph.D.,
Director, Surgery Educational Research
Evaluation, University of Kentucky, Lexington, KY
3
(No Transcript)
4
(No Transcript)
5
PURPOSE

• To identify the attributes of a junior and senior
  residents who are perceived to be role models...

• To determine which of these attribute(s) is the
  keystone

6
METHODS

• Develop customized peer evaluation tools 2

7
3
2
1
Teaching (Outside OR)
Lectures Teaching Rounds Daily Encourages
literature search Stimulates thinking
Technical/Cases (In OR)
Confident/prepared for OR Take juniors through
cases Offers opportunities for Juniors to perform
surgery/skills Ombudsman for Juniors to attending
Professionalism
Altruism Dress up/clean cut for morning report On
time Excellent bedside manner Addresses
colleagues with professionalism Does not take
advantage of the seniority
Attitude/Expectations
Rarely complains Shows enthusiasm Respect to
junior and students Runs a tight ship with little
room for slacking, mistakes or inefficiency. Defin
es expectations when the rotation starts. Fair
and listens to what juniors say Cares about the
well being of the junior
Approachability/Advisor
Available 24 hours/day to discuss patient care or
personal matter. No problems with juniors calling
home Offer advice and guidance with
career Represents juniors well at SJCC and
attendings
Summary
Comments are appreciated by the person being
evaluated, as they indicate areas of self
improvement.
8
3
2
1
Responsible
Trustworthy Honest Prompt/Punctual Humble Efficien
t
Attitude
Altruism Enthusiastic Team Player Communicates
Well Compassion Liked by ancillary staff
Knowledge
Reads about cases Reads about patients Eager to
learn Asks intelligent questions Knows patient
details like labs, x-rays, etc.
Summary
Comments are appreciated by the person being
evaluated, as they indicate areas of self
improvement.
9

• 15 senior residents were evaluated by the junior
  residents using the six item scale.
• They were evaluated by from 3 to 9 junior
  residents.
• There were 81 evaluations.
• 12 junior level residents were evaluated on the
  four item scale by the senior residents.
• They were evaluated by from 3 to 6 senior
  residents.
• There were 41 evaluations.

10
Report Cards
11
Evaluation of Seniors By Juniors
3.00
2.75
2.50
2.25
2.00
1.75
1.50
1.25
1.00
Professional
Att-Expect
Approach
Teaching out OR
Role Model
Tech/Cases In OR

O.K. SMITH
GOLD STANDARD
12
Evaluation of Juniors by Seniors
RISINGSTAR JILL
GOLD STANDARD
13
Evaluation of Juniors by Seniors
PROBLEM CHILD
GOLD STANDARD
14
Comments

• Good knowledge, No clinical sense!!

• Dual faced, Disappearing act after attending
  leaves.

• Scared but great at work.

• Stink.. Please use deodorant..

• Best senior ever

15
2006 PEER EVALUATIONS Statistical
Analyses Opportunities for Improvement
16
Senior ROLE MODEL Score
KK
SP
MK
RN
IT
3.00
JL
SS
2.80
TW
2.60
KW
AV
AB
JT
2.40
2.20
2.00
1.80
1.60
1.40
1.20
1.00
17
EVALUATIONS OF JUNIORS by Seniors
3.00
2.75
2.50
2.25
2.00
1.75
1.50
1.25
1.00
1
2
3
4
1. RESPONSIBLE
1
2. ATTITUDE
2
3. KNOWLEDGE
3
4. ROLE MODEL
4
18
Peer Evaluations Relationship to Absite Score
2005 Absite Score
Senior Evaluations
19
Mean Ratings 95 CI for Senior Residents
Role Model
Not Role Model
Performance Characteristic
20
Mean Ratings 95 CI for Junior Residents
Not a Role Model
Role Model
Performance Characteristic
21
Figure 4. Partial Eta Squared for Differences
Between Role Model and Non- Role Model Senior
Residents
0.40
0.30
Mean partial h2
0.20
0.10
0.00
Technical/ Cases in OR
Teaching Outside OR
Professionalism
Attitude/ Expectations
Approachability/ Advisor
Performance Characteristic
22
Partial Eta Squares for the Differentiation of
PerformanceCharacteristics of Role Model and
Non-Role Model Junior residents
.60
.50
.40
.30
.20
.10
0
Attitude
Knowledge
Responsible
Performance Characteristic
23
p-value from t-test
Not Role Model Mean
Role Model Mean
Performance Characteristic
lt.001
2.40
2.90
Teaching (outside OR)
lt.001
2.60
2.95
Technical/Cases (In OR)
lt.001
2.40
2.96
Professionalism
lt.001
2.20
2.94
Attitude/Expectations
lt.001
2.70
2.99
Approachability/Advisor
24
Partial Eta Squared
p-value from t-test
Not Role Model Mean
Role Model Mean
Performance Characteristic
.47
lt .001
2.12
2.95
Responsible
Attitude
.52
lt .001
1.94
2.98
Knowledge
.21
lt .001
1.94
2.97
25
CONCLUSIONS
Both junior and senior residents value
professionalism in each group as the most
important differentiating attribute of a role
model
26
CONCLUSIONS
27
Seniors felt that knowledge was a less important
attribute of a junior role model.