Are the Core Clerkship Grades Helpful in the Residency Selection Process

1
Are the Core Clerkship Grades Helpful in the
Residency Selection Process?

• Hiroo Takayama MD Rebecca Grinsell MD
• Douglas Brock PhD Hugh Foy MD
• Carlos Pellegrini MD Karen Horvath MD
• University of Washington, Seattle

2
AAMC Guidelines

• Guide to the Preparation of the Medical School
  Deans Letter in 1989
• Update in 2002
• Renamed Medical Student Performance
  Evaluation (MSPE)
• Ensure consistency across medical schools
• Improve collaboration and communication between
  senders and recipients
• Establish quality improvement process across
  medical schools

http//www.aamc.org/members/gsa/mspeguide.htm
3
AAMC Guidelines
4
MSPE- Sample 1
5
MSPE- Sample 2
6
Study Objective

• We sought to characterize the variability in
  grading systems for the 3rd year core clinical
  clerkships in U.S. medical schools

7
Methods

• Retrospective cross-sectional study
• Review of MSPEs from U.S. medical schools from
  2004 and 2005
• Variables
• Medical school
• 3rd year core clerkship grading system
• of students in each grade category
• Statistics Descriptive analyses repeated
  measure ANOVA

8
Definitions

• Grading System the name of the system used (eg.
  Honors System, Alphabetic System)
• Scores The number of possible grades or
  scores available within each grading system
• 4 scores H, HP, P, F
• 3 scores Outstanding, Satisfactory, Fail

9
Results - Research Population

• 121/122 U.S. Medical Schools
• Data n/a for Mercer Univ SOM, Georgia

10
Variations in Grading Systems

Schools
Grading system
63
76
Honors system
18
22
Letter system
3
4
Pass / Fail
6
6
Other
3
4
No interpretable grading system provided
7
9
No grading system used
100
121
total
11
Variations of the Honors System

• honors only
• honors, pass
• H, S, S H, PC, P
• honors, pass, fail honors, high pass, pass
• honor, credit, no credit honors, near honors,
  pass
• honors, commendation, satisfactory honors,
  satisfactory, existing deficiency
• honors, satisfactory, unsatisfactory high
  honors, excellent, good
• satisfactory with honors, satisfactory,
  unsatisfactory
• H, HP, P, F H, A, B, C HN, LC, S, U
• honors, excellent, satisfactory, no pass
• honors, high satisfactory, satisfactory,
  unsatisfactory
• honors, high pass, satisfactory pass, marginal
  pass
• honors, high pass, pass, conditional,
  fail honors, high pass, pass, marginal
• honors, high pass, pass, low pass, fail honors,
  pass, conditional pass, no pass
• honors, high pass, pass, marginal pass, fail
• honors, commendation, satisfactory, conditional
  unsatisfactory, unsatisfactory
• honors, near honors, satisfactory, marginal,
  fail
• honors, high satisfactory, satisfactory, low
  satisfactory, unsatisfactory
• excellent with honors, excellent, good,
  marginal, unsatisfactory, pass, fail

12
Variations of the Letter Other Systems

• A, B, C
• A, B, C, D
• A, B, C, F
• A, A-, B, B
• A, B, B, C, C
• A, B, C, D, F
• A, B, C, U, E
• A, B, C, D, F, I, R
• A, B, B, C, C, D, E
• A, A, A-, B, B, B-, C, C, C-, D, F
• Outstanding, Advanced, Proficient
• Superior, High satisfactory, Satisfactory, Low
  satisfactory
• Outstanding, Good, Satisfactory, Marginal,
  Unsatisfactory
• Outstanding, Above expected, Expected, Below
  expected, Fail
• continuous number from 0 to 4
• numeric score (0-2, 2-2.5, 2.5-3, 3-3.5, 3.5-4)

13
Variation in of Scales
14
Are different clerkships within an institution
easier or more stringent graders?
15
Variation between Clerkships Using gt3 Scores of
students receiving highest grade
P.001
16
Is grading easier in different geographic regions
of the U.S. or between private vs. public
schools?

17
Comparison between Geographic Regions
50
45
40
35
Mean of students receiving highest grade
All Clerkships
30
25
20
15
10
5
0
South
Northeast
Midwest
West
Total
Geographic Regions
18
Comparison between Private and Public Schools
All clerkships
psychiatry

p.02
surgery
pediatrics
OB/GYN
family medicine
Public school
Private school
internal medicine
0
10
20
30
40
50
60
Mean of of students received highest grade
19
What is the consistency of surgery clerkship
grading across the U.S.?

20
Distribution of Students with the highest grade
in Surgery
20
15
Mean 27.2
of schools
Std. Dev. 13.3
N 93
10
5
0
70
60
50
40
30
20
10
0
of students who received the highest grade
21
of Scores and of Highest Grade
22
Conclusions

• Clinical clerkship grading systems are highly
  variable across U.S. medical schools
• There exists significant variability in the
  chance of receiving a highest grade between
  different clerkships and in different geographic
  regions of the country.
• For these reasons it is very difficult to compare
  students from different medical schools using the
  clerkship grades.

23
Recommendations

• Medical schools comply with AAMC minimal
  standards for the MSPE.
• A more reliable and comparable MSPE
• Might include a uniform grading system
• But would definitely include an ideal number of
  scores
• And the school would reliably show the
  distribution of students in each grade
• In the 21st century of increasing standardization
  and compliance with guidelines, we encourage U.S.
  medical schools to consider these
  recommendations.