American Board of Surgery Examinations: Can We Identify Surgery Residency Applicants Who Will Pass t

1
American Board of Surgery Examinations Can We
Identify Surgery Residency Applicants Who Will
Pass the Examinations on the First Attempt?

• John L. Shellito, M.D., Jacqueline S. Osland,
  M.D., Stephen D. Helmer, Ph.D., Frederic C.
  Chang, M.D.
• Department of Surgery
• University of Kansas School of Medicine Wichita

2
Introduction The Importance of Board
Certification

• One of the key demonstrations of the
  effectiveness of a residencys training program
• ACGME Program Requirements-Surgery
• 65 first attempt pass rate for both the
  qualifying and certifying exam

3
Objective

• This study was performed to identify and evaluate
  factors which predict successful completion of
  both the American Board of Surgery Qualifying
  Exam (QE) and Certifying Exam (CE) on the first
  attempt

4
Methods

• Retrospective review
• University of Kansas SOM-Wichita Surgery
  Residency Program
• Included residents entering the program from
  1990-2001

5
Pre-Admission Measures

• Age and gender
• Class rank
• AOA status
• Honors in third year surgery clerkship
• Interview scores
• National Board of Medical Examiners (NBME) Part 1
  and Part 2 scores or United States Medical
  Licensing Examination (USMLE) Step 1 and Step 2
  scores
• Rank list number

6
Post-admission Measures

• American Board of Surgery In Training Examination
  (ABSITE) scores
• Resident awards (Research, Teaching, Outstanding
  Senior Resident)
• Faculty evaluations of the 5th year residents

7
Statistical Analysis

• Univariate analyses were performed using
  Fischers exact test for the dichotomous
  variables (i.e. gender, AOA status, honors)
• Two-sample Wilcoxon rank-sum test was used for
  continuous variables (i.e. scores, age) and
  ordinal data
• Logistic regression analysis was performed on the
  pre-admission variables

8
Results

• 77 medical students and residents
• 15 subjects excluded
• 5 NBME or USMLE scores not available
• 8 Left the program
• 2 never attempted the QE (both plastic surgeons)
• 62 subjects remained after exclusions

9
UKSM-W Surgery Residency Program 1990-2001

• gt90 of our graduates attended medical school in
  the Midwest (50 from KU)
• One foreign medical graduate
• 16 (10/62) AOA
• Pass rates
• First time 77.4
• Eventual 96.8
• 2/62 (plastic surgery)
• Majority entered private practice (73)
• No difference in first time pass rates over the
  study period

10
Results

• No significant difference based on gender
• 50 (80.7) were male
• First time pass rate
• 74 for males/ 91.7 for females (p0.179)
• Residents who successfully passed were slightly
  younger at time of application
• 27.1 years vs. 28.6 years (p0.047)

11
P0.0073
12
P0.0247
13
P0.0447
14
Results

• Cumulative scores for step 1 and 2
• gt382 (top 2/3rds)
• 81.1 first time pass rate
• lt382 (bottom 1/3rd)
• 42.9 first time pass rate

15
Results

• Logistic regression analysis showed Step 1 scores
  to be the only independent predictor from the
  group of pre-admission factors for first time
  pass rates

16
(No Transcript)
17
(No Transcript)
18
Results

• Residents winning an award during residency had a
  higher first time pass rate
• 92 versus 67
• Faculty evaluations of 5th year residents did not
  predict board success

19
Results

• Pre-admission factors that failed to predict
  success for first time pass rates
• Honors in 3rd year surgery clerkship
• AOA status
• 10/10 first time pass rate, p0.061
• Interview score
• Ranking in the match

20
Discussion

• Clerkship grades
• Honors grade in third year surgery clerkship
• Deans letters
• Letters of recommendations
• Interview Score
• Class rank
• USMLE scores

21
Discussion

• Xu 1998
• Positive correlation between eventual board
  certification status and past academic
  performance
• Included other surgical specialties in addition
  to general surgery
• Silber 2005
• Step 2 score single most important predictor for
  eventual OB/GYN board certification
• Graduates from a single medical school, multiple
  residency programs

22
(No Transcript)
23
Summary

• This is first study to examine factors predicting
  first time board certification rates for general
  surgery residency graduates for both the QE and
  CE

24
Limitations

• Small sample size
• Bias injected from our selection process
• No information about the success or failure of
  applicants who were excluded by our selection
  committee
• Single residency training program

25
Summary

• Academic aptitude by itself does not predict the
  multifaceted state of competence of a surgeon
• Professionalism, manual dexterity, communication
  abilities, and integrity are not measured by
  standardized tests
• Sorting applicants that are above a certain
  threshold of intellectual capacity solely by test
  scores may not be a good way to obtain the best
  residents

26
Conclusion

• Our study indicates a positive correlation
  between class rank, USMLE scores, ABSITE scores
  and first time pass rates for ABS certification

27
(No Transcript)