Orthopaedic Resident Selection Criteria

1
Orthopaedic Resident Selection Criteria

• Chris Anderson

2
Introduction

• Orthopaedics is consistently one of the most
  competitive specialties
• In 2000 there were 1116 applicants for 554 PGY-1
  spots (both foreign and US grads)
• 30 yrs ago only 2/3 positions filled, last year
  98 filled.
• Trends reflect increasing popularity

3
Introduction (cont.)

• Oversupply of outstanding applicants creates
  intense competition among students
• In past students have relied heavily on rumor and
  anecdotal information
• Currently, there are several articles that give
  insight to what program directors value most
  highly in a candidate

4
Bernstein Article (JBJS 2002)

• Wagoner and Suriano assessed program directors on
  resident-selection criteria, however less than
  1/3 sampled returned questionnaire results
  statistically invalid
• Clarke evaluated characteristics of successful
  applicants, but not data on program directors
• Concluded lack of data that evaluate
  resident-selection process across multiple
  programs

5
Methods (Bernstein JBJS 02)

• Two part questionnaire sent to 156 ortho
  residency programs
• Part I
• List of 26 items based on what has been used in
  previous reports and some specialty specific
  selection criteria
• Rated importance on scale 1 to 10
• Part II
• Four multiple choice and four true-false
  questions on opinions of importance of certain
  aspects of criteria PS, letters, interview, and
  AOA
• Asked to indicate current residents that were
  AOA, and that had completed a rotation at the
  institution as a medical student

6
Results (Bernstein JBJS 02)

• 109/156 program directors completed evaluation
  (70) therefore statistically valid follow-up

7
(No Transcript)
8
(No Transcript)
9
True/False Questions

• Psychologist/Psychiatrist involved in interview
  process- 2
• Manual skills exam- 5
• Clinical Scenarios- 18
• Candidates considered equal once selected for
  interview- 22

10
AOA membership and Away Rotations

• Matching residents over last three years that
  were AOA - 54
• Programs with at least 50 membership in AOA
  among residents matching over last three years
  65
• Residents over last three years that performed
  rotation at institution they eventually matched
  to 56
• Programs with at least 50 of matched residents
  over last three years performing rotation at the
  program prior to match 66

11
Interpretation of Results

• Rotation at institution rated most important
  criteria, 56 residents that matched did rotation
  at the institution
• Affective domain personal/professional values
• Best evaluated by personal interaction and direct
  observation
• Other data
• Simon - argued best chance of matching was doing
  an away at an institution
• Wagoner found 86 program directors gave
  preference to students that performed well on
  elective at institution
• Bajaj et al reported performance on externship
  was rated as most important criteria for
  residency selection

12
Interpretation of Results

• USMLE I Score rated 2nd most important
• To maintain accreditation programs must have 75
  pass rate on orthopaedic boards part I
• Ronai et al, 1984
• Found USMLE I and II both predictors of OITE
  performance
• Dirschl et al, 2002
• Found no correlation of USMLE I or II with OITE
  or ABOS
• Klein et al, 2004
• Found low to moderate correlation, although
  significant, with USMLE I and ABOS I
• Black et al, 2006
• Found moderate correlation with USMLE II and
  OITE, but no correlation of Step I and OITE

13
Interpretation of Results

• Rank in Medical School 3rd most important but
  for us its N/A
• Interview Components are ranked 4th, 5th, and
  6th most important
• Clark et al, 1989 found very unreliable predictor
  
• Simon et al, 2001 found gave no indication of
  ethics or professionalism
• Formality/politeness
• Personal Appearance
• Performance on ethical questions

14
Interpretation of Results

• Letters of recommendation 7th
• Clark et al, 1989 found no correlation of letters
  with performance
• Dirschl and Adams, 2000 showed significant
  variability in interpretation
• Dirschl et al, 2002 also found no correlation
  with performance
• Good letters come from good work
• Do well on 3rd and 4th year ortho rotations
• ID a mentor, do research with them starting early
  
• AOA 8th
• Dirschl et al, 2002 found AOA second strongest
  predictor of performance
• Important but not necessary
• About ½ people that matched in last three years
  were not AOA.

15
Interpretation of Results

• Medical School Reputation 9th
• Vanderbilt highly respected, esp in southeast
• Deans letter 10th
• Literature describes as vague, inconsistent, and
  unreliable in predicting performance
• Basically your CV written out in text by dean
  Rodgers

16
Interpretation of Results

• Personal Statement 11th
• 75 of directors used PS to see if person can
  write/communicate and what their
  background/interests are
• 20 thought not very important

17
Interpretation of Results

• Failed attempt 12th
• Phone call 13th
• Research 14th, 15th
• Accreditation council recommending more research
• Most programs are mandating published articles
  during residency
• Bajaj et al, 2004 - Likely to become a more
  important variable in next 3-5 years
• Larger, more academic programs place more
  emphasis on research

18
Interpretation of Results

• Letter of rec by non-orthopod 16th
• MD/PhD 17th
• Reputation of undergrad 18th
• Undergrad GPA 19th
• Appearance of CV 20th
• Letter of rec by senior resident 21st
• Relative affiliated with institution 22nd
• Engineering major 23rd
• Thank-you letter 24th
• Performance on manual skills 25th
• Evaluation by psychologist/psychiatrist 26th

19
Conclusions

• Bottom line everyone has good applications, so
  all selection criteria are important, so dont
  neglect less important criteria
• Minimum USMLE score?
• Klein et al, 2004 found step I of below 204 had
  16 chance of failure of ABOS I, and a failure
  rate 2.63 times higher than students above 204
• Bajaj et al, 2004 found average step I for
  applicants ranked at UTMB then matched in any
  program was 236, average for those that did not
  match was 215
• Bajaj also found faculty rated Step I and II 4th
  and 5th respectively in attributes that were
  important to ranking residents

20
Conclusions

• Bajaj et al, 2004 Minimum Step I scores by
  orthopaedic faculty in all Texas programs
• Below 200 was rated as too low to secure position
• 201-220 rated by 29 as minimum
• 221-230 rated by 48 as minimum
• 231-240 rated by 16 as minimum
• 240 rated by 7 as minimum
• Step II minimum scores
• lt200 acceptable to 0
• 201-220 acceptable to 33
• 221-230 acceptable to 50
• 231-240 acceptable to 12
• gt240 acceptable to 5

21
Conclusions

• Clerkship grades found to be important predictor
  of performance used by many programs

22
Thanks!