DOES ACADEMIC INTERVENTION IMPACT ABS QUALIFYING EXAM RESULTS

1
DOES ACADEMIC INTERVENTION IMPACT ABS QUALIFYING
EXAM RESULTS?
KAREN R. BORMAN, MD UNIVERSITY OF MISSISSIPPI
MEDICAL CENTER
2
COMPETENCY
3
RESIDENCY COMPETENCY
PD
RESIDENCY REVIEW COMMITTEE for SURGERY
4
RESIDENCY COMPETENCY
PR I.A The goal of a surgical residency program
is to prepare the resident to function as a
qualified practitioner of surgery at the high
level of performance expected of a
board-certified specialist.
PR VII.C.3 The performance of program graduates
should be used as one measure of evaluating
program effectiveness. At minimum, for the most
recent 5-year period, 60 of the graduates must
pass the qualifying and certifying examinations
on the first attempt.
5
ABSITE
RRC for SURGERY
6
ABSITE QUIZ TRUE OR FALSE

• PURPOSE TO TERRORIZE RESIDENTS
• PURPOSE TO TORTURE PD
• TO DEFINE CONTENT DOMAIN KNOWLEDGE COMPETENT
  GENERAL SURGEON
• TO MEASURE RESIDENT KNOWLEDGE EARLY ALLOW
  SUCCESSFUL REMEDIATION
• TO HELP PD DETECT PROGRAM STRENGTH WEAKNESS
• ABS STARTED 1975 RESPONSE QE FAIL RATE
• PURPOSE TO PREDICT QE1 SUCCESS

GROSS, 1980
7
gt 90 PASS
CUMULATIVE ABS EXPECTANCY 1993-2004
8
PGY-5 ABSITE
ABS QE1ABSITE INTERVENTIONS RESIDENCY
WIDE

• ASSIGNED TEXT READING ? QUIZZES
• DEAN (1984), BERRY (1984), HIRVELA (1991),
  diVIRGILIO (2003)
• CONFERENCE CHANGES
• ITANI (1997), GODELLAS (2000)
• SET ABSITE STANDARD
• POFAHL (2001)
• IMMEDIATE POSITIVE IMPACT ABSITE
• NO LONG-TERM OR QE DATA

9
1999
2001
1996
25 SD SS
10
(No Transcript)
11
FOCUSED ACADEMIC INTERVENTIONPROGRAM STRUCTURE

• HYPOTHESIS AI for RESIDENTS with TT ABSITE lt 30
  WILL INCREASE QE PASS
• INITIAL ELEMENTS MENTORED PLP, VIDEOTAPE
  COURSE, MORE FREQUENT EVALUATIONS, SESAP, PD
  FEEDBACK
• ENHANCEMENTS PE, PSYCHOLOGIST MOCK ORAL, POST
  AI MENTORING
• SATISFACTORY COMPLETION NEXT ABSITE TT
  PERCENTILE gt 31

12
FOCUSED ACADEMIC INTERVENTIONPARTICIPANTS AND
PEERS

• FIRST AI RESIDENTS COMPLETED CHIEF YEAR 2003
• MOST RECENT AI CHIEF 2005
• 4 MORE IN PROGRESS NOT REPORTED
• STUDY PERIOD 2003-2005
• 15 CHIEFS
• ALL FINISHED IN 5 YEARS
• NO ATTRITION
• QE TAKEN FIRST OPPORTUNITY

13
FOCUSED ACADEMIC INTERVENTIONPARTICIPANTS AND
PEERS

• NO INTERVENTION (NI) GROUP n 7
• ACADEMIC INTERVENTION (AI) GROUP n 8
• 8 RESIDENTS / 9 INTERVENTIONS
• PGY2 (1) PGY3 (1) PGY4 (6) PGY5 (1)
• 1 RESIDENT 2 INTERVENTIONS PGY3 5
• 7/8 (87) RESIDENTS and 8/9 (89) INTERVENTIONS
  SATISFACTORY

14
FOCUSED AI ABSITE RESULTS
15
FOCUSED AI ABSITE RESULTS
GAINS IN STANDARD SCORE and CORRECT ANSWERS
MEDIAN TOTAL TEST PERCENTILES
16
ABSITE CORRELATIONS

• MEDICAL STUDENT PERFORMANCE
• ERLANDSON 1982 ?
• CONFERENCE ATTENDANCE
• SHETLER 1982 ? , POLLAK 1988 - , ITANI 1997
• ROTATION EVALUATIONS
• LAZAR 1980 - , BORLASE 1985 - , RISUCCI 1989 ,
  WADE 1993 - , SCHWARTZ 1995
• TEACHING SKILLS
• SEELY 1999 ?
• OPERATIVE VOLUME MIX
• LUCHETTE 1992
• ON-CALL STATUS
• STONE 2000 -

17
FOCUSED AI ABSITE RECIDIVISM

• POST-AI TEST PERCENTILE HISTORY
• 5 / 11 (46) AI POST-TESTS TT lt 31

18
FOCUSED AI LITERATURE

• GARVIN 1984, WADE 1993, WADE 1995
• 1984 BEGAN AI for ABSITE TT lt40 n ?
• COUNSELING, WARNING, PRACTICE EXAMS
• SOME IMPROVED DRAMATICALLY
• 23 ATTRITION, 5/11 ABSITE - BASED
• 1988 QE FIRST-TIME PASS 100

19
FOCUSED AI LITERATURE

• ABDU 1996
• PD SURVEY 231 RESPONDENTS
• FOCUSED AI 27 - 45 COUNSELING 45, PROBATION
  28, DISMISSAL 6
• NO ABSITE OR QE OUTCOMES
• HARTHUN 2005
• JR lt 40 / SENIOR lt 30 TT, n 8 1999-2004
• 1 FACULTY / 3 MEETINGS / PLP / TEST SKILLS
• 7/8 (88) SATISFACTORY, RECIDIVISM 2/8 (25)
• 3 CHIEFS FINISHED, NO QE DATA

20
FOCUSED AI QE RESULTS

• NO INTERVENTION (NI) GROUP

21
FOCUSED AI QE RESULTS

• FOCUSED AI GROUP

22
FOCUSED AI QE RESULTS

• HIGH RISK GROUP
• lt 30 TT 33 FAILURE RATE
• ABS 1993-2005 CUMULATIVE EXPECTANCY DATA

23
FOCUSED AI CONCLUSIONS

• FOCUSED AI IS FEASIBLE SUSTAINABLE
• FOCUSED AI IMPROVES IMMEDIATE POST - AI ABSITE
• ABSITE GAIN IS AT LEAST PARTIALLY INDEPENDENT
  EFFECT OF FOCUSED AI
• ABSITE GAINS ARE OFTEN TRANSIENT
• ABS QE FAILURE RISK IS PROBABLY REDUCED BY AI

24
FOCUSED AI FUTURE DIRECTIONS

• CONTINUING AI PROGRAM
• TARGETING RECIDIVISM
• MANDATORY POST - AI MENTORING REMAINDER OF
  RESIDENCY
• AT RISK PROGRAM
• TT PERCENTILE lt 40
• MENTOR
• VIDEOTAPES DISCRETIONARY TIME
• EARLY SESAP

25

• UNIVERSITY of MISSISSIPPI MEDICAL CENTER