Bridge over Troubled Water Linking UME and GME

1
Bridge over Troubled WaterLinking UME and GME

• Monica L. Lypson, MD Assistant Dean, Graduate
  Medical EducationUniversity of Michigan
• Jeff Fabri, MD
• Rita M. Patel, MD

2
Education Across the Continuum Standardization
of Assessment Student Portfolios to Baseline
Assessment

• Monica L. Lypson, MD Assistant Dean, Graduate
  Medical EducationUniversity of Michigan
• Associate Chief of Staff, VA Ann Arbor Health
  Care System

3
Institutional OSCE Post-Graduate Orientation
Assessment (POA)

• Developed by the Graduate Medical Education
  Committee (GMEC) at the University of Michigan
• It is our initial step in training our residents
• It was established to determine residents
  baseline proficiency in particular aspects of the
  ACGMEs six general competencies

Lypson ML. et.al. Academic Medicine.
79(6)564-70, 2004 Jun.
4
POA CONTENT

• Knowledge and skills needed during the first six
  to eighteen weeks of residency/internship
• Emphasizes clinical situations that are often
  encountered without formal supervision
• 9 Assessment Educational Stations

5
The POA as Formative Assessment

• Results of the POA determine the basis for
  individualized learning agendas
• Remediation is provided after the completion of
  each station
• Residents received educational materials that
  provide the answers to the information assessed
  during the POA
• Standardized Patient feedback is provided to the
  program director within 24 hours if the resident
  performs exceptionally well or poorly during the
  POA

6
Implementation

• Administered over 4 days of Paid hospital
  orientation
• Provided to approximately 150 PGY-1 residents in
  over 15 specialties
• Cost Approximately 250 per resident for the
  assessment - 500 if you add in salary
• There are some discipline specific scenarios
  e.g. Pediatric cases and examples
• Scores are provided to the resident and program
  director within 7-10 days of the POA

7
Subspecialty Programs

• Dentistry
• Emergency Medicine
• Family Medicine
• General Surgery
• Preliminary Residents
• Internal Medicine
• Preliminary Residents
• Internal Medicine Pediatrics
• Neurosurgery
• Obstetrics Gynecology

• Otolaryngology
• Orthopedics
• Pathology
• Pediatrics
• Plastic Surgery
• Psychiatry
• Urology
• PGY-2s
• (Residents that did not complete their
  Internships at UMHS)
• Physical Medicine Rehabilitation
• Dermatology
• Neurology

8
Station Content

• Socio-Cultural Communication
• Standardized patient assessment
• Assesses the understanding of disease and
  treatment recommendations in the context of the
  patients health beliefs and socio-economic
  setting
• Competencies
• Interpersonal and Communication Skills
• Professionalism
• Patient Care
• Practice-Based Learning and Improvement

• Critical Values / Multiple Choice
  
• Computer Based assessment
• Review and diagnose 17 short patient scenarios
• Multiple Choice Format similar to United States
  medical Licensing Exam (USMLE) Step 3
• Competencies
• Patient Care
• Medical Knowledge

9
Station Content

• Evidence-Based Medicine (EBM)
• Computer-based Assessment
• Generate a clinical question
• Residents reviewed abstracts identify the
  appropriate treatment
• Competencies
• Practice-based Learning and Improvement
• Medical Knowledge

• Images (X-Rays)
• Computer-based Assessment
• Review diagnose 18 images
• Many of the common films reviewed in the middle
  of the night while on call
• Competencies
• Patient Care
• Medical Knowledge

10
Station Content

• Informed Consent
• Standardized patient Assessment
• Obtain informed consent from a patient for a
  procedure
• Hospital JCAHO standards and policies
• Competencies
• Interpersonal and Communication Skills
• Professionalism
• Patient Care

• Patient Safety
• Pen Paper Assessment
• Order Writing Station
• Legibility
• Signature, Date, Time
• Respiratory Distress
• Treatment of the acute Asthmatic
• Competencies
• Systems-Based Practice
• Professionalism
• Patient Care
• Medical Knowledge

11
Station Content

• Aseptic Technique
• Checklist Evaluation by Expert Nursing Staff and
  Standardized Patients
• Create maintain a sterile field while
  performing a mock ID
• Universal Protocol Time-out procedures
  covered
• Assessment/Remediation materials include a review
  of JCAHO requirements
• Improved nursing and house officer interactions
• Competencies
• Patient Care
• Medical Knowledge
• Systems-Based Practice

12
Station Content

• System Compliance / Fire Safety Station
• Surgical Fire Safety (20/20)
• Housestaff involvement with a Patient fire
• Questions covering use of safety equipment
  review of JCAHO requirements
• Video Computer Based Assessment
• Competencies
• Patient Care
• Systems-Based Practice

• Pain Assessment
• Pain Assessment Tools
• Educates on the appropriate medications to use
  for pain
• Explains hospital and JCAHO Pain assessment
  mandates
• PowerPoint Computer Based Assessment
• Competencies
• Patient Care
• Medical Knowledge

13
Resident Satisfaction with the POA-Survey
Response Rate 932002-2004
Question YES
Have you learned any new clinical skills during this assessment? 70.6
Do you think this was a useful way to spend part of orientation? 83.4
Do you feel better prepared for some aspects of internship after this assessment? 80.6
Do you expect to refer back to the teaching materials handed to you today? 84.7
Would you recommend that we continue the POA next year? 84.7
14
The POA was based on the following

• The University of Michigan
• Comprehensive Clinical Assessment
• (CCA)

15
Comprehensive Clinical Assessment 1991-Present

• A high-stakes examination for University of
  Michigan Medical Students
• Fourth Year students must pass the CCA in order
  to Graduate
• 10-15 stations over 4-5 hours
• 150 M4 students per year
• Summative Evaluation
• Standards have been set for Pass/Fail
• Remediation for failures

Rochester AB. et.al.. Academic Radiology.
5(3)169-72, 1998
16
Summary of Station ComparisonCCA POA

• POA- GME
• Formative
• Informed Consent Policy
• Pain Assessment
• Socio-Cultural Communication
• Evidence Based Medicine
• Images

• CCA-UGME
• Summative
• History Physical Diagnosis
• Geriatrics
• Socio-Cultural Communication
• Evidence Based Medicine
• Images

17
Institutional Implications for GME UGME

• Training the Faculty in the Teaching Skills of
  competency assessment
• Medical School Objectives Project (MSOP)
• Accreditation Council of Graduate Medical
  Education
• Reimbursement of faculty time and effort
• Payment for both assessments Medical School vs.
  Hospital
• Is the medical school or the hospital responsible
  for educational cost?
• Trainee awareness of competency gap between the
  expectations of medical School and residency

18
(No Transcript)
19
POA Demographics 2002-2004
Gender Male 54
Race White / Non-Hispanic Underrepresented Minorities Other 70.1 6.5 23.4
Medical Schools Public International 62 1.5
United States Medical Licensing Examination (USMLE) Step 1 Step 2 231 233
20
The University of Michigan Health System Plan for
Competency Based Resident Education

• Joint Hire faculty member with the Department of
  Medical Education
• Centralized OSCE Post-Graduate Orientation
  Assessment (POA)
• This is based on the system already in place for
  Undergraduate Medical Education at UMHS.

21
The Department of Medical EducationChairman
Larry Gruppen, PhD

• One of 5 departments of medical education at
  LCME accredited medical schools
• The department had focused on CME and UGME in the
  past with assessment and research expertise
• The office of Graduate Medical Education and the
  Department of Medical Education
• Joint Hire
• Job Description
• 50 appointment GME office, 50 appointment Dept.
  of Med. Ed.
• Aid Residency programs with curriculum
  development, core competency assessment
• Use this work as a vehicle of scholarly pursuit
• The department has a long history of UGME now
  will develop its expertise in GME