The Residency Review Committee

1
The Residency Review Committee

• Jerry Vasilias, PhD, Executive Director
• Caroline Fischer, Associate Executive Director

APPD 10/07
2
Setting the Stage

• Overview to the RRC
• Review of RRC work in 2006
• Frequent Citations
• New ACGME Changes
• Preparing for the site visit
• ACGMEs Strategic Priorities
• Website

3
RRC Composition

• 3 appointing organizations - AAP, ABP, AMA
• 13 voting members
• 4 from each appointing org 1 resident member
• Members serve a term of 6 years -- except
  resident
• Generalists, subspecialists, Chairs and DIOs
• Critical Care Medicine, Hematology/Oncology,
  Pediatric Emergency Medicine, Gastroenterology,
  Neurology, Nephrology, Rheumatology
• Geographic distribution CA, CT, DC, GA, MI, MD,
  OH, OR, PA
• Ex-officio members from each appointing
  organization (non-voting)

4
RRC Review of Programs

• Peer Review 2 reviewers
• Reviewers use the following information when they
  review a program to determine whether it is in
  compliance with the requirements
• Program Information Forms (PIF) prepared by the
  program
• The site visitors report
• Board scores
• Program Directors this is an open book test
  
• Reviewers present program to Committee
• Independent reviews followed by reconciliation
• Committee assigns accreditation status along with
  review cycle, range of 1-5 years

5
Summary of Activities in 2006

• The RRC meets twice a year Spring and Fall
• Workload per meeting in 2006
• approximately
• 30 core programs
• 120 subspecialty programs
• 15 progress reports

6
Accreditation Decisions in 2006 -- Core
Pediatrics
7
Most Frequent Citations in 2006 Core Pediatrics

8
New ACGME Changes

• Requirements have been updated with new Common
  Requirements
• Appear in bold font in document
• Specialty specific content has not changed
• However, during update some language in
  competencies was inadvertently removed the
  document on the website/distributed today is
  correct.
• E-mail status of programs on RRC agenda
• 2-3 days after meeting will receive email w/
  status and review cycle.
• Proposed adverse actions will receive notice as
  well
• E-mail that letter will be posted on
  Accreditation Data System (ADS)
• Letter is posted one business day after the email
  notification
• Printed copies of letters will not be provided
• E-mail notification of site visit date

9
New ACGME Changes

• More information will be collected/communicated
  through ADS.
• Common PIF Common questions all specialties
  required to complete in ADS.
• Along with information already collected in ADS
  (institutions used for training and resident
  info) will collect
• Information on faculty/teaching staff
• Evaluation (resident, faculty and program)
• Duty hours
• Board scores
• ADS to be used for changes in complement, PD,
  institution /-, VW
• Responses to previous citations will be done in
  ADS. 1st part of PIF.
• Progress reports not yet in ADS.

10
New ACGME Changes

• Common competency questions inserted in all
  specialties PIFs (common but not hard-wired into
  ADS).
• Typical question asks PD to identify learning
  activity used to achieve a particular
  outcome/competency
• Describe one learning activity in which residents
  engage to identify strengths, deficiencies and
  limits in their knowledge and expertise
• Describe develop competence in communicating
  effectively with patients and families across a
  broad range of SES and cultural backgrounds
• Questions like these in PIF since late fall of 05
• PD Guide to the Common Requirements
• PD Virtual Handbook
• Webpage redesign

11
Update on Case Logs

• All core Pediatrics programs have been required
  to have their residents enter data in the ACGMEs
  case log system
• RRC has stated that it will not use for
  accreditation purposes until after its collected
  3 years of data
• Can use this system to track continuity not
  required
• Committee will review 3 years worth of data at
  its upcoming meeting

12
Preparing for the Site Visit The Five Stages
that Precede the Site Visit
First Stage Denial
13
Preparing for the Site Visit The Five Stages
that Precede the Site Visit

• 1st Stage Denial No, way! Theyre not coming
  already?
• 2nd stage Bargaining We can get a
  postponement!
• 3rd stage Anger --We really have to go ahead
  with this!
• 4th Stage Depression --I have sooooo much else
  to do? How will this get done?
• 5th Stage Acceptance -- Well be ready.
• With apologies to Elisabeth Kubler Ross

14
ACGME Expectations for Site Visits

• Conducted in accord with established policies
• Provide accurate, meaningful data for the RRC
• Note the PIF is the focus of the site visit (not
  the program)
• Verify/clarify info in the PIF
• Address all relevant aspects of the program
• Are educational and non-adversarial

15
Site Visit to Site Visit An Ideal Time Line

• 6 to 18 months before the visit
• Residents complete ACGME Resident Survey (all
  programs with 4 or more, surveyed every two
  years)
• ACGME expects programs to reach a 70 threshold
• Summary of results become available to PD and DIO
  only if reach 70 response rate
• Intent of the survey
• (1) focus and direct the resident interview, and
• (2) identify potential areas of non-compliance
  with duty hour standards for RC follow-up

16
Site Visit to Site Visit An Ideal Time Line

• On-going
• Review and know the program requirements
• ACGME web site, RRC specific updates and notices
• Contact RRC executive director or staff for
  clarification
• Attend meetings/workshops
• 1/2 way between last and next review
• Internal review occurs
• Involves a broad, representative group
• Candidly explores strengths and weaknesses
• Identifies improvements for implementation

17
Site Visit to Site Visit An Ideal Time Line

• The next survey date on your last notification
  letter is an approximate
• Site visit will take place between 2 4 months
  around the approximate date
• 3 - 4 months before the actual visit
• You will receive an email notification announcing
  actual site visit date
• You should begin your PIF
• If you havent already done so

18
Site Visit to Site Visit An Ideal Time Line

• 4 - 6 weeks before the visit
• Site Visitor will contact you to set up schedule
  for site visit date
• Here are a few of things the site visitor will
  probably ask for
• List of required documents (policy manual, GOs
  etc.)
• One large conference room
• Be prepared to setup meetings with PD, faculty,
  residents, DIO
• Large programs 10-12 residents should be peer
  selected
• Small programs all residents should be present
• Sometimes tour of facilities
• If possible, no beepers and available for the
  duration
• 14 days (10 business days) before the site visit
• Mail PIF to the site visitor

19
Site Visit to Site Visit An Ideal Time Line

• The Site Visit Occurs
• 1 - 6 weeks after the visit
• Site visit report is completed
• Report is sent to RRC Staff
• 2 - 8 months after the visit
• Program is assigned to a RRC reviewers
• The RRC meets, reviews your program and makes an
  accreditation decision and assigns review cycle
  date

20
Preparing for the Visit The site visit date

• The Question What if the site assigned site
  visit date is a problem?
• The Answer There is never a good time for a
  site visit.
• However, if its a major problem
• Ingrid Philibert [phone] or
  iphilibert_at_acgme.org
• Jim Cichon [phone] or jcichon_at_acgme.org
• Penny Lawrence [phone] or pil_at_acgme.org
• Do not call the site visitor, the RRC staff, or
  the RRC Chair.
• Call soon after you receive your email/letter
  announcing the visit
• Requesting a postponement does not guarantee one

21
A Few Pointers PIF Preparation

• The PIF is a self study document that describes
  how your program complies with the requirements
• Start EARLY!
• Read questions carefully and start preparing
  initial responses
• Review prior citations and make sure they have
  been addressed
• Cross check information to avoid internal
  inconsistencies
• Answer every question!
• Think systems-based practice and improvement
  involve faculty and residents and other program
  personnel in the completion and editing of the
  document

22
The Day of the Visit ACGME Interviews

• ACGME interviews program director, faculty,
  residents and others.
• Site visitors triangulate response to verify
  and clarify the PIF.
• Compares responses in the PIF to interview data
  and actual observations (e.g., call rooms,
  libraries, exam rooms etc.)
• Compares participants responses to each other
• Discrepancies presented to the program director
  for clarification and resolution

23
The Day of the Visit Resident Interview

• The resident interview explores the educational
  program from the residents perspective (as
  consumers)
• Residents are peer selected (if small program,
  all residents)
• Site visitor seeks consensus from residents
• Sample Questions
• Have you seen/received a copy of programs
  educational goals and objectives?
• How and how often are you evaluated?
• Do you evaluate your faculty and your rotations?
• How are you supervised?
• If you had due process concerns, where would
  you find the procedures? Whom would you contact?
• What are the programs strengths? Areas for
  improvement?
• What are the reasons you chose this program?

24
The Day of the Visit PIF Corrections

• The PIF submitted to site visitor is
• considered final. However,..
• If there are revisions on the day of the visit
• Fix and provide SV with changed pages
• Highlight changes made
• Replace pages in copies to go to ACGME
• If there are revisions after the site visit
  only if absolutely necessary
• Always done as addendum to the PIF

25
After the Site Visit The site visitors report

• What goes into the report?
• Review of the program history
• Review of previous actions and actions taken to
  correct
• Review of institutional issues that pertain to
  the program
• Clarification and verification of the PIF
• Specialty specific content
• What does not go into the report?
• NO opinions or non-factual based comments
• Just the factsnothing but the facts

26
After the visit If you have questions

• About the PIF/Case logs -- technical questions
• Visit the ACGME web site at www.acgme.org
• Contact ACGME Help Desk helpdesk_at_acgme.org or
  312/755-7464
• For Part I of PIF off of WebADS reng_at_acgme.org
• For Case logs oplog_at_acgme.org
• About PIF content/Program Requirements
• For PIF questions dbraun_at_acgme.org or (312)
  755-5492
• For program requirements questions
  jvasilias_at_acgme.org or [phone]
• cfischer_at_acgme.org or [phone]
• Look on Peds webpage and FAQ
• About the site visit/After the site visit
• Call the site visitor or the Department of Field
  Activities
• Look at the site visit FAQ
• Complete site visitor evaluation

27
ACGMEs Strategic Plan

• In November 2005, ACGME endorsed 4 strategic
  priorities.
• 1) Foster innovation and improvement in the
  learning environment
• 2) increase the accreditation emphasis on
  educational outcomes
• 3) Increase efficiency and reduce burden from
  accreditation
• 4) Improve communication and collaboration with
  key internal and external stakeholders
• Many of the changes ACGME has implemented in the
  recent past and will continue to roll out in the
  future can be seen from the vantage point of the
  4 strategic priorities.

28
Innovations and Improvements to the Learning
Environment

• Committee on Innovation and the Learning
  Environment (CILE)
• Recommendation 4 Develop a set of
  ACGME-supported pilots to assist the RC in
  advancing innovation in the learning environment
  and meeting ACGMEs strategic priorities.
• CILE Pilots to help in refining/revising select
  common requirement standards, specifically duty
  hours and competencies.
• RRC will decide whether to participate
• ACGMEs Learning Innovation and Improvement
  Project studying the characteristics of
  institutions that foster innovation and sharing
  w/ others
• Conversations between RRC and R3P

29
Increase Emphasis on Outcomes

• Common Requirements are a move towards more
  outcome-focused requirements and less process.
• Competency based G Os
• Fuller/fleshed out language within the
  competencies
• Evaluation of program written to focus on how
  evaluation can improve program
• RRC Companion Document
• RRC front-runner in including competencies as
  organizing structure for requirements and
  competency questions in PIF
• A different type of citation
  constructive/improvement based citations related
  to the competencies

30
Increase Efficiency and Reduce Burden

• New Common Requirements less overall
  requirements outcomes focused
• E-mail accreditation status of programs on agenda
  w/in 2-3 days
• E-mail to inform you that notification letter is
  posted on ADS.
• Standard letters of notification
• Increased information being collected through ADS
• Revised glossary of terms
• PD Guide to Common Program Requirements
• PD Virtual handbook
• RRC removed request for documentation of
  continuity numbers site visitor will verify

31
Improve Communication and Collaboration

• Redesigned webpage
• Email updates when necessary
• Post presentations on website
• Newsletter will be implemented soon
• Conversations related to innovation
• Conversations related to the case log system
• jvasilias_at_acgme.org
• cfischer_at_acgme.org
• dbraun_at_acgme.org

32
www.acgme.org

• Staff contact information
• ACGME Policies Procedures
• Competencies/Outcomes Project
• List of accredited programs
• ADS
• Duty hours Information/FAQ
• Affiliation Agreements FAQ
• General information on site visit process and
  your site visitor
• Pediatrics Webpage
• Resident complement increase
• Program Requirements and PIFs
• Case Log System
• Archive of RRC Updates/Email Communications
• Pediatrics FAQ

33

34
Completing a PIF is Preparing for a site visit
is

• Simple Complicated
  Complex