R3P Colloquium

1
R3P Colloquium American Board of Pediatrics Jan.
31 Feb. 2, 2007 The Past, Present and Future
Assessments of Clinical Competence A Canadian
Perspective
Bob Hilliard MD EdD FRCPC Pediatric Specialty
Committee
2
Many, many thanks for the invitation!! Overview
No Debate Whatsoever Assessments and
evaluations in a pediatric residency program must
be an integral part of the curriculum (teaching
and learning experiences) and based on clear,
obtainable and measurable educational
objectives!! Examinations and evaluations
influence, direct and drive resident learning
what, how, why. This includes not just the
content of what is on the examination but also
the format.
3

• Pediatric Training In Canada
• Royal College of Physicians and Surgeons of
  Canada
• Surveys accredits programs, credentials
  certifies specialists
• Pediatric residencies are all university based
  (16)
• Goals Objectives the pediatric consultant
• Three (3) Core years
• ? one (1) additional year of general pediatrics,
  or
• ? the first year (1) of pediatric subspecialty
  training
• Prior to 2001 the final exam was
• ? Multiple Choice Questions (MCQ) exam
• ? Two-part oral exam
• ? long case observed discussion / short
  case
• ? four cards selected from a box of questions

4

• Objectives of Training
• Specialty Training Requirements in Pediatrics
• CanMEDS Roles ABMS Competencies
• (Essential Key Competencies)
• Medical Expert ? Medical Knowledge
• ? Systems Based Knowledge !
• ? Skills ? Patient Care
• ? Attitudes
• ? Problems
• Communicator ? Interpersonal / communication
• Collaborator skills
• Manager ? Systems-based practice
• Health Advocate
• Scholar ? Practice-based learning
• Professional ? Professionalism

5
(No Transcript)
6

• In Training Evaluations
• Rotation Specific In-Training Evaluation
  Reports (Global Assessment of
  Performance) CanMEDS format
• Formative - face-to-face feedback
• Written Multiple Choice Questions (MCQ) Exam
• Oral / Clinical Exam
• Observed History and Physical Exam / Mock Oral
• Objective Structured Clinical Exam
• 360º Evaluations
• Evaluation of Research / Quality Assurance
  Projects
• Learning Portfolios

7

• Final In-Training Evaluation Report
• Certificate of Confirmation of Completion of
  Training (CCT)
• FITER - Summative - compilation CanMEDS roles
• STACER (Standardized Assessment of a Clinical
  Encounter Report)
• Observed History and Physical Examination
• two (2) observers / examiners, structured
  check list
• may be repeated until satisfactory performance

8

• The Final Examination in Pediatrics
• A condensed, comprehensive three-part exam
• blue print systems based content CanMEDS
  roles
• Multiple Choice Questions (MCQ) Exam 150 / 3
  hrs
• Short Answer Questions (SAQ) 60 75 / 3 hrs
• Structured Oral Questions (OSCE) 10 X 15 mins
• - attempt to evaluate all CanMEDS roles
• - standardized patients history taking,
  counseling
• - parents and adolescents ( 12 years)
• - telephone advice stations
• - structured oral stations
• - physical examination stations (standardized
  patients)
• - pictures / videos
• - critical appraisal stations
• - reliability 0.80

9

• The Final Examination in Pediatrics
• (Reflections after six years experience)
• Reliable all exams - reliability ? 0.80
• High face validity (credibility)
• High content validity
• Canadian trainees do very well
• Non-Canadian trainees much lower pass rate
• ? language, lack of basic skills, ? exam
  experience, ? anxiety
• Logistically time consuming, labour intensive
• 1 2 1 days in Ottawa
• 2 days paired stations, both English and
  French
• 3 sessions candidates sequestered to avoid
  contacts
• 35 examiners, 2006 170 candidates

10
The question still is What should be the
standards of a practicing pediatrician in Canada
and how should these competencies be
evaluated? One Grade Only and That the Best!!
11
Thank You Very Much for your Attention!! Any
Comments or Questions??
12

• Maintenance of Competence
• a. Canadian Perspective 80 hours of continuing
  education accredited CE based on adult learning
  principles individual personal learning
  projects accredited self-assessment
  programs practice audits, practice reviews and
  appraisal
• b. USA Perspective
• evidence of professional standing
• evidence of life-long learning
• evidence of cognitive expertise
• evidence of satisfactory performance in Pediatrics

13

• Physicians who have not
• Trained in Canada
• Only training in a limited number of specific
  jurisdictions is accepted by the RCPSC.
• Many immigrant physicians have not had the same
  evaluations as Canadian residents.
• It is difficult for immigrant physicians to get
  further specialty training.
• Some provincial licensing authorities (CPS) grant
  temporary or restricted licenses to practice in
  supervised setting provided that the physician
  pass the RCPSC exams and complete a five year
  cycle of Maintenance of Certification.
• Pediatricians who have trained outside of Canada
  have difficulties passing the RCPSC certification
  exams.