Clinical Skills Verification Rater Training MODULE 4 Strategies for Clinical Skills Assessment: Mode

1
Clinical Skills Verification Rater
TrainingMODULE 4Strategies for Clinical Skills
AssessmentModels and Best Practices

• Michael Jibson, M.D., Ph.D.
• David Kaye , M.D.
• Sandra Sexson , M.D.

2
Module 4 Pre-Test

• The strength of a Clinical Skills Verification
  Examination stems from its ability to serve as a
  forum for all of the following except
• A. A high stakes examination
• B. direct observation of clinical skills
• C. provision of immediate feedback on
  performance
• D. repeated administrations during residency

3
Module 4 Pre-Test

• Which of the following models for CSV exam is the
  least standardized?
• A. Workshop Model
• B. Mock Board Model
• C. Clinical Evaluation Model

4
Module 4 Pre-Test

• 3. Which of the following models for the CSV exam
  makes the greatest use of resident peer feedback?
• A. Workshop Model
• B. Mock Board Model
• C. Clinical Evaluation Model

5
Module 4 Pre-Test

• 4. Within a program, consistency of faculty
  expectations and rating of residents on the CSV
  exam can be maximized by
• A. All raters using the same evaluation model
• B. Limiting assessment to the three competency
  areas outlined by the ABPN
• C. Using standardized patients
• D. Monitoring inter-rater reliability

6
Clinical Skill Verification (CSV)

• Strengths of Clinical Skills Verification
• Direct observation of clinical skill
• Multiple observations and observers
• Defined areas for evaluation
• Specific anchors for performance
• Immediate feedback
• Ample opportunity for remediation

7
Evaluation Model

• We recommend that the evaluation
• Be done early in residency and repeated
  frequently
• Include immediate feedback
• Avoid the high-stakes exam model

8
Evaluation Model

• Mock Board Model
• Use the CSV for the current practice of Mock
  Board exams
• Prearranged patients
• Scheduled interviews

9
Mock Board Model

• Advantages
• Familiar format
• Highly standardized

• Disadvantages
• Retains high-stakes exam format
• Narrow patient selection
• Difficult to schedule

10
Evaluation Model

• Workshop Model
• Use the CSV form for a workshop or class on
  interviewing
• Involve peer resident observers
• Videotape the encounter

11
Workshop Model

• Advantages
• Multiple residents involved
• Highly standardized
• Excellent feedback (if videotaped)

• Disadvantages
• Few opportunities for each resident to interview
• Narrow patient selection

12
Evaluation Model

• Clinical Model
• Use the CSV form for any initial clinical
  encounter with supervising faculty
• Inpatient admission
• Emergency room evaluation
• Outpatient assessment

13
Clinical Model

• Advantages
• Broad range of patients
• Many opportunities to interview
• Easy to schedule

• Disadvantages
• Moderately standardized
• Faculty buy-in essential

14
Evaluation Model

• We recommend that the evaluation
• Occur during regularly scheduled clinical
  rotations
• Involve the standard interview used in that
  clinical setting
• Involve randomly selected patients
• Be done as part of the routines of clinical care

15
Additional Issues

• Programs may want to consider
• Using the exam to test additional competencies
  for their own purposes
• Requiring residents to pass the examination
  before advancing through training

16
Additional Competencies

• CSV requirements may be combined with other
  clinical competencies
• Case formulation
• Diagnostic assessment
• Treatment planning

17
Faculty Training

• We recommend that programs
• Offer training to faculty in how to conduct and
  score the examination
• Make an effort to standardize scoring on the
  examination
• Monitor inter-rater reliability internally
• Consider credentialing faculty as evaluators

18
Post-Test Module 4

• The strength of a Clinical Skills Verification
  Examination stems from its ability to serve as a
  forum for all of the following except
• A. A high stakes examination
• B. direct observation of clinical skills
• C. provision of immediate feedback on
  performance
• D. repeated administrations during residency

19
Post-Test Module 4

• The strength of a Clinical Skills Verification
  Examination stems from its ability to serve as a
  forum for all of the following except
• A. A high stakes examination
• The in residency CSV exam allows programs to
  administer repeatedly and avoid the high-stakes
  aspect.

20
Post-Test Module 4

• Which of the following models for CSV exam is the
  least standardized?
• A. Workshop Model
• B. Mock Board Model
• C. Clinical Evaluation Model

21
Post-Test Module 4

• Which of the following models for CSV exam is the
  least standardized?
• C. Clinical Model
• The clinical model utilizes patients seen on
  regular clinical services without pre-selection
  of patients

22
Post-Test Module 4

• 3. Which of the following models for the CSV exam
  makes the greatest use of resident peer feedback?
• A. Workshop Model
• B. Mock Board Model
• C. Clinical Evaluation Model

23
Post-Test Module 4

• Which of the following models for the CSV exam
  makes the greatest use of resident peer feedback?
• A. Workshop Model

24
Post-Test Module 4

• 4. Within a program, consistency of faculty
  expectations and rating of residents on the CSV
  exam can be maximized by
• A. All raters using the same evaluation model
• B. Limiting assessment to the three competency
  areas outlined by the ABPN
• C. Using standardized patients
• D. Monitoring inter-rater reliability

25
Post-Test Module 4

• Within a program, consistency of faculty
  expectations and rating of residents on the CSV
  exam can be maximized by
• D. Monitoring inter-rater reliability
• Along with faculty training on how to conduct
  and score the examination, tracking inter-rater
  reliability among faculty after observing the
  same performance and giving feedback to faculty
  regarding their ratings compared to others is a
  very effective way of standardizing ratings.