Best Practices Precepting Models Student Self-Assessment

1
Best Practices Precepting ModelsStudent
Self-Assessment

• Michelle E. Allen, PharmD, BCPS
• Clinical Specialist, Adult Critical Care
• Shands at the University of Florida
• Clinical Associate Professor
• University of Florida, College of Pharmacy

UF Fall Faculty Workshop Sept. 12, 2008
2
Disclosures

• None
• Opinion and practice based

3
The What and How

• Rotation presentation(s)
• UF rotation competencies
• 1-month rotation 3 self-evaluations
• 2-month rotation 4 5 evaluations

4
Benefits

• Saves me some time
• Documentation, just in case
• Question of grade determination
• Letters of recommendation
• Student value
• Learning how to be a life-long learner
• Education vs. evaluation
• Maintaining professional development and
  competence
• Grade satisfaction

5
Potential Problems

• Expectations
• Competency is a good thing, a great thing, that
  is the goal
• Inaccurate self-assessment
• Overestimation
• Underestimation
• Time consuming particularly in the over-estimator

6
Conflicting EvidenceAbilityAccuracy
7
Inability to Self-Assess

• Inability to peer-reference
• Knowledge of standards
• Knowledge of expectations
• Process nonverbal feedback
• Rewards or punishments to change one's actions

Maki RH, Johas D, Kallod M. T. Psychonomic Bull
Rev. 199411269. Metcalf J. Pers Soc Psychol
Rev. 1998210010.
8
Self-Assessment Accuracy

• Systematic review 17 / 725 studies met inclusion
  criteria
• 13 little, no or inverse relationship
• 7 positive association

Davis DA, Mazmanian PE, Fordis M, et.al. JAMA
20062961094-1102.
9
Self-Assessment Accuracy

• Procedure performance
• Dementia assessment
• Health promotion counseling
• Critical care skills
• Palliative care
• Explain evidence-based medicine
• Predict students assessments of teaching
  abilities
• Knowledge in primary care

• Dementia assessment
• Basic clinical skills
• Managing psychological aspects of family practice
• Sexual abuse by physical exam
• Diagnostic features of small pox
• Language and cultural competence
• Sensitivity to emotional and psychological stress

Davis DA, Mazmanian PE, Fordis M, et.al. JAMA
20062961094-1102.
10
Self-Assessment Accuracy
Those that performed the poorest by
external review are the worst at self-assessment
Davis DA, Mazmanian PE, Fordis M, et.al. JAMA
20062961094-1102.
11
Self-Assessment Accuracy

• Evidence-based guidelines (required ACGME
  competency)
• Self-perceived competence (1st part of the
  survey) vs. actual performance (2nd)
• Average actual score 55
• Chasm between perceived and actual performance

Accreditation Council for Graduate Medical
Educations Practice-Based Learning
Caspi O, McKnight P, Kruse L, et al. Medical
Teacher 200624318-25.
12
Self-Assessment Accuracy

• Student self-assessment vs. resident/faculty
• 3rd year medical students on Ob/gyn rotation
• 5-point Likert scale
• Suggest that students do not have an
  understanding of their own strengths and
  weaknesses ? may not know personal or
  professional limitations

Knowledge Attitude
Problem-solving Technical skills
Written/verbal skills (agreement) Written/verbal skills (agreement)
Weiss PM, Koller CA, Hess LW, et.al. Medical
Teacher 200527445-9.
13
Feedback

• Shown a video, given a pre-test and randomized
  into groups
• Computer generated economy feedback
• Above expert reference values
• Above expert verbal feedback
• 18 practice trials, post-test

Porte MC, Xeroulis G, Reznick RK, et.al. Am J
Surgery 2007193105-10.
14
Feedback

• 1-month later 5 sutures evaluated
• Only the expert feedback group was able to
  predictably retain the skills
• Evaluation vs. education
• For education technology cannot replace the
  expert

Porte MC, Xeroulis G, Reznick RK, et.al. Am J
Surgery 2007193105-10.
15

• William is a short shy man. He has a passion for
  reading and loves strolling through museums. As
  a child, he was bullied by his classmates.
• Do you suppose William is a
• a) truck driver
• b) classics professor

Austin Z, Gregory PAM, Chiu S. Am J Pharm Ed
2008721-8.
16
Reflection and Self-Assessment

• 24-item critical thinking test
• Rule-based (logical) vs. short-cut (flawed)
• Completed test without interferenceor
• Prompted at question 13 and 18 to reflect and
  self-assess
• Rate confidence in the answer
• Provide an explanation of how they arrived at the
  confidence rating
• Provide explanation, rationalization or
  justification of why they selected the answer

Austin Z, Gregory PAM, Chiu S. Am J Pharm Ed
2008721-8.
17
Take Home Messages

• Feedback direct and immediate
• Feedback opportunity to develop skills under
  supervision and mentorship
• Reflection does not have to be complicated or
  cumbersome
• Feedback expert feedback cannot be replaced by
  technology

18
Knowing is not enoughwe must apply.Willing is
not enoughwe must do.

• Johann Wolfgang von Goethe
• 1749-1832

19
Best Practices Precepting ModelsStudent
Self-Assessment

• Michelle E. Allen, PharmD, BCPS
• Clinical Specialist, Adult Critical Care
• Shands at the University of Florida
• Clinical Associate Professor
• University of Florida, College of Pharmacy

UF Fall Faculty Workshop Sept. 12, 2008