What circumstances promote accurate selfassessment

1
What circumstances promote accurate
self-assessment?

• SACME Fall Meeting, Oct 2008
• San Antonio, Texas
• Joan Sargeant PhD
• Director Research and Evaluation, CME
• Director Program in Health and Medical Education
  Research Associate Professor, DME

Dalhousie University Halifax Nova Scotia, Canada
2
Basic dilemma

• On one hand, self-assessment and self- monitoring
  are integral to being a professional
• But on the other, research shows we dont do this
  very well (Eva, Regehr 2005, 2008 Davis et al
  2006, )

3
Self-assessing on ones own is like
4
We dont know what we dont know

• To make matters worse, poorly performing
  individuals seem less able to recognize
  competence when they see it (Kruger, Dunning
  1999 Violato, Lockyer 2006)
• Because theyre unable to recognize competence in
  others, theyre less able to use information
  about the performances of others to form more
  accurate impressions of their own ability.
  (Kruger, Dunning 1999)

5
So what can we as CE providers possibly do to
make a difference?
6
Objectives for this session

• 1. From theory, understand models or ways of
  thinking about informing self-assessment
• 2. From the research evidence, identify
  circumstances approaches which can promote/
  inform more accurate self-assessment
• 2. For our own CE/ CPD practice, discuss
  practical applications we can use to help inform
  physician self-assessment

7
One helpful literature source

• Journal Continuing Education in the Health
  Professions. 2008. 28(1)
• Focus on self-assessment 8 articles
• 3 theory
• 2 research
• 3 practical

8
1. From theory, understand models or ways of
thinking about informing self-assessment

• Self-assessment is a process of interpreting data
  about our own performance and comparing it to an
  explicit or implicit standard (Epstein, Seigal,
  Silberman 2008 Boud 1995)

9
From theory .

• The power of self-assessment lies in two major
  domains (Epstein, Seigal, Silberman 2008 Eva,
  Regehr 2008)
• the integration of high-quality external and
  internal data to assess current performance and
  promote future learning, and
• the capacity for ongoing self-monitoring during
  everyday clinical practice

10
Theory

• Self-monitoring and self-assessment involve
  interpreting both external, observable actions
  and subjectively-experienced internal states.
  (Epstein et al 2008)

11
From theory, a beginning picture of
self-assessment
Processes Interpreting data (Reflecting, Cogniti
ve processing)
External cues/ data

Internal data, states
Self-assessment
12
Research evidence External data, interventions
to promote accurate SA

• In Kruger, Dunnings 4th study,
• when individuals with low scores were left on
  their own to improve, they didnt improve as
  couldnt see what was wrong
• But, when facilitators provided information about
  what they were doing wrong and what to do to
  improve their scores, they improved (Kruger,
  Dunning 1999)

13
Research evidence External circumstances to
promote accurate self-assessment

• Clear performance standards (E.g., CPGs for
  managing diabetes)
• Clear criteria for measuring them (E.g., knowing
  measures to determine whether meeting CPGs)
• AND
• Ability to use the criteria to make judgments
  about ones own performance (Boud 1995)

14
Sources of external data Formal, informal
Teachers, tests
Medical colleagues
Practice data
Patients
Co-workers
Practice Assess- ment
15
Research evidence Approaches to promote use of
external data to inform more accurate
self-assessment

• Observation and feedback (Rees 2005)
• Portfolios (Dreissen et al 2007)
• Clinical practice guidelines and chart/ practice
  audit (Duffy 2008, Galbraith 2008)
• Patient questionnaires (Duffy 2008)
• Recording answering questions arising in
  practice e.g. slowing down (Silver 2008)
• Multi-source feedback (Sargeant 2008, Lockyer
  2006)

16
From theory, a beginning picture of
self-assessment
Processes Interpreting data (Reflecting, Cogniti
ve processing)
External cues/ data

Internal data, states
Self-assessment
17
Research evidence Internal data/ states to
inform more accurate self-assessment

• Habits of mind, mindfulness
• Openness
• Curiosity
• Emotions
• Attentiveness
• Presence
• Ability to see multiple perspectives
• (Epstein 1999, 2002, 2008)

18
Research evidence Internal data to inform more
accurate self-assessment

• Physicians compare external feedback with
  self-perceptions of performance. May discount it
  depending upon perceptions of (Sargeant 2006)
• Degree of difference from own perceptions
• Feedback credibility
• Feedback specificity
• Self-efficacy (beliefs in ability to use
  feedback and to change)
• Beliefs about malleability of performance domains
  (e.g., communication skills learned skill or
  personality trait?)

19
From theory, a beginning picture of
self-assessment
Processes Interpreting data (Reflecting, Cogniti
ve processing)
External cues/ data

Internal data, states
Self-assessment
20
Research evidence Interpretative processes to
process external and internal data

• Schon (1985) reflective cycle reflection in
  action, on action
• Learning from experience, reflecting on
  experience (Boud 1999)
• Critical reflection and its components (Mamede,
  Schmidt 2002)
• Deliberate practice (Ericsson,1999)
• Neurocognition (Epstein et al, 2008)

21
Research evidence Approaches to enhance
interpretation of external and internal data to
inform self-assessment

• Silver, Cambell, et al (2008) - CFPC, RCPSC
  formal process for reflection on practice for CME
  credit
• Dornan (2008) electronic learning management
  system / competencies
• Dreissen et al (2007) portfolios
• Galbraith et al (2008), Duffy et al clinical
  practice audit report compared to practice
  guidelines
• Sargeant et al (2008) facilitated reflection
  used to reconcile external data with internal
  perceptions

22
Research evidence Approaches to enhance
interpretation of external and internal data to
inform self-assessment

• Facilitation or guidance re use of external
  resources (Dornan 2008, Duffy et al 2008)
• In using specific approaches and tools
• In accessing appropriate standards
• Facilitation of reflection upon (Goodstone,
  Diclemente 1998, Sargeant et al, 2008)
• External data
• Its assimilation with self-assessments
• Integration of disconfirming data
• Plans for feedback use and improvements

23
From theory, a beginning picture of
self-assessment
Processes Interpreting data (Reflecting, Cogniti
ve processing)
External cues/ data

Internal data, states
Self-assessment
Facilitation/ coaching
24
Research evidence Two additional emerging pieces
about self-assessment

• Culture
• Professional, local (workplace, learning
  environment)
• Perceived cultural expectations
• Check out RIME, GEA sessions at AAMC
• Fluid, dynamic nature of all dimensions
• (Stewart J, Med Ed, Sept, 2008 To call or not
  to call)

25
From theory and evidence, a developing picture of
self-assessment
Culture
Processes Interpreting data (Reflecting, Cogniti
ve processing)
External cues/ data

Internal data, states
Self-assessment
Facilitation/ coaching
26
Questions for discussion

• What practical approaches are available to you to
  aid the informing of physician self-assessment?
• What strategies might you try to enhance each
  dimension and process to inform self-assessment?
• Thank you