Current Concepts in Clinical Assessment

1
Current Concepts in Clinical Assessment

• Texas Chiropractic College
• August 26-27, 2006
• Julia Bartlett Mary Berg
  Warren Moe

2
Presenters

• Alan Adams, DC TCC
• Delia Anderson, MS Tulane University
• Steven Downing, PhD University of Illinois
• Steve Foster, DC TCC
• John Littlefield, PhD Univ. of Texas, San Antonio
• Karen Szauter, MD Univ. of Texas, Galveston
• Balazs Moldovanyi CEO

3
OSCEs

• Objective Structured Clinical Examinations
• A good OSCE has 7-11 cases
• Each case can be different length- 15-20 minutes
  each is common.

4
OSCEsThin Slices of Expressive Behavior

• Definition Brief excerpt of behavior sampled
  from the behavioral stream, less than 5 minutes
  in length.
• Accurate and reliable conclusions can be formed
  rather quickly.
• Ambady N et. Al. Thin-Slice Judgments as a
  Measure of Interpersonal Sensitivity. In J hall
  F Bernieri (Ed) Interpersonal Sensitivity Theory
  Measurement, Erlbaum Assoc. Pub., 2001.

5
Where do OSCEs fit in assessment?

• Millers pyramid
• KNOWS written exams, facts principles and
  theories
• KNOWS HOW - Simulation tests, ability, competence
• SHOWS HOW OSCEs/Standardized patients,
  performance
• DOES clinical records, observation, action in
  practices
• Miller FE, The Assessment of Clinical Skills/
  Competence/Performance
• ACED Med 1990 563-7

6
Developing OSCE cases

• Before developing the OSCE, we must decide if we
  are going through a ritual or are we willing to
  take action?
• Faculty should be submitting the cases and
  working with the SP trainer.
• What if a case has been compromised?

7
Developing Cases- Before you Start Karen Szauter,
MD

• What is the goal of the assessment?
• Formative
• obtaining information to see if specific learning
  objectives are being met narrow focus
• feedback given
• Summative
• end of the term or course exam
• Standardized exams
• Carries a lot of weight in todays culture

8
Developing Cases- Before you Start Karen Szauter,
MD

• Which course(s) are you supplementing or
  assessing?
• Create a blueprint of the exercise

9
Developing Cases- Before you Start Karen Szauter,
MD

• What do you hope the case(s) will demonstrate?
• Do the assessment objectives match the course
  objectives? This should be reviewed yearly.
• Have a clear statement of purpose. Dont have too
  many purposes for one assessment.
• Sometimes, a vignette is as helpful as an SP

10
Developing Cases- Before you Start Karen Szauter,
MD

• Who do you have available for patients?
• How much time do you have?
• Some cases take longer to prepare
• Training activities will vary depending on
  portrayal/checklist/feedback

11
Case Development Karen Szauter, MD

• Think about the person first
• Keep a visual image of the person while you write
  the case
• Develop the primary problem
• Consider comorbid conditions
• Give the SP adequate information to make the
  portrayal credible.
• Specify any simulated physical findings
• Assess the SP for contradictory findings

12
Checklist Development Karen Szauter, MD

• Refer back to the case objectives.
• Think about the key components of the case.
• Ascertain that the case has all of the relevant
  information for the checklist responses.

13
Work in Groups Karen Szauter, MD

• Avoid single case authors if possible.
• Let experienced SPs read the case for details.
• Have one of the case authors run the case before
  using it.

14
Setting the Standards

• Why do we trust multiple choice scores, but not
  practical performance scores?
• Various methods to setting the standard.
• Future discussion?

15
Advantages of SPs to LearnersDelia Anderson, MS

• Authenticity
• Consistency and accuracy
• Focus on learners performance
• Educated unbiased feedback from the patient
  perspective
• Sometimes it is hard for the faculty to write
  candid evaluations
• Encouragement of active learning
• Teach sensitive topics and examinations

16
Advantages to Program Curriculum Delia Anderson,
MS

• Reinforces and documents curricular goals
• Provides feedback about teaching effectiveness
• Requires and emphasizes defined objectives
• Requires defined performance criteria
• Forces critical look at curriculum/goals
• Allows programmatic assessment of overall
  curriculum

17
Advantages to Faculty/Administration Delia
Anderson, MS

• Control content and complexity
• Relieves faculty of time-consuming instruction
  and assessment of basic skills
• Provides a reproducible, reliable experience for
  all learners, allowing training and assessment of
  care skills
• Provides quantitative feedback about learner
  performance and applied skills not available
  through other methods

18
Standardized Patients Delia Anderson, MS

• Not students
• Share with other institutions
• Teaching students skills
• History start 2nd week in some med schools
• Physical exam
• Can change the enviroment to teach observation
  right off the bat
• Assessment
• SP provides an assessment and/or debriefing to
  learner
• Unannounced SPs in the clinic system

19
Other uses for SPs

• Unannounced SPs in the clinics to assess true
  practice patters
• Telephone consultations
• Blend with simulations apparatuses
• ie. In med school, cysts removal from a pad
  strapped to a patient's arm
• Report clinical behavioral issues
• At Tulane, these reports go to Student Affairs
  starting in Year One.

20
Debriefing

• Some OSCEs have debriefing sessions immediately
  following the assessment.
• Examiner and/or SP
• Individual or group feedback

21
ASPE

• Association of Standardized Patient Educators
• www.aspeducators.org

22
Student Evaluations

• It is important to obtain student evaluation of
  the assessment program.
• The evaluation results should be compiled in a
  report to support (or not) the program.

23
What are we already doing?(Chiropractic program)

• Models for gynecological and prostate exam
• These are paid models.
• History course
• Developmental Assessment
• Student simulated patients
• New recording equipment in UHS

24
What could we add?

• Multiple OSCEs at different educational stages
• Pick one to start
• Use SPs in courses for
• history
• exam procedures
• Report of findings
• Create difficult patients/cases
• Group settings

25
How do we start an SP program

• Contact the University of Minnesota Medical
  School and other local health care institutions
• Share Patients
• Advertise at local community colleges
• Start small
• At Tulane Medical School, their SP program costs
  roughly 215/student.

26
WebSP

• Case Authoring
• Event Setup
• Scheduling
• Data collection
• Reporting
• Reports by assessment or learner
• Records assessment

27
Some final thoughts

• Faculty role
• What youre doing speaks so loud I cant hear
  what you are saying.
• Tell me, Ill forget
• Show me, I may remember
• But involve me, and Ill understand
• Chinese proverb