36x60 poster template

1
Quantitative Assessment of Interviewing
Competencies David A Goldberg MD, Steven P
Reidbord MD, Dongmei Yue MD California Pacific
Medical Center, San Francisco CA
BACKGROUND
METHOD
DISCUSSION
We adapted standardized measures from the
literature that offer a face-valid metric to
assess resident Core Competencies, specifically
aspects of Clinical Skills, Interpersonal
Communication, and Practice-based Learning
Improvement. Our simplified versions are easily
learned and applied. As expected, the two
measures are inversely correlated when initial
patient interviews are rated. Our preliminary
results suggest that meaningful variations may
occur as a result of training level. As used
here, they also serve as 360 degree evaluations
of the resident, and offer the resident direct
patient feedback under faculty supervision.
In recent years, educators have sought to
develop means to assess the six Core Competencies
defined by ACGME. It is centrally important in
psychiatry that we develop reliable and valid
methods to assess aspects of the doctor-patient
relationship, especially the competent formation
of a working alliance and competent conduct of
patient interviews. While assessment of resident
competency is especially challenging in these
areas, reliable and valid measures of working
alliance and therapist interview behavior have
been developed outside the educational context.
This project adapts and simplifies two such
measures. By applying them to a standardized
oral board interview format, and potentially in
other clinical settings, training programs can
quantitatively assess these crucial competencies.
In this pilot study, we selected a subset of six
questions from the Working Alliance Inventory
(WAI) Horvath and Greenberg, 1989 and ten from
the Inventory of Interview Behavior (IIB)
Friedman and Gelso, 2000. The WAI asks raters
to assess the degree to which goals, tasks, and
bonds are shared between the interviewing
resident and the interviewed patient. The IIB
asks raters to assess observable positive or
negative behaviors in the resident that may
interfere with a patient interview (and that,
from a psychodynamic viewpoint, may suggest
countertransference reactions). The item subsets
were chosen to simplify administration of the
questionnaires, and to exclude items that were
deemed hard to rate in the context of an initial
evaluation interview. Residents conducted a
30-minute oral board type interview with a
previously unknown patient. Each interview was
observed by a pair of faculty members out of a
pool of 11 such faculty. After the interview,
and a 30-minute case presentation and examination
period, each faculty observer, resident, and
patient independently completed shortened
versions of the WAI. Faculty also completed a
shortened version of the IIB, and rated, on a
seven-level Likert type scale, how much
difficulty a fully competent psychiatrist would
have forming a working alliance with this
patient. Likewise, the resident was asked to
rate on the same scale, How difficult was it to
form the working alliance with this patient?
FUTURE DIRECTIONS
Larger studies, of greater statistical power, are
needed to support our pilot effort, and to
confirm the reliability and validity of these
adapted measures. We have begun using the same
protocol for faculty-supervised PGY-3 outpatient
evaluations. We seek interested programs for
collaboration, in order to collect more data, and
to publish the method.
EDUCATIONAL OBJECTIVES
1. Develop an easily applied metric to measure
the working alliance and therapist behaviors in
initial evaluations. 2. Provide a quantitative
assessment of the ACGME Core Competencies of
Clinical Skills, Interpersonal Communication, and
Practice-based Learning Improvement. 3. Compare
multiple perspectives (resident, faculty, and
patient) on these measures. 4. Encourage
collaborative research on these and related
measures.
RESULTS
Ten residents participated in the practice oral
board interviews. There was a nonsignificant
trend (p 0.07) for patients to rate a stronger
working alliance than residents or faculty did,
and a nonsignificant trend (p 0.11) for working
alliance to improve as PGY level increases (see
Fig. 1). While adverse interview behaviors did
not vary statistically across PGY level, the rate
in PGY-1s was somewhat higher than at more
advanced PGY levels (see Fig. 2). Working
alliance showed a robust inverse correlation with
adverse interview behaviors (p .001). Rated
difficulty of the patient did not predict
alliance or therapist behavior ratings.
BIBLIOGRAPHY
Horvath, A. O., Greenberg, L. S. (1989).
Development and validation of the working
alliance. Journal of Counseling Psychology, 36,
223-233. Friedman S.M Gelso C. J. (2000). The
Development of the inventory of
countertransference behavior. Journal of Clinical
Psychology, 56 (9), 1221-1235.
FIGURE 1, LEFT FIGURE 2, RIGHT