Medical Interpreting: Outcomes, Errors, and Understanding

1
Medical Interpreting Outcomes, Errors, and
Understanding

• The Center for Immigrant Health
• New York University School of Medicine

2
Thanks!

• UHF
• Altman
• New York Community Trust
• Commonwealth Fund
• California Endowment

3
The Center for Immigrant Health NYU School of
Medicine

• Founded in 1989
• Network of community members/CBOs/FBOs,
  providers, researchers, facilities and
  administrators, program and policymakers
• Mission To facilitate the delivery of
  linguistically, culturally, and epidemiologically
  sensitive healthcare services to newcomer
  populations to reduce health disparities
• Research, Education, Program/Policy Dvpt

4
PARTNERS
5
Linguistic Diversity United States

• 1990 Census
• 31 million spoke a language other than English
• 14 million considered limited English proficient
  
• 2000 Census
• 47 million speak a language other than English
• 21 million considered limited English proficient

6
LEP Change by State

Source Access Project National Health Law
Program
7
Proportion of NYC Immigrant Population that is LEP
Source New York City Department of Planning
8
Study Series A Series of Firsts

• Randomized study impact of MI modes upon medical
  outcomes and costs
• Comparative study of accuracy
• Determination of efficiencies across modalities

9
InterventionRemote Simultaneous Medical
Interpreting System(RSMI)

• Trained Simultaneous Medical Interpreters
• Remotely Located, Pooled Resource
• Spanish, Mandarin, Cantonese, Bengla

10
Research Questions

• Does RSMI Improve Timely Diagnosis of Depression?
  
• Does RSMI Facilitate Appropriate Follow-up Care?

11
Research Questions

• Does RSMI Improve Adherence to Screening
  Guidelines?
• Does RSMI Improve Outcomes for Chronic Diseases?
• Diabetes, Hypertension, and Hypercholesterolemi
  a

12
Research Questions

• Does RSMI lead to fewer interpreting errors?
• Is RSMI a more efficient form of interpreting?
• Does RSMI lead to improved understanding of exit
  instructions

13
Cost Analysis Research Questions Pending

• Are visit lengths different across different
  modalities?
• Are there fewer repeat visits to achieve the same
  outcomes?
• Are there differences in test ordering behaviors,
  hence, costs?
• What are the opportunity costs vis-a-vis staff
  time?
• Hospitalizations/ER visits prevented

14
Error Analysis and Efficiency

• Scripted Encounters
• Spanish and Chinese TB, Menopause, Diabetes,
  Depression
• Bengali Breast Cancer
• RSMI, Proximate Consecutive, Over-the- telephone
  Consecutive, Ad Hoc
• Patient/Doctor Actors
• Encounters Audiotaped and Transcribed

15
Error Analysis Tool

• Word-by-word, and by concept
• Linguistic errors meaningful and non-meaningful
• Medical errors no, mild, moderate, high, and
  life-threatening significance
• HPI, meds/allergies, family history, diagnosis,
  plan, psychosocial, F/U, patient education

16
Error Analysis Panel

• Linguist and 3 physicians, at least 2 bilingual
• Scored separately, then discussed differences
  until consensus

17
Error Analysis

• Error rate per utterance
• Medically significant/Category
• Time
• Control for training

18
Spanish Error Analysis

• RSMI versus non-RSMI
• RSMI 30 as likely to result in potential
  medical error
• plt0.05

19
Spanish Error Analysis Odds Ratio of a
moderately significant to life-threatening error

• Trained Proximate Consecutive
• Trained Remote Consecutive
• Ad Hoc(18 yrs experience)
• Trained Remote Simultaneous

• 6.3
• 7.54
• 1.71
• 1.00
• plt0.001

20
Error Analysis Efficiency Results
Mean time (in seconds) for each group
21
Spanish Efficiency

• RSMI is 30 faster than the next fastest
  mode(ad hoc)
• RSMI two times faster than over-the-phone
  consecutive
• Spanish encounters more accurate and efficient
  with RSMI

22
Different Languages May be Different

• Mandarin Analyses in Progress

23
Bengali Error Analysis

• Standardized Training
• Standardized Practice
• One Script Across All Modes

24
Results/BengaliTraining
25
Training Matters

• 27 of errors made by untrained interpreters were
  of moderate or greater clinical significance vs.
  8.5 of errors made by trained interpreters
• Vocabulary precision rate .69 for trained vs.
  0.34 for the untrained

26
Training Error Examples

• Dr The results were positive which means that
  you carry the gene that puts you at risk for
  developing breast cancer
• Int The results were correct
• Dr One important thing that you have going for
  you is the fact that the cancer has probably been
  caught early
• Int One important thing is the fact that the
  cancer is working quickly in your body
• Dr The doxy could hurt your heart
• Int The doxy can give you pain

27
Study Design Outcomes

• Randomized Control for Discordant(Spanish-English,
  Mandarin-English, Cantonese-English)
• RSMI
• Usual and Customary
• Language Concordant Encounters English-English,
  Spanish-Spanish, Chinese-Chinese

28
Data CollectionDepression, Medical
Outcomes,Knowledge

• Clinic
• Intake Questionnaire, including Beck Depression
  Index
• Chart Reviews and Computerized Tracking for 1
  Year after Enrollment
• Exit Interviews
• Several Hundred Enrolled

• ER
• Intake Questionnaire
• Audiotaped Visits
• Exit Interviews
• Patient Understanding Scale

29
Clinic Population

• 782 patients enrolled
• RSMI and UC comparable in demographics including
  
• age, gender, education, years in U.S., primary
  language, English proficiency, acculturation, and
  self-reported health status

30
Randomization Assignment and Exposure Group
31
Spanish-speaking Clinic Study Patients, n465
Age
Gender
Country of Origin
n
Years in the U.S
n282
32
Chinese-speaking Study Patients, n208
Age
Gender
.
Years in the U.S
33
Spanish-speaking ER Patients, n225
Age
Gender
Country of Origin
Length of Stay in U.S.
34
Immigrants at Risk Language and Influenza
Vaccination

• 462 patients were enrolled in the study between
  November 2003 and July 2004
• 102 were at the highest risk of complications
  from influenza (chronic medical condition, age,
  or pregnancy)
• Only 10 patients in this group were referred for
  vaccination
• 9 received vaccination
• 54 patients aged 50 to 64 years without
  underlying medical conditions composed a second
  group who were eligible
• 4 in this group were referred for and
  received flu vaccination
• None of the Cantonese or Mandarin-speaking
  patients in either group received vaccination.

35
CLEAN

• RSMI associated with a higher referral rate for
  screening colonoscopy (OR of 1.7) compared with
  UC
• Physicians in language concordant encounters
  had lower rates of referrals for screening
  colonoscopy than language discordant

36
Instructions Given

• Audiotape analyses of 214 ER Spanish language
  encounters Spanish language concordant, RSMI, UC
• Trained RSMI
• mean instructions per encounter 14.29,std
  dv 6.9 equal to Spanish language concordant
• (14.33, std dev 6.33)
• Usual and customary interpreting
• significantly fewer instructions mean
  11.9, std dev 6.17

37
Diabetes Management

• Research question Does RSMI lead to improved
  management of diabetes mellitus?
• 74 patients with DM
• Young patient population at clinic
• Guidelines for DM management as per the American
  Diabetes Association (ADA)

38
Methodology

• Score computed based on ADA guidelines
  considering following
• A1C 1st visit Podiatry referral
• A1C 3 months later Eye doc referral
• BP recorded Nutrition referral
• LDL ordered Flu shot referral
• Weight recorded Pneumovax referral
• Urine spot or 24h ordered
• Smoking cessation referral (if eligible)
• Prescribed aspirin (if eligible)

39
Results
40
Diagnosis of Depression

• Research question Does RSMI Improve Timely
  Diagnosis of Depression?
• Why depression?
• Common disorder in the primary care setting
• Associated with significant morbidity
• Effective communication is key in diagnosis

41
Methodology

• Becks Depression Inventory
• Validated in Spanish and Chinese
• Screening tool
• Administered to patients at intake by research
  assistant
• BDI score of 4 or more considered as positive
• RSMI, UC, LC compared in terms of rate of
  diagnosis by physicians and matched against the
  expected rate from the BDI score

42
Results

• BDI ve rates
• BDI ve 153
• BDI -ve 309
• BDI not done 320
• intake process, BDI at end of interview
• Demographics comparable in BDI ve, -ve, and not
  done
• Significant trends observed
• Diagnosis rate best in LC, followed by RSMI and
  then UC
• Time to medication in new diagnosis of depression
  better in RSMI compared with UC

43
 Randomization  Comparing all 3 groups
 Chi-square P0.66, Fisher P0.65    Comparing
RSMI vs UC  Chi-square P0.41, Fisher P0.47
   Exposure       Comparing all 3 groups
 Chi-square P0.83, Fisher P0.85      
Comparing RSMI vs UC  Chi-square P0.58, Fisher
P0.76
44
Patient Satisfaction/Understanding Perception

• ER and Clinic Combined
• First Visits
• Language Concordant(E-E,Sp-Sp, M-M, Ca-Ca), RSMI,
  Usual and Customary

45
Understanding MD
Understands Pt Underst Explan Pt
Under Instructions
46
Satisfaction

• How would you rate the MD overall?
• LC RSMI UC
• Ex 63 56 49
• Gd 32 40 44
• Fa 4 4 4

• How satisfied with care overall?
• LC RSMI UC
• Very 57 57 47
• Swht 38 40 48
• SwtDis 4 3 5

47
How well did the method protect your privacy?

• RSMI UC
• Very Well 49 40
• Well 44 49
• Not Well 7 10
• Poor 0 1

48
No Difference

• How well did the interpreter understand you?
• Did the interpreter listen carefully(yes/no)?
• Did the interpreter treat you with
  respect?(trend)
• How well did the interpreter interpret?(trend)

49

• Implications for Policy

Training matters.training programs should be
systematized. Investment in dissemination of
RSMI to users of telephone interpreting services
will provide cost savings even without
consideration of seemingly improved
outcomes(?). For Spanish-language encounters,
RSMI will likely provide the most accurate
results, and better patient outcomes. First ever
randomized trial of impact of varying modes of
interpreting. Results can provide basis for
institutional and federal/state/local policy
evidence-based decision-making.