Comparison of SelfReported and Medical Record Health Care Utilization measures

1
Comparison of Self-Reported and Medical Record
Health Care Utilization measures

• Concurrent criterion validation
  of a domain on health care
  utilization as part of a QOL instrument
  by comparing it with
  charts from a centralized medical chart center
  for the area

2
Information not provided Abstract
1992Evaluative tool QOL Benign prostatic
hyperplasia

• Random selection of 110 men
• in male across the span of severity of BPH (no to
  severe symptoms)
• 6 domains symptoms, degree of bother, BPH
  specific interference with activities, psycho,
  worries-concerns, sexual satisfaction
• tested for test-retest reliability,
  responsiveness to change,
• internal consistency, validity

3
Measurement

• Pre-existing self-reported questionnaire on QOL
  for benign prostate hyperplasia,
• with a 5-item domain on health care utilization
• domains same 6 health care use, medical Hx,
  smoking, sociodemo info

4
Proposed use of the measurement

• Predictive tool
• for research
• for administrative (health care planning, policy)
  purposes

5
Research question

• Not clearly stated
• Pop male 40-79 community-dwelling, Olmsted
  county
• Intervention self-administered questionnaire QOL
• Outcomes comparison with centralized medical
  records of the county,
• in-pt nights in last yr
• in-pt nights for mental reasons,
• visits to MD in last 2 wks,
• visits to MD in 1 yr,
• visits to MD for any of 12 symptoms of HBP
  (yes/no)
• Direction ?

6
Aspect of validity

• Concurrent criterion validity

7
Study design and methods

• Question not clear
• Pop random sample (size 500,10 hospitalized in
  last yr)
• exclusion criteria not clear
• Meaningful criterion yes
• Appropriate comparison independant, blind?
• Tool
• ideally only measuring health care use
• scaling of answers not mentionned, questions not
  stated
• Statistical analysis no pre-set kappa or phi

8
Bias

• Selection bias
• Response bias (55 returned the quest)
• Information bias
• Misclassification related to extraction data
  chart
• Recall bias
• Social desirability (underreporting)
• Reporting bias ( in-hosp for mental prob)

9
Answered the research question?

• What was the research question?
• They answered the question we guessed they
  addressed
• Tool
• 1 question did not report any result
• 4 questions reported as reliability!

10
Implications

• Administrative, planning, research use for male
  40-79
• validity mostly
• for nights in-pt in last yr
• recalling seeing a MD in last 2 wks
• Clinical use no application foreseen

11
Analysis of criterion validity

• Dichotomous phi (?)
• Continuous weighted kappa, Pearson, ICC
• Ordinal consider as if continuous or use
  Wilcoxon signed rank test

12
Analysis criterion validity

• Dichotomous based on 2X2 table
• F?BC-AD? /(AB)(CD)(AC)(BD)1/2
• a b
• c d

13
In the last year, have you seen a physician for
any of the above symptoms?

• Computerized charts
• Y N
• Questionnaire Y 50 20
• N 10 30
• ? 0.45
• However, not reported in methods or results

14
Analysis of ordinal data

• Wilcoxon signed sum rank

15
How many in-pt nights in last yr?

• Quest 0 2 5 5 5 7 15
• Charts 0 0 5 7 10 3 12
• sign? 0 2 0 -2 -5 4 3
• signRank? 1.5 3.5 1.5 -3.5 -7 6 5
• add the positives (1.5 3.5 1.5 6 5)
  17.5
• add the negatives (-3.5 - 7) -10.5
• Ho sumpositive ?sum negative?
• then use a table to find p-value or use an
  approximate z test then find p-value
• appropriate use of Wilcoxon in the statistical
  analysis

16
Ordinal treat as continuous

• Choice of weighted kappa, Pearson, ICC
• used Weighted kappa
• kappa (Po- Pe) / (1-Pe)
• quadratic weight use (? exact disagreement)2
• Here, weight use (?exact agreement)2
• wKappa 1- ?wij X Poij /(?wij X Peij) why use
  simple agreement, kappa and Wilcoxon?

17
a word on reliabilityPearson vs ICC vs kappa

• If use quadratic weight (?disagreement)2 then
  kappa ICC
• kappa only for dichotomous variables
• Pearson linear regression ANOVA
• measures how well you may fit the obs on a
  straight line

18
Pearson vs ICC

• Pearson compares 2 obs at a time
• if 10 raters 45 Pearson correlation
  coefficients!
• or only 1 ICC

19
References

• Health Measurement Scales. A practical guide to
  their development and use
• DL Steiner, GR Norman
• Statistics The Bare Essentials
• DL Steiner, GR Norman
• HRM 727 Measurement Tool, course package G Norman