Measuring patient experiences: Who wants to know what and why

1
Measuring patient experiences Who wants to know
what and why?

• Diana Delnoij
• April 25, 2008

2
Consumer Quality Index

• National standard for measuring patient
  experiences
• Based on CAHPS and QUOTE surveys
• Questionnaires and interview protocols measuring
• What patients/consumers find important in health
  care
• What their actual experiences are
• How they rate the overall quality of care.
• Protocols, instructions and guidelines for
• Sampling
• Data collection
• Analyses
• Reporting formats.
• Registered trademark owned by the Centre for
  Consumer Experience in Health Care

3
CQI serves seven purposes

• CQI information is used as
• Consumer information for
• (1) Individual consumers
• (2) Patient organisations
• Purchasing information for (3) health insurers
• Quality information for
• (4) Providers
• (5) The inspectorate for health care
• Policy information for the (6) Ministry of Health
• Scientific information for (7) researchers

4
CQI questionnaires

• Available
• CQI health care and health insurance
• CQI cataract surgery
• CQI THA/TKA
• CQI longterm care
• CQI disabilities
• (CQI physiotherapy)
• (CQI diabetes)
• (CQI general practice)
• In development
• CQI hospital care
• CQI renal failure
• CQI mental health care
• CQI pharmaceutical care
• CQI maternity care
• CQI rheumatoid arthritis
• CQI breast cancer care
• CQI varicose veins

National surveys
5
Types of questions

• Frequency with which quality criteria are met
• Never, sometimes, usually, always
• Importance of quality criteria
• Not important ?? extremely important
• Access to care degree to which lack of access
  is perceived as a problem
• A big problem, a small problem, not a problem
• General rating of the quality of care
• Scale from 0 (worst possible) to 10 (best
  possible)
• Effects of care and adherence to professional
  guidelines
• Background characteristics
• Age, gender, ethnicity, education, general health
  status, comorbidity

6
Development of a CQI instrument (1)

• Preparatory phase
• Demarcation of target group
• Gaining support from providers, health insurers,
  patient/consumer organisations
• Determining sample strategy
• Construction phase
• Qualitative research
• Focus group discussions
• Interviews
• Professional guidelines
• Existing questionnaires
• Input from professionals, health insurers,
  patient/consumer organisations etc.
• Small scale testing

7
Development of a CQI instrument (2)

• Psychometric testing
• n300-1200
• Factor analyses
• Reliability testing
• Total and item non-response
• Skewness etc.
• ? adjustment of instrument ?
• Testing discriminative power
• n1000-20000 nested in n15-25 or more
• Case mix adjusters
• Intraclass correlation
• Hierarchical regression analyses to look for
  performance above or below average
• ? final adjustment of instrument

8
Examples of utilization of CQI data
9
Consumer information Who are the best diabetes
providers?
Accessibility of the general practice
Scale 1 never ?? 4 always Source Rupp e.a.,
AMC University of Amsterdam
10
Consumer information What is the best health
plan?
11
Strategic purchasing Which hospitals follow
guidelines with respect to cataract surgery?
32 hospitals
12
Quality information Which areas need improvement
in the pharmacy?
Source NIVEL/ARGO RU Groningen, in press
13
Quality information Which areas need improvement
in the pharmacy?
Source NIVEL/ARGO RU Groningen, in press
14
Scientific informationDoes public disclosure
lead to improvement?
15
Who wants to know what and why?

• Answer this question before you develop a patient
  survey!
• Answer has consequences for
• Stakeholder involvement
• Types of questions (satisfaction, experience,
  importance)
• Degree of standardization
• Method of testing (psychometric or also
  discriminative)
• Survey implementation (external survey vendor or
  internal research?)
• Scientific rigor

16
Lessons learned

• Purpose external accountability
• Scientific rigor
• Objectivity and data quality
• Composite measures or ratings
• Purpose comparative information
• Standardization
• Test discriminative power
• Advanced statistical methods
• Adjustment for patient mix/reporting behaviour
• Purpose quality improvement
• Involvement of managers and professionals
• Importance scores
• Detailed internal feedback
• Test sensitivity to changes over time