Using the Web for Surveys of Medical Providers

1
Using the Web for Surveys of Medical Providers

• Vasudha Narayanan
• Presented at
• Third International Conference on Establishment
  Surveys
• June 21, 2007

2
Acknowledgements

• Co-Authors
• Stephanie Fry, Jennifer Crafts Pamela
  Giambo from Westat
• Project Officers
• David Clark, William Taylor, Gladys Valentin
  Mei Wang from Centers for Medicare Medicaid
  Services
• This material was prepared by Westat Inc., under
  contract with the Centers for Medicare Medicaid
  Services (CMS), an agency of the U.S. Department
  of Health and Human Services. The contents
  presented do not necessarily reflect CMS policy

3
Overview

• About the Surveys
• Survey Methods
• Questionnaire Design Issues
• Sampling and Contacting Providers
• Completes by Mode
• Factors Affecting Web Response
• Conclusions

4
About MCPSS QIO Survey

• Two Surveys for Centers for Medicare Medicaid
  Services (CMS)
• QIO Provider Survey (2004)
• MCPSS (2006 2007)
• National Sample of Medicare Providers
• Satisfaction Surveys
• Approx. 15 minute survey
• Multi-Mode Data Collection

5
Survey Methods

• Web with telephone follow-up paper on request
• Telephone and paper surveys to
• Increase response rates
• Reduce potential bias
• Importance of
• Screening
• High quality mail outs
• Professional Interviewers

6
Survey Methods (cont.)

• Field Period (12-16 weeks)
• Screener
• Notification of web survey
• Reminder letters/e-mails
• Hard copy questionnaire mailing/fax
• Telephone non-response
• Response rates (AAPOR-RR3)
• QIO 90
• MCPSS 65

7
Questionnaire Design Usability Testing

• Purpose of Testing
• Ease of navigation
• User satisfaction and perceived burden
• Profile of Testers
• Response mode bias

8
Questionnaire Design Usability Testing Issues

• Allow for multiple respondents
• Clarity of instructions to designate a
  colleague
• Clarity of instruction for submitting survey
• Ability to skip sections and questions

9
Elements of the Final Design

• Allow multiple respondents for each survey
• Track section-level designees
• Track section-level status
• Crisp and easy to navigate web survey
• Seamless link between CATI Web

10
(No Transcript)
11
(No Transcript)
12
Completes by Mode

• non-response mail out of hard-copy survey in
  2004
• Highest Rates of Web response Hospitals
• Lowest Rates of Web response Physicians

Web Mail/Fax Telephone Completes
2004 32.2 21.9 45.9 17,628
2006 40.1 1.7 58.1 16,121
2007 55.1 0.9 43.9 17,747
13
Factors Affecting Web Response

• Percentage Web access
• 93 physician practices (2007)
• 97 other providers (2007)
• Hypotheses
• Ease of Web access at work station
• Ease of survey website navigation
• Ability to obtain help when needed

14
Conclusions

• Feasibility of a Web surveys no longer an issue
• Necessary factors for survey management and
  achieving high response rates
• telephone non-response follow up essential
• Frequent interface between data collection modes
• Allow the respondent maximum flexibility while
  maintaining survey integrity

15
Contacting the Authors

• Vasudha Narayanan (vasudhanarayanan_at_westat.com)
• Stephanie Fry (stephaniefry_at_westat.com)
• Jennifer Crafts (jennifercrafts_at_westat.com)
• Pamela Giambo (pamelagiambo_at_westat.com)

16
Thank you