Designing Quality Patient Surveys

1
Designing Quality Patient Surveys

• Brian F. Pendleton, Ph.D.
• Susan Labuda Schrop, M.S.
• The University of Akron
• Northeastern Ohio Universities Colleges of
  Medicine and Pharmacy
• OAFC 9th Annual Conference, October 2008

2
Why survey patients?

• Advantages of written surveys
• acquire information
• response comes directly from patients
• not dependent on personal contact
• compare patient or your clinic to an established
  norm

3
Goal of the Seminar

• To provide you with an introduction to the
  knowledge and skills needed to develop and
  critique surveys to elicit information from
  patients

4
Objectives for the Seminar

• Following the seminar, you will be able to
• design a survey considering factors such as
  content, format, wording, order of items,
  readability, understandability, literacy level,
  and aesthetics
• examine a survey to assess its strengths and
  identify areas needing improvement and
• identify the consequences of various survey
  administration methods in order to optimize the
  use of surveys to collect information from
  patients.

5
Seminar Outline

• Key elements of effective survey design
• Implications and applications
• Practice reviewing and editing a survey
• Appendix for further reference

6
A survey is . . .

• a system for collecting information to describe,
  compare, or explain knowledge, attitudes and
  behavior. Surveys involve setting objectives for
  information collection, preparing a sound data
  collection instrument, administering the survey,
  and analyzing and interpreting the data.

7
Surveys Youve Received

• The Good,
• the Bad
• and
• the Ugly

8
(No Transcript)
9
Key Elements of Effective Survey Design

• Design
• Content

10
Design-Length

• Long enough to ask necessary questions
• Consider time to complete

11
Design-Readability

• Font
• Reading level
• Layout
• Note Principles apply to both to survey items
  and instructions/directions

12
Design-Font

• Use font size no smaller than 12 14 pt
• Use both upper and lower case letters
• Use dark print on light background
• White, non-glossy paper
• Black ink
• Typeface with serifs
• For emphasis
• Use bolding or underlining
• Avoid all UPPERCASE or italics

13
Design-Sample Fonts

• This is a serif font
• Times New Roman
• This is a sans serif font
• Arial

14
Design-Reading Level

• Short sentences or stems (8 10 words 15 max)
• Avoid multi-syllabic words
• Avoid long strings of text
• Avoid landscaped pages
• Avoid doctor speak
• Active voice
• Lowest possible level (readability measures)

15
Design-Format/Layout

• Appearance of survey
• Not crowded
• White space (target 50)
• Placement of items
• Demographic questions first
• Sensitive topics near the end

16
Design-Format/Layout (contd)

• Group items with a similar format
• Provide clear instructions with each new question
  type
• If you must include a sample question, show the
  correct way to respond.
• Include instructions at the end of each page
• Left justify all text (right ragged edge)
• No images behind the text (ghost images)
• Help your reader follow lists in tables by
  shading every other row

17
Design-Sample Table
18
(No Transcript)
19
Content Questions should produce response
variability

• Are you against substance abuse? Circle
• Yes No
• Most people are going to answer yes and provide
  no response variability.

20
Content Transitions between questions should be
smooth

• Group similar questions
• Increases comfort of respondents
• Increases response rate

21
Content Questions should not presuppose a
certain state ofaffairs among patients

• Are you satisfied with your current primary care
  provider? Circle Yes No
• Will present a problem for someone who does not
  have a primary care provider
• Provides for no response variability
• Better
• Are you satisfied with your current primary care
  provider? Yes No Do not have a primary
  care provider

22
Questions should not presupposea certain state
of affairs among patients (contd)

• What percent of your household budget is spent on
  prescription drugs?
• Few will know this amount without looking it up
  most likely they will not take the time to look
  it up. Some may not have access to budget detail
  some will not have a household budget.

23
Content The question should be less than 20
words

• For our purposes, let us consider breakfast to
  be a meal, eaten before 1000 in the morning,
  that includes some protein such as eggs, meat, or
  milk, some grain such as toast or cereal, and
  some fruit or vegetable. Using that definition,
  did you have breakfast yesterday?
• Better
• What did you eat for a morning meal yesterday?

24
Content Avoid mixing instructions with the
question

• On the line next to the statement indicate the
  number that corresponds to your agreement with
  the statement using a scale of 1 to 5, with 1
  being the strongest disagreement and 5 being the
  strongest agreement. Marijuana use is not
  healthy.
• Better Marijuana use is not healthy.
• or
• Marijuana use is bad for my health.
• 1strongly disagree 2disagree 3dont
  know 4agree 5strongly agree

25
Content Avoid bias

• Leading questions
• Modifiers such as almost, everyone, usually
• Implication of a desired answer
• Judgmental
• Vague terms

26
Content Avoid leading questions

• Most people agree that public education reform is
  needed. Do you?
• Implies a judgment if the respondent does not
  agree
• Better
• Public education reform is needed.
• Agree Disagree

27
Content Do not imply a desired answer, be
judgmental, or use vaguely defined words

• How the question is worded may affect the
  response
• Dont you agree that abortion is murder?
• How often did you use marijuana as a high school
  student?
• Better
• Have you ever used marijuana?

28
Content Do not use emotionally loaded words or
assume a socially conventional interpretation

• Indicate how you feel about putting your loved
  one in a nursing home.
• 1) not emotional
• 2) somewhat emotional
• 3) very emotional
• The question does not recognize that very
  emotional could reflect relief or distress, very
  opposite emotions.

29
Content Do not use unfamiliar words or
abbreviations

• Consider the literacy level and range of
  expertise of respondents
• How many UTIs did you experience in the past
  year? or
• Would you agree that Bgt2S.E. is better than t.05
  for plt.05?

30
Content Avoid branching and skipping, if possible

• 1. Do you currently have car insurance?
• Yes No If no, go to question 4.
• 2. How much is your annual car insurance premium?
  ________
• 3. What percent of your time is spent dealing
  with car insurance questions? _____

31
Content Avoid branching and skipping, if
possible (contd)

• Better Rewrite so it applies to everyone.
• How much is your annual car insurance premium?
• (write 0 if none)

32
Content Do not ask the patient to order or rank
a series of more than five items

• Rank order the following according to how much
  you feel each contributes to stress in your life.

33
Content Limit content to what you need to
say/ask

• Ask only necessary questions
• Avoid unnecessary words or phrases (and big
  words, too)
• For example
• Does your most recent change in medication
  cause you to experience dizziness?
• Better
• Does your new medicine make you dizzy?

34
Content Ask only necessary questions

• This question was asked of female patients at a
  gynecology practice
• Gender 1) male 2) female

35
(No Transcript)
36
Response Formats

• Selected response/fixed choice
• Open-ended/interpretive
• Likert scale
• Value scale
• Semantic differential scale
• Paired comparison

37
Selected response orfixed choice

• Choose from variety of possible answers i.e.,
  multiple choice
• Fixed-choice/multiple choice
• forces the patient to choose from categories
  provided
• opposite of open-ended (interpretive) questions

38
Open-ended or interpretive

• Requires patients to develop their own answers
• advantage obtain more depth and insight into the
  question
• disadvantage time it takes to code and analyze
  open-ended questions

39
Likert scale

• Often takes the form of
• 1strongly disagree 2disagree 3neutral
  4agree 5strongly agree
• Four or five response categories are ideal
• odd number with a neutral choice allows the
  patient to select a neutral position
• even number removes the neutral option and
  forces a commitment from the patient
• Too many Likert-scaled questions may result in
  regression toward the mean

40
Value scale

• Measures a patients preference with response of
  yes/no or good/bad or agree/disagree, etc.

41
Semantic differential scale

• Measures the patients reaction to stimulus words
• Provides the patient with a 7-choice scale
  between two bi-polar, contrasting adjectives
• My visit to this clinic was
• Good ______________________________
  Bad
• Interesting ______________________________
  Boring
• Enjoyable ______________________________
  Painful
• The visual scale can include numbers, labels or
  be unlabeled under the horizontal line

42
Paired comparison(forced choice)

• Patient selects more favorable of two choices
• Example
• Do you prefer X or Y?

43
General ConsiderationsAccommodate all possible
answers

• Which of the following doctors have you seen in
  the past year?
• OB/GYN
• Family doctor
• Not comprehensive of all specialties
• What if the patient didnt see a specialist or
  any physician in the past year?

44
Accommodate all possible answers(contd)

• Better Add an option for not having seen a
  physician in the past year and add categories (if
  not too lengthy) to cover all specialties or
  areas.
• Which of the following doctors have you seen in
  the past year?
• OB/GYN
• Family doctor
• Other specialties
• Did not see a doctor

45
Provide clear, unambiguous choices

• Where did you grow up?
• City
• Suburbs
• Farm
• Country
• What if the patient moved frequently while
  growing up, and is there a difference between a
  farm and the country, or between the city
  and the suburbs?

46
Provide clear, unambiguous choices(contd)

• Better
• Circle the letter that best reflects the size of
  the
• community in which you spent most of your life
• large city/suburb (population 500,000 or more)
• city/suburb of moderate size (population
  50,000-500,000)
• small city/suburb (population 10,000-50,000)
• town or rural unincorporated area (population
  10,000 or less)

47
Implications

• Field Trials/Pilot Testing
• Response Rate
• Impact
• Regulations

48
Field Trials/Pilot Testing

• Purpose detect problems/areas for improvement in
  data collection instruments and data collection
  procedures before survey is actually administered
• Use pilot/pretest subjects who reflect the
  characteristics of those who will be your final
  respondents
• All procedures and instructions should be as
  similar as possible to the actual project

49
Response Rate

• If lt 70 response rate, even after repeated
  follow-ups, findings are not generalizable beyond
  the sample

50
Increasing Response Rate

• Topic is of interest and/or importance to the
  group being surveyed
• Use of proven distribution techniques
• personalize the request
• select paper considering features such as quality
  and color
• use professional duplicating
• if mailed, provide stamped, self-addressed
  envelope as well as return address on the survey

51
Administration Techniques and Response Rate

• Time of day
• Who makes the request (receptionist, nurse,
• physician, researcher)
• Where it is distributed
• How is it administered
• How it is distributed (mail, in person, Internet)

52
Additional Necessary Considerations

• Literacy issues
• Cultural issues
• Ethnic
• Age
• Regional
• Context issues
• Legal issues
• Medical record
• Anonymity

53
Impact

• Increased comfort of respondent
• Increased response rate
• More reliable responses

54
Regulations

• Informed consent/IRB
• expedited review except when survey is not
  anonymous and it places the person at criminal or
  financial risk
• research involving children and vulnerable
  populations (individuals with diminished
  autonomy) require additional safeguards
  generally must have full board review
• for those in private practice, obtain from a
  hospital or university
• need for approval from multiple IRBs
• HIPAA
• must consider issues of individually identifiable
  health information (IIHI), if collecting any
  IIHI, then written informed consent is required

55
FYI

• NIH Office of Extramural Research
• On-line training course
• Protecting Human Research Participants
• http//phrp.nihtraining.com/users/login.php

56
Application
57
Small Group Activity

• Review the scenario
• Examine the survey
• List positive features
• List features that need to be improved
• Report suggestions to the group

58
Sample Survey(Part 1)

• Getwell Healthcare Center
• PLEASE ANSWER THE FOLLOWING QUESTIONS ABOUT FLU
  SHOTS AND GIVE YOUR ANSWERS BACK TO THE
  RECEPTIONIST. YOUR ANSWERS ARE CONFIDENTIAL AND
  ANONYMOUS. THANKS!
• Patient ID _____________________________
• Financial class __________________________
• Your age _____________________
• Your sex _____________________
• Did you receive the letter of flu and pneumonia
  shots that Dr. Getwell sent to you? Y N
• Do you plan to get a flu shot this year? Y
  N Have not decided
• Plan to get it elsewhere? N Y
  Where?__________________
• Highest grade in school 4 5 6 7 8
  9 10 11 12
• OR college 1 2 3
  4 5 6 more than 6

59
Sample Survey(Part 2)

• 1. I have had a flu shot in the last 5 years.
  Y N DK
• 2. My doctor recommended that I get a flu shot
  this year. T F DK
• 3. Because of my age or health, I could have
  serious
• complications if I get sick with the flu.
  T F DK
• 4. Because of my age or health, I am a person who
  should
• get a flu shot. Y N DK
• 5. If I get a flu shot, it might make me
  sick. T F DK
• 6. If I get a flu shot, it will probably give me
  the flu. T F DK
• 7. Flu shots can cause pain in the arm where the
  shot is given. T F DK
• 8. I am afraid of any immunizations. T F DK
• 9. If my doctor tells me I should get a flu shot,
  I will get one. Y N DK
• 10. Flu shots are expensive. T F DK
• 11. Flu shots almost always keep you from getting
  the flu. T F DK
• 12. I think that there is a good chance that I
  will get the flu
• this year.
  T F DK

60
Sample Survey(Part 3)

• Give this questionnaire to the receptionist.
  She will give it to the doctor who will discuss
  any of these questions with you if you like. Do
  you have any other concerns about the flu shot
  that you would like to discuss with the doctor?
  If yes, __________________________________________
  __________________________________________________
  __________________________________________________
  ______________________________________
• Did the patient get a flu shot? Y N
• YYes, NNo, TTrue, FFalse, DKDont know

61
Questions?
62
Key Elements of Your Survey Toolbox

• Know (and stick to) your purpose
• Consider your target audience
• Prepare an easy-to-read (and answer) survey

63
Keys to Success

• Consider
• design
• length, readability, font, reading level,
  format/layout
• content
• question format, response format
• implications
• response rate, administration, regulations

64
Appendix

• Technical information about measurement pitfalls
• Options you give your patients can affect how
  they answer and what the analyses will tell you

65
Additional considerationsLevels of measurement

• Nominal, ordinal, interval, ratio
• versus
• Discrete (nonmetric) or continuous (metric)

65
66
Use an undecided category cautiously

• A similar concern when undecided is used as a
  response category between agree and disagree
  options.
• For example
• 5strongly disagree 4disagree 3 neutral
  2agree 1 strongly agree
• No assurance that a patient choosing the
  neutral position actually has a neutral opinion
  about the topic. A neutral choice may be the
  result of
• ignorance
• uncooperativeness
• reading difficulty
• reluctance to answer
• inapplicability

66
67
Undecided (contd)

• Better
• Word responses so that neutral can be avoided
  (e.g., use tend to disagree) include other
  options like not applicable, or no basis for
  judgment.

67
68
The apple pie problem

• Difficulty patients may have when asked to rate
  items for which the general level of approval is
  already high.
• For example If patients are asked to consider
  the following scale when asked to rate the
  importance of preventive health practices
  (preventing diabetes, heart problems, kidney
  problems, eye disease, ulcers, etc.) using
• 1) no importance 3) moderate importance
• 2) low importance 4) high importance

68
69
Apple pie (contd)

• Patients will tend to rate all the prevention
  topics as highly important making it difficult to
  separate topics of greatest importance from those
  of less.
• Better Ask patients to rank if there are many
  topics ask the patient to select the 3 or 5 they
  consider to be the most important.

69