QUESTIONNAIRE DESIGN

1
QUESTIONNAIRE DESIGN

• for the GEMP 3 student

2
Objectives

• Advantages and disadvantages of questionnaires
• Design of questionnaires
• Different types of questions used
• How to avoid common problems and pitfalls

3
Method of Data Collection

• Three main methods
• Observation
• Interviews
• Questionnaires
• A study may employ more than one method of data
  collection.

4
Examples

• Health and the eating habits of school children
  at school
• (Survey, questionnaire)
• The average waiting time at clinics and the
  factors that influence the waiting time.
• (Survey, on-site observation)
• Analysis of the causes of cor pulmonale in
  children in Johannesburg..
• (Survey, secondary data)
• The understanding of the Core PHC package among
  health workers employed by the city
• (Survey, on-site observation)

5
What is a questionnaire?

• An instrument (form) to
• collect answers to questions
• collect factual data gathers information or
  measures
• A series of written questions / items in a fixed,
  rational order

6
Why using a questionnaire?

• A well designed questionnaire
• Gives appropriate data which answers your
  research question
• Minimises potential sources of bias
• Will more likely be completed

7
What kind of questions can you ask?

• Facts (Demographics)
• age, sex, post code, number of children,
  religious affiliations, etc
• Attitudes
• perceptions beliefs , belief in aliens,
  attitude towards crime, etc.
• Knowledge
• description of homeopathy, good dietary
  practice,
• Behaviours
• dental attendance, medical history, criminal
  record, drug taking habits

8
Where to start?

• 1. List variables
• Sources include patients/subjects (focus groups,
  key informant interviews), clinical observation,
  theory or conceptual framework, prior research,
  and expert opinion.
•  
• 2. Borrow from other instruments
• Save development effort (reinventing the wheel)
• Borrow reliability, validity, variance estimates
• Facilitate comparison with previous studies
• 3. Solicit input from colleagues and friends

9
Advantages of questionnaires

• Can reach a large number of people relatively
  easily and economically (especially postal/E-mail
  questionnaires)
• Provide quantifiable answers
• Relatively easy to analyse

10
Disadvantages of questionnaires

• Provides only limited insight into problem
• Limited response allowed by questions
• Maybe not the right questions are asked
• Varying response
• Misunderstanding/misinterpretation
• Need to get it right first time
• Hard to chase after missing data

11
Types of Questionnaire

• Self-administrated
• On-site
• By post
• E-mail/Internet
• Interviewer-administrated
• Face to face
• Telephone

12
Self-administered questionnaire

• Advantages
• Cheap and easy to administer
• Preserves confidentiality
• Completed at respondent's convenience
• Administered in a standard manner
• No influence by interviewer

13
Self-administered questionnaire

• Disadvantages
• Low response rate
• Questions can be misunderstood
• No control by interviewer
• Time loss

14
Interview-administered questionnaire

• Advantages
• Interviewer can clarify unclear questions
• Literacy is not required
• Interviewer can collect more complex answers and
  observations
• Interviewer can minimize missing and
  inappropriate responses
• Interviewer can prevent respondentfrom answering
  out of sequence
• Clarification of ambiguity
• Quick answers

15
Interview-administered questionnaire

• Disadvantages
• Interviewer bias
• Needs more resources
• Only short questionnaires possible
• Difficult for sensitive issues

16
What makes a well designed questionnaire?

• Highly structured
• Collects the same types of information
• Standardised
• Allows quantitative and systematic analysis

17
Before starting..

• Questionnaire design is integral part of the
  study, not an add-on!

18
Stages in designing a questionnaire (1)

• Planning the study
• Decide on goals of study
• Know the subject
• literature, experts
• Formulate a hypothesis
• Define information needed to test the hypothesis

19
Stages in designing a questionnaire (2)

• Determine study population
• Know the respondents
• Occupation
• Special sensitivities
• Education
• Ethnic
• Language

20
Stages in designing a questionnaire (3)

• Design questions
• Content of the questions
• Format of the questions
• Presentation and layout
• Coding schedule (if appropriate)
• Pilot and refine questionnaire

21
Stages in designing a questionnaire (4)

• Complete study protocol
• Determine
• Cost
• Time
• Sample size
• Response rate

22
Questionnaire Structure

• Three parts
• Introduction
• Questions
• Concluding remarks

23
Five Functions of the Introduction

• Identification of the survey or respondent
• Who you are
• Who you work for
• How and where you can be contacted
• Purpose of survey
• What you are investigating/researching
• Explanation of respondent selection
• Where you obtained the respondents name
• Request for participation/provide incentive
• Incentives
• Anonymity
• Confidentiality
• Screening of respondent

24
Other hints

• Encourage the respondent to answer all questions
  as truthfully as possible
• Emphasise that there are no "right" or "wrong"
  answers
• Try to make the research sound interesting and
  relevant.
• Use a form of language, in terms of word and
  sentence length, which will be comprehensible to
  the respondent. Fleisch Reading Ease Score
  (Grade 8 maximum)
• Provide an estimate of how long it takes to
  complete the questionnaire (be honest)

25
Concluding Remarks (at the end of questionnaire)

• Thank the respondent again for taking part
• Remind them to check they have answered all
  questions
• Ask them to return the questionnaire in the
  envelope provided

26
Basic Rules (1)

• KISS ? keep it short and simple
• Length of questionnaire shorter ? ?response
  rates
• Appearance affects
• Response rate
• Data summarisation and analysis
• ? easy on the eye

27
Types of Questionnaire
28
Basic Rules (2)

• Number all items and pages
• Put an identifying mark on all pages
• Put study title in bold on first page
• Print directions in bold
• Remember the Unique Identifier

29
Basic Rules (3)

• Question order
• Easy ? difficult
• General ? particular
• Factual ? abstract
• Starting questions
• Simple
• With closed format
• Relevant to main subject
• Non-threatening
• Neither demographic nor personal questions
• Be aware of ordering effects!

30
Basic rules (4)

• Group questions by
• Topic/ response options
• Dont put most important item last
• Questionnaire likely to be completed if
• relevant
• logical

31
Content of Questions

• Clear focus on research question
• Avoid sidetracking
• Avoid unnecessary information
• Demographic information
• Contact information (if non-anonymised)

32
Questionnaires Questions - What do you ask?

• Ensure that the opening questions are clearly
  relevant to the purpose of the study
• Consider putting factual questions at the end of
  the questionnaire
• Personal or sensitive questions should appear
  later
• Consider including some open questions

33
Questionnaire Design General Principles
34
Questionnaire DesignGeneral Principles (cont)
35
Questionnaire DesignGeneral Principles (cont)

• Length
• No longer than absolutely necessary (24 pages)
• Use branching questions and skip patterns to
  reduce length
• Avoid deceptively short questions with high
  respondent burden (complex tables, rank ordering,
  mental calculations, check all that apply)

36
Questionnaire DesignGeneral Principles (cont)
37
Questionnaire DesignGeneral Principles (cont)
38
Questionnaire DesignGeneral Principles (cont)
39
Questionnaire DesignGeneral Principles (cont)
40
Coding Schedule

• Questionnaire can be pre-coded
• Quicker and easier data entry
• Examples
• Male ? 1 Female ? 2 Dont know ? 3
• Ill ? 1 Not ill ? 0 Dont know ? 9
• Single ? 1 Married ? 2 Separated ? 3
• Divorced ? 4 Dont know ? 5

41
Precoding the Questionnaire

• Numbers are preferred for two reasons
• Numbers are easier and faster to keystroke into a
  computer file
• Computer tabulation programs are more efficient
  when they process numbers

42
Phrasing Questions
43
Phrasing Questions (cont)
44
Phrasing Questions (cont)

• 5. One topic per question
• Avoid double-barreled questions
• Do black Africans and Coloureds suffer from
  discrimination
•  
• 6. Specify an appropriate time frame
• Typical/usual versus maximal/minimal
• Depends on salience of topic
• 7. Improving recall
• Aided recall (memory cues, prompts)
• Bounded recall (time window)
• Records or diaries

45
Phrasing Questions (cont)

• 8. Mutually exclusive and exhaustive response
  options
• Use of other fields 
• 9. Consider question polarity and sequence
  (survey as conversation)
• Avoid yea-saying patterns and carryover
  effects, but maintain consistency when possible
• 10. Check for technical accuracy

46
Lets try a few examples, and see if you can
identify the problem
47
Example 1
Do you approve or disapprove of the recreational
use of cannabis? (Please tick the appropriate
box) Yes ? No ?
Cant answer question using answer categories
provided Change to Approve/ Disapprove? Can you
assume respondent knows what recreational
means?
48
Example 2
Have you suffered from headaches or sickness
recently? (Please tick the appropriate box) Yes
? No ?
Question double barrelled What does recently
mean?
49
Example 3
Would you rather not use a medicated
shampoo? (Please tick the appropriate box) Yes
? No ?
Avoid double negatives
50
Example 4
If homeopathic remedies were available to buy in
large chemists, would you buy them? (Please tick
the appropriate box) Yes ? No ?
Do people behave as they say they will?
51
Example 5

• Do you go to the toilet a lot?

Ambiguity
52
Example 6

• How often do you get up at night to PU? (pass
  urine)
• Should IVDUs be treated in the community?

Avoid jargon/abbreviations/slang
53
Example 7

• Do you think that the food in the hotel made you
  sick?
• Did the hotel staff seem unhygenic to you?
• Do you agree that the hospital staff were close
  to exhaustion?

Avoid leading questions
54
Example 8

• What age are you?
• 16-20 ?
• 20-25 ?
• 25-30 ?
• 35-40 ?

Use simple language- old instead of age Avoid
not mutually exclusive options
55
Format of Questions

• Be precise
• Did you swim often in the pool?
• Yes