BSc Health Sciences Questionnaire design

1
BSc Health SciencesQuestionnaire design

• Fiona Marshall/Steve Brindle/Colin Simpson

2
Session Aim

• Provide an overview of the development and use of
  a questionnaire in sports studies and health
  research.
• Stress importance of preparing/piloting
  questionnaires

3
Objectives

• Outline issues to consider when designing a
  questionnaire
• Describe the process of planning a questionnaire
  survey
• Describe the main types of question formats and
  the occasions appropriate to their use.
• Highlight the key characteristics of a good (or
  bad!!) questionnaire

4
Why a questionnaire survey

• Questionnaire is a data collection tool designed
  to address your study aims and objectives
• structured and organised manner
• Target particular groups
• Not an easy option - difficult to construct a
  good questionnaire

5
Stages of Design

• Decide what information you want to collect
  (research question(s))
• Select a list of items to be collected
• Design individual questions
• Decide on wording
• Arrange layout
• Coding to help with your analyses
• Prepare first draft
• Pilot
• Conduct survey

6

• Strengths weaknesses

7
Strengths of questionnaires

• Low cost in time and money (compared to
  qualitative)
• Easy to get information from people quickly
• Respondents can complete the questionnaire when
  it suits them
• Analysis of answers to closed questions is
  straight forward pre-coded

8
Strengths

• Low pressure for an immediate response
• Respondents anonymity
• Lack of interviewer bias - characteristics of the
  interviewer will not affect participants
  response?
• Standardisation of questions - everyone gets the
  same questions
• Can provide data for testing a hypothesis

9
Limitations of questionnaires

• Problems of data quality (completeness and
  accuracy)
• Typically low response rate unless captive sample
  
• Problems of motivating respondents
• The need for concise and relatively simple
  questions
• Misunderstandings cannot be corrected

10
Limitations

• Seeks information just by asking questions (can
  we believe what respondents say?)
• Assumes respondents have answers available in an
  organised fashion
• Lack of control over the context questions are
  answered

11
Limitations

• Question wording can have a major effect on
  answers
• Respondent literacy problems
• People talk more easily than they write
• Impossible to check the honesty of answers
• Respondent uncertainty as to what happens to the
  data

12
Deciding information to collect

• Focus on the aims and objectives of the study
• Previous research - Has it been done before?
• Literature review - there may a validated
  instrument that you can use Euroquol quality of
  life scale generic indicator of health status,
  DEPCAT rating zones by postcode
• Contact researchers in the field
• Focus groups or loosely structures interviews to
  identify relevant lines of enquiry

13
Deciding information to collect

• Operationalisation of a concept
• Data are collected using pre-determined measures
  of the concepts of interest
• The operationalisation of a concept is crucial to
  the issues of validity. Is the question
  measuring what we think it is?
• There must be a link between theory and data
  collection.

14
Operationalising your research topic

• Create and focus the main research questions to
  identify theoretical and substantive rivals
• identify methodological rivals
• operationalise research concepts
• Extablish a connection between the concepts and
  the concrete reality in which you live
• measure indicators with research methods

15
Concept Operationalisation Health

• Health is a multi-dimensional concept, so how can
  we measure it simply and concisely? In other
  words, what questions will indicate the
  respondents health?
• Census do you have any long term illness? or
  Over the last twelve months would you say your
  health on the whole has been? (Good/Fairly
  good/Not good)
• General Health Questionnaire (GHQ) a scale based
  on an aggregation of 30 items, including
  happiness and self confidence as indicators of
  health

16
Things to watch out forOperationalisation
Education

• Education is usually measured by the highest
  qualification attained by the respondent. But
  does age matter here?

17
Operationalisation Social Class

• Many ways to measure social class occupation,
  income, house ownership, choice of newspaper, etc
  etc
• but there is another issue here is social class
  tied to household, do you inherit it from your
  parents or...

18
Operationalisation

• Key take home message
• Some concepts require much more thought when
  considering what their indicators are e.g.
  socio-economic status/deprivation.

19
Questionnaire design - Select key items

• You cant include everything so - compile a list
  of items for inclusion
• Which are most relevant?
• What can you leave out?
• Think with the end in mind- i.e. how you will
  present the results for that particular question.
• E.g. Categorical table or pie chart
• Think about how would you present your results
  stratification - differences in responses of
  males vs females, or older vs younger respondents

20
Topic areas

• Questionnaires tend to explore topics that fall
  into three main categories
• Facts e.g. How old are you
• Opinions, beliefs, attitudes e.g. what do you
  think of
• Behaviour e.g. what people do, or at least what
  people say they do!!)

21
Break- out Critique questionnaires

• Read through qustionnaire
• groups of 4
• Critique questionnaire
• Identify different types of questions e.g. closed
• Good points
• Bad points

22
Designing questions

• Types of questions
• Open-ended What is your favourite restaurant
  _______
• Closed What is your Favourite restaurant Pizza
  place_ Top tattie_ Central Ref_
• Mix of open and closed
• Rating scales
• Please describe how you felt about your last
  meal at Pizza placeUnsatisfied (1) Somewhat
  Satisfied (2) Satisfied (3) Very Satisfied (4)
  Extremely Satisfied (5)
• Ranking
• Please rank the following Pizza place menu items
  in order of preference (starting with 1 for your
  favourite activity).___Garlic bread ___Hot and
  spicy pizza __Apple pie ___Tomato soup

23
Good questionnaire design and layout

• Give respondents reasons why their information is
  valued and how long it is likely to take to fill
  in - could also go in covering letter
• Thank people for their time and help
• Ethical considerations - what about the visually
  impaired, for instance?

24
Good questionnaire design and layout

• Cover neutral questions first and sensitive
  issues later on - demographic questions can be
  used for this purpose (I would swap Sections A
  and B)
• Adopt a funnel approach - from general to
  specific
• Use filter questions - respondents can be routed
  round irrelevant questions. If no, go to question
  4 etc.
• Try different layouts - should be clear and
  logical, and flow like a normal conversation

25
Bad questionaire design

• Dont use complicated language -Use simple
  language (but not too simple!)
• avoid ambiguity, and vague words (fairly,
  generally, often, etc) Question 6a
• Dont use abbreviations RAST,SPT (section A,
  question 4,)
• Avoid long questions (section A, question 1a)

26
Example bad questions

• Double barrelled questions
• "Does your company have pension and health
  insurance benefits? (Section A, question 1a)
• avoid questions which require a long memory
  what happened in the last 50 years (Section A,
  question 7b)

27
Example bad questions

• Leading questions Question 10.
• Now there is a Scottish Parliament and a Welsh
  assembly the next Prime Minister ought to come
  from England - 40 agreed, 53 disagreed.

28
Example bad questions

• Negatives and double-negatives (particularly when
  answer is agree/disagree) Question 8
• Fox hunting should not be abolished? Vs Fox
  hunting should continue?

29
Example bad questions Bizarre questions

• Question 6d
• 15. Good practice guides on
• burials
• noisy parties
• ritual slaughtering of animals
• communication
• positive action
• consumer protection
• hotels and bed and breakfast
• Other
• (Example question Race and Environmental
  Health, Local Authority Questionnaire, Equal
  Opportunities policy)

30
Other considerations Coding your questionnaire

• Analyse by hand or with statistical package SPSS,
  SAS, Access
• Decide on format in advance get template ready
• List all possible answers for closed
  questions/rating scales
• Include an option for missing
• Assign each possible response a numerical value

31
First Draft

• Think of presentation as well as content - are
  boxes aligned and equal question 10 wonky
• Circulate your first draft to colleagues and
  other researchers - ask for criticism and
  comments

32

• http//www.analytictech.com/mb313/principl.htm
• Pilot, pilot, pilot

33
Pilot questionnaire

• Essential to pilot questionnaire
• Select a sub sample to test questionnaire -
  approx 10 main sample, (should not use pilot
  group again)
• Try to use similar group to main study
• Enter results in database - test categories for
  coding
• Alter questions accordingly -then conduct main
  survey

34
Practical Issues

• Plan ahead
• Calculate and order stationary - no. of pages,
  cover letters, pre-paid envelopes - extra for
  reminders
• Photocopying costs
• Time scale - send reminders 2/3 weeks after first
  batch
• Time to enter results in database (are you using
  automatic scanning equipment)- time for analysis

35
Practical issues

• Is it anonymous? How will you know who has
  responded?
• Assign an identifying subject number to each
  participant (unique identifier) have on
  questionnaire
• Important for reminders and also if you want to
  repeat questionnaire at a later stage (follow-up)

36
Cover letter

• First impressions count
• Use organisations headed paper
• Short and concise
• Contact name and address for further information

37
Conclusions

• Think about the research process from start to
  finish, bearing in mind the time scale.
• Always pilot your questionnaire, it prevents
  problems later on.
• Questionnaire should be concise, direct and easy
  to understand.

38
Conclusions

• Does your questionnaire measure what it is
  attempting to measure?

39
References

• Gillingham, B. 2000. Developing a Questionnaire.
  London Continuum
• Bowling, A. 2002. Research Methods in Health. 2nd
  Edition. Buckingham Open University Press.
• McDowell, I and Newell, C. 1996. Measuring
  Health A guide to rating scales and
  questionnaires. 2nd edition. OxfordOxford
  University Pres.