Course on Designing Survey Questions and Questionnaires

1
Course on Designing Survey Questions and
Questionnaires

• Roger Thomas
• Balkans Summer School 2006 Questionnaire Design 1

2
Questions

• The main tool used to collect survey data is
  questions.
• Survey questions must be designed.
• The designer has to think about -
• conceptual and operational definitions
• measurement
• communication with respondents
• response tasks and response burden

3
Questionnaires

• Questionnaires are not just lists of questions.
• The designer needs to think about -
• user-friendliness
• logical data structure
• good layout and presentation
• efficient document production.

4
Impact of computer-assisted data collection

• Computer-assisted survey information collection
  (CASIC) presents new demands and opportunities.
• These affect questionnaire construction more than
  the design of questions
• On this course we will refer to CASIC, but will
  not discuss it in technical detail.

5
Skills needed in designing questions and
questionnaires

• To design effective questions and questionnaires
  a person needs skills in -
• applied statistics
• applied social psychology
• applied cognitive psychology
• document design and production

6
Relevance of statistics

• Survey questions are designed to measure defined
  social variables e.g.
• attributes
• behaviour
• attitudes
• The level of measurement determines what analyses
  can be applied.

7
Social psychology 1

• Social psychology is about how people interact
  with each other.
• In interactions they typically -
• adopt various social roles
• use language to communicate attitudes as well as
  facts
• subconsciously send and receive messages through
  body language.

8
Social psychology 2

• In surveys the central roles are
• researcher (the person responsible for survey and
  questionnaire design)
• survey interviewer (Int)
• survey respondent (R)
• Question-response in surveys is a social
  interaction, bound by implicit rules.

9
Social psychology 3

• In two-way social interactions each person tries
  to understand and control -
• the role they are expected to play
• the intentions of the other person
• how they appear to the other person
• Rs are not always sure what being a respondent
  requires.
• This needs to be made clear to them.

10
Cognitive psychology

• Cognitive psychology is about what goes on in
  peoples minds when they are asked a survey
  question, e.g. -
• understanding and interpreting question wording
• understanding what mental tasks are required
• performing these response tasks.

11
Cognitive psychology

• Cognitive psychology is also relevant to how Rs
  read, understand and interact with survey
  questionnaires.
• We do not have enough time to discuss in detail
  the physical design of documents (layout,
  typography etc).
• But this is an important practical topic and I
  have brought some examples.

12
About this course

• In this course we will cover
• methods of quantitative social research
  9
• survey measurement and response error )
• response tasks, respondent psychology, ) 50
  question wording
  )
• measuring behaviour and personal events 26
• different modes of data collection
  10
• construction of questionnaires
  27

13
About this course

• Listening continuously to someone speak in a
  foreign language is tiring. Please tell me if I -
• speak too fast, or not clearly enough
• use expressions that are unfamiliar
• am covering the material too fast.
• I am happy to answer your questions at any time.

14
My survey experience

• During my career I have specialised in
  quantitative social survey methods.
• This includes, for example
• survey sampling
• survey data collection
• designing questions and questionnaires
• data management and data analysis

15
My survey experience

• I have worked on many quantitative social
  surveys, on topics such as -
• economic activity and employment
• health and health-related behaviour
• family budgets and finance
• travel behaviour
• personal attitudes and values
• etc.