Research Methodology Designing a questionnaire Asking the right questions

1
Research MethodologyDesigning a questionnaire -
Asking the right questions

• June 11, 2008

2
Asking the right questions

• What is the purpose of the study?
• List the major sections (topics) and sub-sections
• How long should the questionnaire be?
• Who are your target respondents?
• Male, female decision-maker head of the
  household, main earning member
• Age groups (children, adolescents, women of
  reproductive age)
• Farmers/ SHG members/ small business owners/
  migrants, etc.
• Rural/urban
• Target respondents for each section could vary
• Reference period for sections/questions
• Now/7 days/30 days/6 months/1 year

3
Asking the right questions

• Ordering the sections
• It is important for the surveyors to try and
  build a relationship in the short time. Start
  them off with easier questions (household
  characteristics, roster), move on to more
  personal ones (income, savings, reproductive
  behavior etc.)
• Could introduce games (risk-seeking behavior
  time discounting) in between.
• Would responses differ? Especially for knowledge
  and behavior
• if you ask about willingness to pay for health
  insurance after asking questions on health events
  and expenditures.
• if you ask a question about political preferences
  after asking about socio-economic status

4
Asking the right questions

• What information do you need (sub-sections)
• Household composition and characteristics (age,
  education, occupation, etc.)
• Indicators of household wealth living
  conditions, food security, house structures,
  access to facilities, etc.
• Financial and economic situation - income,
  savings, indebtedness, consumption expenditure,
  sale and purchase of assets, etc.

5
Writing a good introduction

• Decide whether you need to state the purpose of
  the study to the respondent
• think about the trade-off ethical issues,
  influence
• medical (drug) trials require full disclosure
• Confidentiality
• important to explain that there is no link
  between participating in the survey and access to
  services
• Other information to include
• does your IRB require a signed consent form? for
  e.g. surveys that involve invasive health tests
• provide information on how long the interview
  will take. if you will compensate them for their
  time state this at the beginning
• explain they are free to not answer a question or
  stop at any point
• give them contact person information in case they
  have questions later

6
How to start

• Using existing surveys addressing similar
  questions
• Consumption section National Sample Survey
  Organization (NSSO)
• Health Rural Family Health Survey (RFHS)
• Education Annual Survey for Education Research
  (ASER)
• Look-up existing survey formats
• World Bank LSMS modules
• Published datasets/ survey instruments

7
Devil is in the detailsGRIDS
Grids for questions that have to be asked
repeatedly for multiple persons, events (e.g.
health events), items (e.g. loans)
8
Gridsthe right side
9
More grids
Grids for questions that have to be asked
repeatedly for multiple persons, events (e.g.
health events), items (e.g. loans)
10
Skip patterns
Sometimes, according to the response to a
question, the next question does not need to be
asked and the respondent needs to jump to another
question.
11
Options
Options can be single or multiple. Indicate
CIRCLE ALL THAT APPLY if this is a multiple
option question.
12
Optionsthings to remember

• Close ended questions must have ALL options
  coded
• Use Others specify to capture factors not
  coded also use one code (777) and code others
  individually after data collection (e.g. Other
  occupations)
• Standardized codes for Do not know (999) and
  Will not Say (888) or (998)
• Use - to distinguish from other numerical data
  (-999), if needed
• Use code sheet, where necessary usually for
  industry codes

13
Interviewer checkpoints
These are instructions to the interviewers.
SHOULD NOT ASK RESPONDENT. Mostly to verify
information / clarifications, etc. Distinguish
ICs by printing, making bold, underlining the
instruction Eg. question was asked earlier and
details follow in another section..
14
Detailing

• Definitions
• who is a household member?
• what is a business?
• who is a migrant?
• what is a health event? etc.
• Dont bias your respondent
• What did you use this loan for? instead of where
  did you invest this loan? (investment) or what
  did you buy with this loan? (consumption)
• Could split questions Have you ever washed your
  mosquito net this year? Next question if yes,
  how many times? instead of how many times do you
  wash your mosquito net in a year? the correct
  answer will be considered to be more than 0

15
More detailing

• Cross-check important questions
• Sources of income assets land ownership health
  events and health expenditure household
  consumption at different points in the survey
• Unit of data and components
• food consumption in Rs or Kg.
• loans, asset ownership is usually recorded at
  household level
• Adapt local terms
• loan could mean ones from only formal financial
  institutions
• Self-reporting v.s . other means of evaluating
  data
• business income capture inputs and sales
• perceived resale value of assets instead of cost
  price (actual expenditure)

16
Things to remember

• Questions should be clear and easy to read /
  understand.
• Avoid complicated and long questions. Break it up
  into sub questions / grids
• Specify where to probe -- dont always take do
  not know for an answer.

17
Things to remember
Increases length of the survey but is easier and
faster to administer
18
A few tricky questions

• Age
• Marital status
• Interest rate on loans
• Income of the household (should you count
  transfers from relatives, government, migrant
  household members, etc,)
• Usage of loans (do you want to capture the stated
  purpose or actual usage? do you want to capture
  all uses of the loan?)

19
Pilot testing

• Pilot your questionnaire always!
• Several reasons to refine questions / options
  and for surveyors to familiarize themselves with
  the survey, estimate productivity, check
  translation, remove ambiguities in questions,
  etc.
• In case of lengthy questionnaires time every
  section will help you determine which
  section/question takes-up a lot of time and if
  you want to slash those questions/sections

20
Pilot testing

• Pilot more if it is a new topic (there are more
  examples of what works and what doesnt in
  surveys on socio-economic levels of household and
  less about financial behavior)
• Pilot individual sections more if needed
• As a general rule 20-30 households for a large
  survey. Smaller surveys, 8-10 households. Do not
  pilot in your study area!

21
Translation

• Always back-translate!
• Outsource to 2 people1 to translate from English
  to local language and another back translate to
  English. Compare
• Contextualize refine during pilots. Find local
  terms for quantities (units acre, bhigas,
  etc.)
• Local language keep it informal meaning should
  be clear (so avoid using language experts!)

22
Training

• Instruction manuals are very important
• Each question clear instructions how to ask,
  skips, type of data to record, meaning/definitions
  
• Training
• Classroom
• Mock training
• Field testing
• Piloting
• Retrain during the survey if necessary