ComputerAssisted Qualitative Interviewing

1
Computer-Assisted Qualitative Interviewing

• Testing and Quality Assessment of CAPI and CATI
  Questionnaires in the Field
• Ger Snijkers,
• Edith de Leeuw, Doortje Hoezen, Isje Kuijpers,
• and others

2
Content

1. Computer-Assisted Qualitative Interviewing
2. Testing the POLS questionnaire 3. CAQI in the
Field 4. Results 5. Experiences with CAQI
6. Conclusion
3
1. Computer-Assisted Qualitative
Interviewing (CAQI)
At SN most household survey interviewing is
computer-assisted. CAQI Computer-Assisted
Testing of Questionnaires (lab and
field) Standardised CA-Interviewing
information on the Quality of the survey
data
4
2. Testing the POLS Questionnaire

• POLS Continuous Survey on Living Conditions
• 1) Defining the questionnaire ? two modules
  base and specific ? base module CAPI and
  CATI 15 minutes background and
  screening variables household composition,
  education level, time use, housing, health,
  Income, safety

5

• Testing the POLS questionnaire
• 2) Qualitative lab test ? the quality of the
  questionnaire QA-process, order and
  transition of topics ? the interviewing mode
  CAPI, CATI
• 3) Qualitative field test ? the data
  collection procedure mixed mode (CAPI-CATI,
  CATI-CAPI) ? the quality of some
  questions4) Large scale quantitative
  pilot5) Survey

6
Qualitative field test

• ? app. 650 households 350 CAPI-CATI 300
  CATI-CAPI
• ? 3 regions
• ? Interviewers, carefully selected and
  trained
• 21 CAPI
• 13 CATI
• ? CAQI questionnaire

7
3. CAQI in the Field

• Computerised questionnaire (Blaise), extended
  with
• 1) Additional interviewing tasks
• ? reaction coding (7 Qs) ? elaboration
  probes (2 Qs)
• ? respondent-debriefing Qs
• ? interviewer-debriefing Qs
• 2) Interviewer instructions ? how to apply the
  additional tasks ? how to handle the computer

8

Reaction coding

• gtgt INT PAY ATTENTION, enter reaction codes
  right after this question How does respondent
  react? ltlt
• What is your most time consuming activity, Apart
  from relaxation?
• Is that --gt
• 1 paid work
• 2 housework at home,
• 3 study or schooling,
• 4 volunteering or working with benefit, or
• 5 something else?

9

• OBSERVE REACTION CODE(S) Enter code(s) how
  respondent reacted to question.
• Respondent
• 1 asks for repetition of the question,2 asks
  for clarification of the question,3 interrupts
  question reading,4 expresses doubt between
  response items,5 gives answer that doesnt fit
  in the item list,6 changes answer7 gets
  irritated by question

10
Question - Answer Process

• 1) Interpretation stage ? repetition ?
  clarification ? interrupts reading2) retrieval
  and/or reporting stage ? changes answer
• 3) judgement and/or reporting stage ? doubt
  between items ? answer doesnt fit in item list
  

11
Elaboration probe

• gtgt INT PAY ATTENTION, elaboration probes will
  follow. ltlt
• How many sleeping, living, and working rooms does
  this house have?

12

• Did you include the kitchen, the bathroom and/or
  the cellar in the number of rooms?
• gtgt INT You are not allowed to change
  the answer to the former question. In case
  the respondents wants to change the number of
  rooms, you can enter that number right after
  this.
• 1 Yes
• 2 No

13

• OBSERVE In case the respondent wants to change
  the number of rooms, you may enter this number
  here. The number given before was Rooms. In
  case the respondent HIM/HERSELF does not
  indicate any changes, press ltENTERgt.

14
4. Results Mixed mode
CAPI-CATI ? Response rate 65 CAPI vs. 40
CATI ? Respondents no preference no
mode effects ? Interviewers CAPI better
refusal conversion one interview instead of
two switching modes
15
Results Quality assessment
Problems in questions as expected ? Reported
codes confirm lab results ? Codes are hard to
interpret. Additional information is
needed lab tests debriefings literature
16
5. Experiences with CAQI
Preparation phase ? Co-operation with fieldwork
staff recruiting and training of
interviewers standardised interview
procedures availability of CAI questionnaire
(Blaise) ? Preparation of CAQI
questionnaire carefully prepared and
tested time-consuming quickly changed
17

Interviewers ? Pay attention to survey
question, additional probing tasks, the
computer, the respondent. ? Well trained (to
do a good job) applying all interviewing
tasks the goals of the test program
handling the computer and the
computerised questionnaire
18

Respondent ? Computer gives professional
status ? React spontaneously ? Listened to
opinion
19

Characteristics of CAQI in the field ? Greater
scope of qualitative research in the
field larger number of respondents ?
Integration of Q-research and CAI structured
Q-interviews reflecting CAI-standard fieldwork
conditions focussing on respondents
reactions ? Computer controls the flow of the
interview appropriate probing what
technique when and how every single action on
a single screen not too much text on each
screen
20
6. Conclusion
CAQI is ? an interviewer-friendly and efficient
way to administer and test a questionnaire
in one and the same interview under realistic
fieldwork conditions. ? a quality assessment
tool for evaluating survey questions and
fieldwork procedures, to get insight in the
quality of survey data.