Diary nonresponse and quality in 2 surveys

1
Diary non-response and quality in 2 surveys
Delphine Roy, INSEE (France)
2
INTRODUCTION

• Diaries as a reliable data collection
  instrument
• Gershuny (2000)
• compares diary information with retrospective
  questionnaire memory effect, loss of information
• What type of data? Lots of information, spread
  in time
• itineraries (National Transportation Survey)
• spending (Household Budget Survey)
• daily activities (Time Use Survey)
• The point as regards quality is prevent loss of
  information due to memory
• must be filled BY RESPONDENT, all over a time
  period - day(s), week(s), month(s)

3
Issues
INTRODUCTION

• Diary quality
• IF
• Filled over time, not in retrospect
• Doesnt amplify non-response
• Compliance?
• Problem a written medium respondents left
  alone, autonomous. Difficult to control the data
  collection process as no one is present.
• What can we learn from previous surveys?

4
OUTLINE

• The data 2 INSEE surveys
• Diary non-response household level
• Diary non-response individual level
• Diary quality when and by whom are diaries
  actually filled?

5
1.1. The 1998 TUS
1. Two diary-based surveys

• Time Use Survey
• February 1998 to February 1999
• 12 000 addresses in the sample
• 1 diary for each household member ? 15
• what is recorded ACTIVITIES
• 10-minute time slots
•  natural language . Ex  washed the dishes 

6
The TUS diary left page
1. Two diary-based surveys
7
The TUS diary left page - example
1. Two diary-based surveys
8
The TUS diary right page - example
1. Two diary-based surveys
9
1.2. The 2006 HBS
1. Two diary-based surveys

• Household Budget Survey
• March 2005 to March 2006
• 20 000 addresses in sample
• 2 diaries for each household member ? 15
• 2 consecutive weeks
• What is recorded purchases
• Receipts can be stapled in the diary but need
   commenting 
• Purchases without receipt must be written down
  by the household

10
The HBS diary
1. Two diary-based surveys
11
The HBS diary - example
1. Two diary-based surveys
12
2. Diary non-response household level

• Several steps when dropout can occur
• HH refuses to answer the survey from the start
• HH doesnt complete Interview n1
• HH reaches the end of ITW n1 but refuses all
  diaries
• 1 diary is accepted but no ITW n2 can be done
• ITW n2 is done but no diary is filled when the
  interviewer returns, and HH wont fill them with
  the interviewer

13
2.1. Non-response figures
2. Diary non-response household level
14
2.2. Methodology
2. Diary non-response household level

• Simple logistic regression refusing the diary1
• Explanatory variables only what is collected at
  the beginning of ITW1 to have information on a
  maximal number of households, even dropouts
• 3 different definitions of the outcome, for HBS
• Refusing to take even 1 diary
• Eschewing ITW2
• Not having filled a single diary when the
  interviewer returns
• Then 2) 3), and 1) 2) 3) together. Is it
  the same thing? Same type of HH in 3 behaviors?

15
2.2. Methodology
2. Diary non-response household level

• Explanatory variables
• Occupational category of HH head
• Education level
• Type of household (couple or not, with or
  without children)
• Age of HH head
• Region of residence
• Survey Wave ( time of the year)
• Number of employed persons in the HH
• Subsidized housing
• Rural / urban size of the urban area
• for TUS HH income (collected at the beginning
  of survey)
• Several specifications were used with / without
  income OLS / logistic. Same results.

16
2.3. Results on refusing to take diary
2. Diary non-response household level
17
2.3. Results HBS details
2. Diary non-response household level
Is there a difference between refusing a diary
and not giving it back?
18
Conclusion on households with no diary
2. Diary non-response household level

• Elderly HH are much harder to convince
• The self-employed are more reluctant, esp.
  farmers for TUS and business owners for HBS
• Regional (dis)inclination towards NIS surveys
• Refusing to take a diary is different from not
  doing ITW2.
• No ITW2 refusal (gt75) lack of availability
• Vacations
• Household w/out couple

19
3.1. TUS
3. Diary non-response individual level
What do we call  individual diary non response ?
Among the 8186 HH who reached the end of the 1st
ITW
(Little information was collected on the data
collection process itself during the 1998 TUS)
20
3. Diary non-response individual level
3.2. A specificity of the HBS
What do we call  individual diary non response ?

• Some individuals can write down their purchases
  on another household members diary a kind of
  proxy (the relevant info is at the household
  level).
• Mostly men and teenagers
• They are considered as having filled their
  diary, even if we might think the quality is not
  as good as if they had had their own (loss of
  information caused by proxy)

21
3. Diary non-response individual level
3.2. HBS more detailed information
What do we call  individual diary non response ?
Among the 10240 HH for whom we have the
individual variables
Is there a difference between the 2 categories
of non-respondents?
22
3. Diary non-response individual level
3.2. Refusal or avoidance?
Is there a difference between the 2 categories of
non-respondents?

• Assuming people with no diary did not want to
  fill it, who said so and who only didnt give it
  back?
• Refuse from the start to take a diary (N224)
• More skilled
• 15 to 25
• Parisians
• Above 55 and even more, above 75
• Do not give it back (N363)
• Self-employed
• Less skilled

23
3.3 No diary explanatory variables
3. Diary non-response individual level

• Explanatory variables
• Gender
• Age
• Lives in a couple
• Education
• Nationality (French by birth, became French, EU,
  non-EU)
• Occupation (6 categories Student Retired
  Other Non LF participant)
• Region of residence

24
3.4. No diary results
3. Diary non-response individual level
Are the 2 surveys diary non-response determined
by the same factors? Dependent variable no
diary 1
25
3. Diary non-response individual level
3.4. Cause of missing diary TUS

• 4 items, answered by the interviewer
• Away 21.5
• Unable to (handicapped, illiterate) 12
• Refusal 42
• Other reasons 10.5
• N/A 14

• Are more prone to refusal
• 35 to 55 (60)
• No diploma (37)
• Are away
• 15 to 25 (45)
• Men
• Are unable to fill the diary
• No diploma (31)
• Citizenship has little, if any influence
• Variable on handicap not available

26
3. Diary non-response individual level
3.4. Cause of missing diary HBS
Excluding the 30 of individuals who do not
receive a diary because they will report on
someone elses

• 5 items, answered by the interviewer at the end
  of ITW1
• Away 58
• Unable to (handicapped, illiterate) 18
• Refusal 12
• Other reasons12

• Are more prone to refusal
• Men (15)
• Self-employed (25)
• 45 to 55 (40)
• Near zero refusal among college graduates
• Are away
• 15 to 35
• College graduates
• Are unable to fill it
• No diploma (43)
• Handicapped (60)
• Elderly gt75 (68)

27
Conclusion on individuals lacking a diary
3. Diary non-response individual level

• Very similar explanatory variables in the 2
  surveys (except regions)
• Effect of educational level more important for
  TUS
• easily understandable since it is much more of a
  narrative exercise
• The possibility of a proxy (reporting on
  someone elses diary) avoids losing too many men
  in the HBS
• Teenagers living with their parents are very
  difficult to catch

28
4. Indicators of diary quality
4.1. When is the diary filled?
HBS
TUS
29
4. Indicators of diary quality
4.1. When is the diary filled?

• Women are better compliers
• HBS 85 every day VS 80 of men
• TUS 38 all along the day VS 31 of men
• The older the respondent, the later the diary is
  filled
• HBS
• TUS

30
4. Indicators of diary quality
4.1. When is the diary filled?

• Occupation clear differences
• Respondents not in the LF (housewives, retirees)
  fill the TUS diary more often all along the
  day, as requested.
• Farmers and other self-employed fill the diary
  later in the evening without notes, the day
  after (TUS) or just before or during ITW2
  (HBS)
• Tertiary sector wage earners (employees,
  managers, intermediate occupations) typically
  fill the diary in the evening, with or without
  notes (TUS), or every day (HBS)

31
4. Indicators of diary quality
4.1. When is the diary filled?

• Education no clear pattern except for those
  w/out diploma
• HBS 76 of no education filled diary
  everyday, 15 irregularly, 9 just before or
  during ITW2
• TUS 30 all along the day, 22 other
• Citizenship language barrier
• HBS 10 just before or during ITW2
• TUS 20 other for non-French citizens
• Again TUS more difficulties

32
4. Indicators of diary quality
4.1. When is the diary filled?

• Region Constants and variations
• TUS West, East and Center-East comply better
  ( 4pts), Mediterranean much less (-7pts)
• HBS East, West and Center comply better ( 2
  to 4 pts), South-West less, Mediterranean OK

33
4. Indicators of diary quality
4.2. By whom is the diary filled? (HBS only)
34
4. Indicators of diary quality
4.2. By whom is the diary filled? (HBS only)

• The interviewer fills the diary more often when
  the respondent is
• Living alone (6.6)
• Without degree (9)
• Handicapped (10)
• A farmer (12)
• Elderly (15)

35
4. Indicators of diary quality
4.2. By whom is the diary filled? (HBS only)

• Someone else in the HH fills the diary (not incl.
  those with a  proxy diary ) more often when the
  respondent is
• Without degree or less than high school (8)
• Living in the North of France (10)
• Self-employed or industrial worker (11, VS. 4
  of white-collar workers, mainly women)
• A man (11, VS. 3 of women)
• Under 25 (30) or between 25-35 (10)

36
CONCLUSION

• Quality is less good at the ends of age
  distribution
• Youth living with their parents
• Elderly
• Men escape data reporting much more than women
• We know less on how they spend their money and
  time
• Self-employed are more difficult than
  wage-earners
• Same as other surveys, they are a less
  well-known population
• Regional variations have some constants (East gt
  Paris and South-East) as well as much unexplained
  variability

37
CONCLUSION

• Much happens during the second ITW
• A lot of information is  rescued 
• Elderly
• Less skilled
• Handicapped
• Non-native speakers
• Importance of the interviewer as a mediator when
  the information must be written down
• Survey design must take that into account and
  allow interviewer to spend time on those
  respondents