The problem of identifying persons with disabilities

1
The problem of identifying persons with
disabilities the importance of questionnaire
design

• Angela Me, Chief Social and Demographic
  Statistics Section

2
Challenge

• How to measure a wider experience of disability
  through a limited number of questions?

3
Design an instrument to identify the defined
population with disability

• The difficult part is
• To logically convert/translate objectives into
  measurement instruments and to link definitions
  with questions

4
Design an instrument to identify the defined
population with disabilities - a Census Example

• Definition
• Any restriction or lack (resulting from an
  impairment) of ability to perform an activity in
  the manner or within the range considered normal
  for a human being
• Question
• Is there anybody in the household who is disabled?

5
Definition-measurement instrument
BFS limitations
Paralyzed
..
Deaf
Activity limitations
..
6
Design an instrument to identify the defined
population with disabilities - a Census Example

• Despite a definition based on activity
  limitations, the questions identified only
  persons with most severe impairments
• 2.5

7
From theory to practice

• Estimates of prevalence of disability are highly
  sensitive to the measures used

8
Measurement Issues

• Design of the study
• Method of data collection
• Question design (wording, place, length, )
• Interview process
• interviewer effect
• Respondent effect
• Socio-cultural Determinants

9
US Survey Example

• The following questions and results were obtained
  in an American survey

'Yes'
Have you ever heard the word AFROHELIA? (no such
word!)
8
Have you ever heard of the famous writer, John
Woodson? (no such writer!) Have you ever heard
of the Midwestern Life Magazine? (no such
magazine!) Do you recall that, as a good citizen
you voted last December in the special election
for your state representative? (no election!)
16 25
33
Have you ever heard of the Taft-Pepper Bill
concerning veteran's housing (no such bill!)
53
10
Example Australian Survey

• Average number of sex partners reported
• By women who were watched as they filled in their
  survey answers 2.6
• By women who knew they were completely anonymous
  3.4
• By women who thought they were attached to a lie
  detector 4.4

Sydney Morning Herald, August 31, 2003
11
Wording

• The most detailed disability survey, using a
  carefully designed and relatively complete set of
  questions covering a wide range of topics, is
  limited when the initial questions used to
  identify the persons with disability is poorly
  designed

12
Developing instruments to identify persons with
disabilities

• Disability is a dynamic complex related to
• Individual attributes
• Environment
• Time
• Two persons with the same impairment may have a
  different perception of disability

13
Issues that we need to consider

• Particular attention is needed to measure
  disability through an interview process
• People may be unwilling to talk about their
  problems
• Difficulty in defining what is meant by
  disability and its various aspects
• Stigma

14
Issues that we need to consider

• In an interview process
• Easier to measure activity limitations
  (day-to-day activities) and participation

15
Developing instruments to identify the complexity
of disability

• Requirement
• Multiple questions to set context, clarify
  terminology, define multiple domains
• Resource availability
• Short questions

16
Developing instruments to identify the complexity
of disability

• Long instruments/modules
• High number of questions, more opportunities to
  capture the different dimensions, intensity
• Short instrument/modules
• 1-5 questions to identify persons with
  disabilities
• Careful design of the question(s) to make sure
  that all persons with disabilities that we want
  to identify can indeed be properly identified

17
What defines a good instrument

• The instrument measures the concept it is
  supposed to measure (Validity-Accuracy)
• Repeated measurements of the same instruments
  give the same results (Reliability-Precision)

18
Design an instrument to identify the target
population- A Census Example

• Definition
• Any restriction or lack (resulting from an
  impairment) of ability to perform an activity in
  the manner or within the range considered normal
  for a human being
• Question
• Is there anybody in the household who is disabled?

19
An example U.K. Census 1991

• Do you have any long-standing illness, health
  problem or handicap which limits your daily
  activities or the work you can do? Include
  problems which are due to old age

20
Conditions that effect the output of a question

• Wording
• Context
• Self/Proxy
• Response categories
• Mode of data collection
• Method of data collection
• Overall survey topics
• Survey sponsor

21
Wording what does affect comprehension?

• Ambiguous syntax
• Complicated syntax
• Unfamiliar terms
• Vague concepts
• Assumptions about respondents knowledge

22
Wording

• Language
• Clear
• Unambiguous
• Simple
• Terms such as long-term, disabilities, handicaps
  are viewed as extremely negative and tend to
  underreport disabilities (Langlois, 2001)

23
2000 US Census

• Because of a physical, mental, or emotional
  condition lasting 6 months or more, does this
  person have any difficulty in doing any of the
  following activities
• d. Working at a job or business?

• Multiple health domains (explicit enumeration)
• Duration
• Difficulty
• capacity
• participation
• Working

24
Question components for a short question

• Preamble
• Health condition (as cause)
• Duration (long/short term disability?)
• ICF domain
• Functioning
• Activities
• Participation

25
Question components for a short question

• Keep it relevant and valid but SIMPLE
• If possible split the different components of the
  measure into different questions

26
Question components for a short question

• Introduction make the respondent think about the
  outputs of an health condition and set the
  duration (conditions that last for 3 months, 6
  months, 12 months, )
• Depending on the domain we want to identify
• Do you have difficulties walking?
• Do you have difficulties concentrating?
• Do you have difficulties participating in daily
  activities
• ..

27
Question components for a short question

• Questions based on activities and participation
  rather than impairments provide a broader view of
  disability and they facilitate the
  identification of persons with disabilities by
  providing a more simple and natural language

28
Response process judgment and response
formulation

• Evaluation of retrieved information
• Perception of accuracy
• Motivation
• Evaluation of response options
• Communication of response

29
Response categories

• Response categories set the context of the
  question
• Avoid to force the respondent to identify
  him/herself with a socially-defined label
  (stigma)
• Scale Response instead of a dichotomy
• None/A little/A lot
• Yes, sometimes/Yes, often/No

30
Response categories

• It has been proven that scaled responses improve
  the respondents ability to report having
  disabilities (Statistics Canada, Austrian Bureau
  of Statistics, Research in the USA)
• If the respondent can not choose among multiple
  dimensions, he/she is likely to misreport his/her
  disability status
• Disability is not a yes/no phenomenon but rather
  a status that varies on a continuum in terms of
  intensity and time

31
Mode of data collection

• Self reported or Proxy?
• Avoid if possible proxy responses
• The disability process relates to the
  individuals experience and can be accurately
  described only by the individual him/her self

32
Type of question

• Avoid household-based questions (Is there anybody
  in the household who has difficulties walking?)
• Use person-based question (Do you have
  difficulties walking?)
• It has been proven that person-based questions
  identify more persons with functional limitations
  (USA, UN)

33
Context

• Place of the question in the questionnaire
• Example disability and economic characteristics
  questions
• Introduction to the question