Title: The problem of identifying persons with disabilities
1The problem of identifying persons with
disabilities the importance of questionnaire
design
- Angela Me, Chief Social and Demographic
Statistics Section
2Challenge
- How to measure a wider experience of disability
through a limited number of questions?
3Design an instrument to identify the defined
population with disability
- The difficult part is
- To logically convert/translate objectives into
measurement instruments and to link definitions
with questions
4Design an instrument to identify the defined
population with disabilities - a Census Example
- Definition
- Any restriction or lack (resulting from an
impairment) of ability to perform an activity in
the manner or within the range considered normal
for a human being - Question
- Is there anybody in the household who is disabled?
5Definition-measurement instrument
BFS limitations
Paralyzed
..
Deaf
Activity limitations
..
6Design an instrument to identify the defined
population with disabilities - a Census Example
- Despite a definition based on activity
limitations, the questions identified only
persons with most severe impairments - 2.5
7From theory to practice
- Estimates of prevalence of disability are highly
sensitive to the measures used
8Measurement Issues
- Design of the study
- Method of data collection
- Question design (wording, place, length, )
- Interview process
- interviewer effect
- Respondent effect
- Socio-cultural Determinants
9US Survey Example
- The following questions and results were obtained
in an American survey
'Yes'
Have you ever heard the word AFROHELIA? (no such
word!)
8
Have you ever heard of the famous writer, John
Woodson? (no such writer!) Have you ever heard
of the Midwestern Life Magazine? (no such
magazine!) Do you recall that, as a good citizen
you voted last December in the special election
for your state representative? (no election!)
16 25
33
Have you ever heard of the Taft-Pepper Bill
concerning veteran's housing (no such bill!)
53
10Example Australian Survey
- Average number of sex partners reported
- By women who were watched as they filled in their
survey answers 2.6 - By women who knew they were completely anonymous
3.4 - By women who thought they were attached to a lie
detector 4.4
Sydney Morning Herald, August 31, 2003
11Wording
- The most detailed disability survey, using a
carefully designed and relatively complete set of
questions covering a wide range of topics, is
limited when the initial questions used to
identify the persons with disability is poorly
designed
12Developing instruments to identify persons with
disabilities
- Disability is a dynamic complex related to
- Individual attributes
- Environment
- Time
- Two persons with the same impairment may have a
different perception of disability
13Issues that we need to consider
- Particular attention is needed to measure
disability through an interview process - People may be unwilling to talk about their
problems - Difficulty in defining what is meant by
disability and its various aspects - Stigma
14Issues that we need to consider
- In an interview process
- Easier to measure activity limitations
(day-to-day activities) and participation
15Developing instruments to identify the complexity
of disability
- Requirement
- Multiple questions to set context, clarify
terminology, define multiple domains - Resource availability
- Short questions
16Developing instruments to identify the complexity
of disability
- Long instruments/modules
- High number of questions, more opportunities to
capture the different dimensions, intensity - Short instrument/modules
- 1-5 questions to identify persons with
disabilities - Careful design of the question(s) to make sure
that all persons with disabilities that we want
to identify can indeed be properly identified
17What defines a good instrument
- The instrument measures the concept it is
supposed to measure (Validity-Accuracy) - Repeated measurements of the same instruments
give the same results (Reliability-Precision)
18Design an instrument to identify the target
population- A Census Example
- Definition
- Any restriction or lack (resulting from an
impairment) of ability to perform an activity in
the manner or within the range considered normal
for a human being - Question
- Is there anybody in the household who is disabled?
19An example U.K. Census 1991
- Do you have any long-standing illness, health
problem or handicap which limits your daily
activities or the work you can do? Include
problems which are due to old age
20Conditions that effect the output of a question
- Wording
- Context
- Self/Proxy
- Response categories
- Mode of data collection
- Method of data collection
- Overall survey topics
- Survey sponsor
21Wording what does affect comprehension?
- Ambiguous syntax
- Complicated syntax
- Unfamiliar terms
- Vague concepts
- Assumptions about respondents knowledge
22Wording
- Language
- Clear
- Unambiguous
- Simple
- Terms such as long-term, disabilities, handicaps
are viewed as extremely negative and tend to
underreport disabilities (Langlois, 2001)
232000 US Census
- Because of a physical, mental, or emotional
condition lasting 6 months or more, does this
person have any difficulty in doing any of the
following activities - d. Working at a job or business?
- Multiple health domains (explicit enumeration)
- Duration
- Difficulty
- capacity
- participation
- Working
24Question components for a short question
- Preamble
- Health condition (as cause)
- Duration (long/short term disability?)
- ICF domain
- Functioning
- Activities
- Participation
25Question components for a short question
- Keep it relevant and valid but SIMPLE
- If possible split the different components of the
measure into different questions
26Question components for a short question
- Introduction make the respondent think about the
outputs of an health condition and set the
duration (conditions that last for 3 months, 6
months, 12 months, ) -
- Depending on the domain we want to identify
-
- Do you have difficulties walking?
- Do you have difficulties concentrating?
- Do you have difficulties participating in daily
activities - ..
27Question components for a short question
- Questions based on activities and participation
rather than impairments provide a broader view of
disability and they facilitate the
identification of persons with disabilities by
providing a more simple and natural language
28Response process judgment and response
formulation
- Evaluation of retrieved information
- Perception of accuracy
- Motivation
- Evaluation of response options
- Communication of response
29Response categories
- Response categories set the context of the
question - Avoid to force the respondent to identify
him/herself with a socially-defined label
(stigma) - Scale Response instead of a dichotomy
- None/A little/A lot
- Yes, sometimes/Yes, often/No
30Response categories
- It has been proven that scaled responses improve
the respondents ability to report having
disabilities (Statistics Canada, Austrian Bureau
of Statistics, Research in the USA) - If the respondent can not choose among multiple
dimensions, he/she is likely to misreport his/her
disability status - Disability is not a yes/no phenomenon but rather
a status that varies on a continuum in terms of
intensity and time
31Mode of data collection
- Self reported or Proxy?
- Avoid if possible proxy responses
- The disability process relates to the
individuals experience and can be accurately
described only by the individual him/her self
32Type of question
- Avoid household-based questions (Is there anybody
in the household who has difficulties walking?) - Use person-based question (Do you have
difficulties walking?) - It has been proven that person-based questions
identify more persons with functional limitations
(USA, UN)
33Context
- Place of the question in the questionnaire
- Example disability and economic characteristics
questions - Introduction to the question