Advanced Psychometric Workshop on Methods for CrossCultural Cognitive Assessment

1
Advanced Psychometric Workshop on Methods for
Cross-Cultural Cognitive Assessment

• Friday Harbor Laboratory
• University of Washington
• August 22-26, 2005

2
Background

• Neuropsychological tests are widely used to
  detect cognitive changes associated with brain
  disease and injury
• Test results are used in diagnosing brain disease
  and injury
• Interpretation of test results can have important
  clinical implications
• Much of the research in this area has been done
  with Caucasians

3
Background

• Cognitive impairment associated with diseases of
  aging is a major clinical problem
• Alzheimers disease, cerebrovascular brain
  injury, other degenerative dementia, etc.
• Neuropsychological tests are widely used in
  clinical evaluation
• Much of the research in this area has been done
  with Caucasians

4
Background

• Well established finding that ethnic minorities
  have lower average performance on
  neuropsychological tests
• Cutpoints based upon Caucasian samples result in
  high false positive rates in minority populations
• Raises important question about measurement bias
• How do we use neuropsychological test results to
  measure cognitive changes of dementing illnesses
  in ethnically diverse groups

5
Psychometric Theory - Measurement of Ability
Domain
6
Determinants of Cognitive Test Performance
7
Determinants of Cognitive Test Performance
8
Determinants of Cognitive Test Performance
9
Determinants of Cognitive Test Performance
10
Measurement Bias

• Differential validity across groups
• Validity is purpose specific
• Therefore, bias is purpose specific
• In context of diseases of aging, primary
  validation is ability to detect and monitor
  disease

11
Sources of Measurement Non-Invariance (Bias)
Across Ethnic Groups

• Item Level
• Differential Item Function (DIF)
• Items have different relationship to ability in
  different groups
• Scale Level
• Differential relationship of ability to disease
  in different groups

12
Determinants of Cognitive Test Performance
Scale Level Bias
Item Level Bias
13
Differential Item Function (DIF)-Free Test
14
Test with Differential Item Function (DIF)
15
Determinants of Cognitive Function
16
Determinants of Cognitive Function
17
Determinants of Cognitive Function
18
Determinants of Cognitive Function
19
Purpose of Workshop

• Develop psychometric methods and skills for
  cognitive assessment in culturally diverse
  populations
• Presentations of conceptual and methodological
  issues and approaches
• Share experience and skills
• Apply methods to existing dataset to address
  important measurement issues

20
Studies/Manuscripts

• Identification of DIF and assessment of impact on
  scale level scores
• External validation of neuropsychological test
  results in a multiethnic sample
• Construct validation of reading tests as measures
  of premorbid ability
• Dimensional structure of tests of executive
  function in multiethnic sample

21
Dataset

• Spanish and English Neuropsychological Assessment
  Scales (SENAS)
• Item and scale level data
• Demographic and linguistic variables
• Clinical diagnosis
• Quantitative MRI

22
Schedule

• Monday Evening
• Introduction and Overview
• Overview of SENAS project

23
Schedule

• Tuesday AM
• Neuropsychology, neuroimaging, and dementia
• Cultural and demographic influences on test
  scores
• Tuesday Afternoon
• Free
• Tuesday Evening
• Logistic regression approaches to DIF
• IRT approaches to DIF

24
Schedule

• Wednesday AM
• Covariance structure/ Mplus applications to
  measurement
• Description of SENAS dataset
• Overview of manuscripts and workgroups
• Wednesday PM
• Workgroups
• Wednesday Evening
• Report of Workgroups

25
Thursday Schedule

• Morning
• Open didactic/discussion slot
• Workgroups
• Afternoon
• Workgroups
• Evening
• Report of workgroups

26
Friday Schedule

• Morning
• Open didactic/discussion slot
• Workgroups
• Afternoon
• Workgroups
• Evening
• Report of workgroups