Criteria for comparability of Cross cultural research

1

• Criteria for comparability of Cross cultural
  research
• by
• Willem E.Saris

2
Cross Cultural Comparisons

• Means and relationships
• of single questions
• of composite scores
• of latent constructs

3
Basic response model
Fjkm
Fokm
lijm
liom
Rijkm
aijm
eijkm
4
Basic model and notation

• The basic measurement model is
• Rijkm aijm lijm Fjkm liom Fokm eijkm
• With E(eijk)0
• E(Rijm) aijm lijm E(Fjkm) liom E(Fokm )
• The scale of the latent variable is fixed by
  assuming for at least one indicator aijm 0 ,
  and lijm1

5
Comparison of means

• Criteria for comparison of means
• of single questions
• of composite scores
• of latent constructs

6
Means of single questions

• In case of a single question we get
• E(Rijm) E(Fjkm) liom E(Fokm )
• assuming aijm 0 , and lijm 1
• E(Rij1) can be compared with E(Rij2)
• If also liom0 or E(Fiokm)0
• But how do we know that
• The scales are used in the same way
• There are no systematic errors in one or both
  countries
• Comparison of means of single questions is
  questionable

7
The Multiple indicators model
Fjm
Fom
R1jm
R2jm
R3jm
e1jm
e2jm
e3jm
8
Notation of composite scores

• The general measurement model can be formulated
  as follows
• rm am Lm fm em
• with (em) 0, E(fm.em ) 0
• and E(em.em) is diagonal

9
The path model for composite scores
Fjm
Fom
R1jm
R1jm
R1jm
cjm
10
Notation of composite scores

• Cjkm composite score of person k over the
  indicators of trait j in country m
• Cjkm S(wij.Rijkm) over i1 p
• Where Rijkm is the response of respondent k of
  country m on question i for trait j
• The composite score can also be expressed in
  matrix notation
• cjm wjrm wjam wjLmfm wjem

11
Means of composite scores

• It follows that E(cjm) wjam wjLmE(fm)
• or
• In general the means of the responses will not
  be equal to the means of the latent variables
• and
• The means will not be comparable unless
• a1 a2 and L1 L2

12
Means of composite scores

• Using Bartlett factor score coefficients
• It can be shown that
• wjLm (0 0 1 .. 0 0 00)
• In this case the means of the composite scores
  and the latent factors are equal
• But comparison is only possible if the loadings,
  intercepts and measurement error variances are
  identical (Lawley and Maxwell 1971)

13
Means of latent constructs

• The general measurement model is the same
• rm am Lm fm em
• with (em) 0, E(fm.em ) 0
• and E(em.em) is diagonal
• But there is extra information
• E(rm ) am Lm E(fm )

14
Means of latent constructs

• If the model is correct
• ULS,GLS,ADF and ML provide consistent estimates
  for all parameters including for E(fm)
• So the means can also be compared.
• Note that not all indicators have to be
  identical.
• It is sufficient if 2 indicators are identical
  with respect to loadings and intercepts

15
Comparison of relationships

• Criteria for comparison of relationships
• between single questions
• between composite scores
• between latent constructs

16
Relationships between single questions

• The general measurement model is
• rm am Lm fm em
• with (em) 0, E(fm.em ) 0
• and E(em.em) is diagonal
• This model can also be used for the relationships
  between single questions.
• It means that Lm is an identity matrix.

17
Relationships between two questions
s21m
F1m
F2m
1
1
R1m
R2m
e1m
e2m
18
Relationships between single questions

• It follows that
• Sm fm Qem
• This result shows Sm fm if
• Lm I and Qem 0 which is unlikely.
• To compare the observed correlations across
  countries the same conditions should be satisfied
  or that
• Qe1 Qe2
• Both are not necessarily true.

19
Relationships between single questions

• ESS Questions concerning Political efficacy
• How far do you agree or disagree with the
  following statements?
• L4 Sometimes politics and government seem so
  complicated that I cant really understand what
  is going on
• L5 I think I could take an active role in a
  group involved with political issues
• L6 I am good at making my mind up about
  political issues

20
Relationships between questions concerning
Political efficacy
21
Relationships between questions concerning Social
Trust
22
Relationships between composite scores

• The composite score has been expressed in matrix
  notation as follows
• cjm wjrm wjam wjLmfm wjem
• In general cjm is not equal to fm
• And the correlation matrix is
• S(c1m,c2m) w1LmfmLmw2
• If the variables are standardized

23
Relationships between composite scores

• S(c1m,c2m) fm if
• the weights (wm) are calculated using the
  Anderson and Rubin method
• Normally these weights are not used and then
  comparison is not possible without corrections

24
Relationships between two composite scores
s21m
F1m
F2m
w2Lm
w1Lm
C1m
C2m
e1m
e2m
25
Relationships between composite scores

• Using the correlations between the factors and
  the composite scores as the loadings and 1-
  squared correlation as the error variance
• the correlations between the composite scores are
  corrected for measurement error

26
Relationships between composite scores

• Using this approach the relationships between the
  variables of interest can be compared across
  countries
• This solution does not require that all the
  loadings and means for all indicators are equal.
• It requires at least two identical indicators for
  each factor

27
Relationships between latent variables

• The covariance matrix fm can be estimated from
  the covariances between the responses
• Sm Lm fm Lm Qem
• This estimator will be consistent
• even if not all indicators have identical
  loadings.

28
Relationships between latent variables

• Also in this case the relationships between the
  variables of interest can be compared across
  countries.
• Each factor should have at least two identical
  indicators

29
Conclusion

• Comparison of means across countries is rather
  difficult for single items and composite scores
• Comparison of means of latent variables is no
  problem using the means of the latent factors
• One even does not need more than two identical
  indicators for each factor

30
Conclusions

• Comparison of relationships across countries is
  again difficult for single items
• With composite scores correction for measurement
  error is required
• With latent variables there is no problem
• Again only two identical indicators are needed
  for each factor

31
Conclusions

• Comparison of means and relationships across
  countries can be done directly using the latent
  variables
• In that case the requirements for comparability
  are less strict as normally assumed
• The reason is that we are not interested in
  individual scores of individual persons