Different ways to think about validity

1
Different ways to think about validity

• To the extent that a measure has validity, we can
  say that it measures what it is supposed to
  measure.
• There are different reasons for measuring
  psychological variables. The precise way in
  which we assess validity depends on the reason
  that were taking the measurements in the first
  place.

2
Prediction

• As an example, if ones goal is to develop a way
  to determine who is at risk for developing
  schizophrenia
• ones goal is prediction.

3
Predictive Validity

• We may begin by obtaining a group of people who
  have schizophrenia and a group of people who do
  not.
• Then, we may try to figure out which kinds of
  antecedent variables differentiate the two groups.

4
Correct classifications
Lost a parent before the age of 10 10


5
Correct classifications
Lost a parent before the age of 10 10
Parent or grandparent had schizophrenia 50

6
Correct classifications
Lost a parent before the age of 10 10
Parent or grandparent had schizophrenia 50
Mother was cold and aloof to the person when he or she was a child 15
7
Predictive Validity

• In short, some of these variables appear to be
  better than others at discriminating
  schizophrenics from non-schizophrenics
• The degree to which a measure can predict what it
  is supposed to predict is called its predictive
  validity.
• When we are taking measurements for the purpose
  of prediction, we assess validity as the degree
  to which those predictions are accurate or useful.

8
Reality Schizophrenic
Yes
10
No
Measure Schizophrenic
40
Yes
9
Reality Schizophrenic
Yes
No
10
10
No
Measure Schizophrenic
40
40
Yes
50 ( 40 10 / 100) people were correctly
classified
10
Reality Schizophrenic
Yes
No
0
98
No
Measure Schizophrenic
1
1
Yes
99 ( 98 1 / 100) people were correctly
classified, but note the base rate problem.
Cohens kappa is used to account for this
problem.
11
Construct Validity

• Sometimes were not interested in measuring
  something just for technological purposes, such
  as prediction.
• We may be interested in measuring a construct in
  order to learn more about it
• Example We may be interested in measuring
  self-esteem not because we want to predict
  something with the measure per se, but because we
  want to know how self-esteem develops, whether it
  develops differently for males and females, etc.

12
Construct Validity

• Notice that this is much different than what we
  were discussing before. In our schizophrenia
  example, it doesnt matter whether our measure of
  schizophrenia really measured schizophrenic
  tendencies per se.
• As long as the measure helps us predict
  schizophrenia well, we dont really care what it
  measures.

13
Construct Validity

• When we are interested in the theoretical
  construct per se, however, the issue of exactly
  what is being measured becomes much more
  important.
• The general strategy for assessing construct
  validity involves (a) explicating the theoretical
  relations among relevant variables and (b)
  examining the degree to which the measure of the
  construct relates to things that it should and
  fails to relate to things that it should not.

14
Nomological Network

• The nomological network represents the
  interrelations among variables involving the
  construct of interest.

achieve in school
ability to cope


self- esteem
-
distrust friends
15
Nomological Network Validity

• The process of assessing construct validity
  basically involves determining the degree to
  which our measure of the construct behaves in the
  way assumed by the theoretical network in which
  it is embedded.
• If, theoretically, people with high self-esteem
  should be more likely to succeed in school, then
  our measure of self-esteem should be able to
  predict peoples grades in school.

16
Construct Validity

• Notice here that establishing construct validity
  involves prediction. The difference between
  prediction in this context and prediction in the
  previous context is that we are no longer trying
  to predict school performance as best as we
  possibly can.
• Our measure of self-esteem should only predict
  performance to the degree to which we would
  expect these two variables to be related
  theoretically.

17
Discriminant Validity

• The measure should also fail to be related to
  variables that, theoretically, are unrelated to
  self-esteem.
• The ability of a measure to fail to predict
  irrelevant variables is referred to as the
  measures discriminant validity.

achieve in school
ability to cope


self- esteem
-
like coffee
distrust friends
18
Validity Assessing validity

• Finally, it is useful, but not necessary, if a
  measure has face validity.
• Face validity The degree to which a measure
  appears to measuring what it is supposed to
  measure.
• A questionnaire item designed to measure
  self-esteem that reads I have high self-esteem
  has face validity. An item that reads I like
  cabbage in my Frosted Flakes does not.
• In the context of prediction, face validity
  doesnt matter.

19
A Final Note on Construct Validity

• The process of establishing construct validity is
  one of the primary enterprises of psychological
  research.
• When we are measuring the association between two
  variables to assess a measures predictive or
  discriminant validity, we are evaluating both the
  quality of the measure and the soundness of the
  nomological network.
• It is not unusual for researchers to refine the
  nomological network as they learn more about how
  various measures are inter-related.