CRITERION-RELATED VALIDITY

1
CRITERION-RELATED VALIDITY PREDICTIVE

• LECTURE 10
• EPSY 625

2
EMPIRICAL METHODS FOR VALIDITY

• Predictive validity
• logistic regression
• discriminant analysis/cluster analysis
• correlation/structural equation modeling
• Concurrent validity
• correlation/structural equation modeling
• factor analysis
• Construct validity
• factor analysis
• multitrait-multimethod analysis

3
PREDICTIVE VALIDITY- logistic regression
Binary group (0,1) such as hired vs. not hired,
general vs. clinical Transform binary score into
logit L(y) logp/(1-p) Predict L(y) b1
x, where x is a test score Can use SPSS LOGISTIC
option in REGRESSION analysis
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
VARIABLE LABELS

• t1 ANXIETY
• t2 ATTITUDE TO PARENTS
• t3 ATTITUDE TO SCHOOL
• t4 ATTITUDE TO TEACHER
• t5 ATYPICALITY
• t6 DEPRESSION
• t7 INTERPERSONAL RELATIONS
• t8 SENSE OF INADEQUACY
• t9 LOCUS OF CONTROL
• t10 SELF ESTEEM
• t11 SELF RELIANCE
• t12 SENSATION SEEKING
• t13 SOMATICIZATION
• t14 SOCIAL STRESS

8
Multinomial regression

• Extension of logistic regression
• 3 or more groups contrasted
• Ordered groups- compute threshhold for
  classification as a 1 or 2 , 2 or 3 etc
• Unordered groups- can do pairwise logistic
  regression or a priori contrasts among groups as
  the organizer for binomial contrasting (eg.
  groups A and B vs. groups C, D, and E)

9
PREDICTIVE VALIDITY DISCRIMINANT ANALYSIS
Group membership
Test scores
eg, which MMPI scales differentiate/separate/predi
ct manic depressives from normal functioning
adults? This will be useful upon intake or
commitment hearings in addition to clinical
judgement
10
DISCRIMINANT ANALYSIS

• 2 Groups statistical procedure is identical to
  multiple regression with group (1 or 2) as
  dependent variable, k test scores as predictors
• 3 or more Groups discriminant analysis separates
  the groups based on a weighted sum of the
  predictors in standardized form

11
2 Group Analysis

• Model
• y b1x1 b2x2 bkxk b0
• y 1 or 2 (or any two discrete numbers)
• creates single predicted score Dhat which is the
  predicted score for each person. Can compare this
  predicted score with actual diagnoses or
  condition to determine hit rate

12
2 Group Analysis
y2
Db1y1b2y2
Group 1 means
y1
R2 SSD / SStot
Group 2 means
13
2 Group hit rate

• Example predict male (1) vs. female (2)
  differences based on interests x1, x2, xk
• Each person receives a score yhat if yhat is
  below 1.5 the person is predicted to be a male,
  if over 1.5, a female.
• Out of 100 persons (50 M, 50 F), by chance we
  would classify 50 correctly by chance

14
2 Group hit rate

• Cohens kappa will provide evidence for correct
  classification beyond chance
• k Pc - P0/1 - P0
• Alternatively, R2 for the regression provides
  evidence for classification beyond chance.

15
Example Gender predicted from music preferences
R2 SSb / SStot .291/344.7 .001
16
(No Transcript)
17
Discriminant Analysis
Wilks lambda 1-R2
18
(No Transcript)
19
males
females
w
0.0
20
3 Group discriminant analysis

• 2 or more discriminant functions possible
• functions min (predictors, gps-1)
• Evaluate greatest function (group separation)
  first, each function successively
• Examine joint classification for all significant
  functions

21
3 Group Analysis1st discriminant function
Group 1 means
y2
y1
Group 3 means
Group 2 means
Maximize SS between groups
D1b1y1b2y2
22
3 Group Analysis2nd discriminant function
Group 1 means
y2
y1
Group 3 means
Group 2 means
D2b3y1b4y2
D1b1y1b2y2
23
3 Group Analysis
Group 1 means
R12 SSD1 / SStot
y2
D1b1y1b2y2
y1
Group 3 means
Group 2 means
D2b3y1b4y2
R22 SSD2 / SStot
24
3 Group Analysis
Group 1 means
Discriminant function coefficients
y2
y1
Group 3 means
Group 2 means
D2b3y1b4y2
D1b1y1b2y2
25
Example Ethnic music prefs
26
(No Transcript)
27
(No Transcript)
28
Territorial Map Function 2 -3.0 -2.0
-1.0 .0 1.0 2.0 3.0
------------------------------------
------------------ 3.0 21

I 21
I I 21
I
I 21
I I 21
I I
21
I 2.0 21
I 21

I I 21
I I 21

I I 21
I I 21
I
1.0 21
I 21
I
I 21
I I 21
I
I 21
I I 21
I .0
21
I 21
I I 21

I I 21
I I 21

I I 21
I -1.0 21

I 21
I I 21
I
I 21
I I 21
I
I 21
I -2.0 21
I
21
I I 21
I I 21

I I 21
I I 21

I -3.0 21

---------------------------------------------
--------- -3.0 -2.0 -1.0
.0 1.0 2.0 3.0
Canonical Discriminant Function
1 Symbol Group Label ------ -----
-------------------- 1 1 white 2
2 black 3 3 other
Indicates a group centroid