AFOQT FACTOR ANALYSIS

1
AFOQT FACTOR ANALYSIS
? 5 FACTORS COMPRISED OF 16 SUBTESTS - Verbal
WK (.88), VA (.81), RC (.81) - Quantitative
MK(.82), AR(.75), DI(.64), SR(.52), GS(.52) -
Spatial/Visualization HF(.82), RB(.71),
EM(.53), BC(.50) - Pilot AI(.88), IC(.66),
MC(.56) - Perceptual Speed TR (.85), BC
(.62), SR (.56) ? 90 OF RELIABLE VARIANCE
ACCOUNTED FOR ? FACTOR STRUCTURE MAINTAINED IF
UNDERLINED SUBTESTS REMOVED NO SIG LOSS IN
FACTOR PREDICTION
2
EFFECTS ON AFOQT COMPOSITE RELIABILITY
AFOQT OPERATIONAL AND PROPOSED COMPOSITE
RELIABILITIES (Rxx) (USING WHERRY-GAYLORD
PROCEDURE) COMPOSITE OPERATIONAL PROPOSED Verba
l .93 .91 Quantitative .94 .92 Academic
Aptitude .96 .94 Pilot .96 NP .92 NPAR .94
Nav-Tech .97 .95
3
RELIABILITY EFFECTS (Contd)
INTERCORRELATIONS BETWEEN OLD NEW
COMPOSITES Verbal .970 Quant .974 Acad
Apt .981 Pilot .965 Nav-Tech .977 NO
SIGNIFICANT RELIABILITY OR CORRELATION LOSS BY
REMOVING RC, DI, MC, EM, SR EFFECTS ON
PREDICTION OF CRITERIA SHOULD BE MARGINAL
4
PILOT COMPOSITE (P) VALIDITY AFTER SUBTEST
REDUCTION
CURRENT P SUBTESTS VA, MC, EM, SR, IC,
BC, TR, AI NEW PILOT (NP) COMPOSITE REMOVES
UNDERLINED SUBTESTS AR IS POSSIBLE ADDITION TO
NP (NP-AR) VALIDITIES (corrected) Criteria--
Ground School (GS), T-37 check ride T-38 check
ride Sum of GS, T-37 T-38 UNIT WEIGHTED
REGRESSION
WEIGHTED COMPOSITE GS T-37 T-38 SUM
GS T-37 T-38 OPERATIONAL -
.3686 - -
- - - NP .3320 .3619
.2279 .3883 .3409 .3828 .2477 NP
AR .3528 .3658 .2296 .3967 .3717
.3944 .2540 NO NAV-TECH VALIDITIES YET NO
OTS OR AFROTC DATA EXPECTED
5
DIFFERENTIAL ITEM FUNCTIONING (DIF) (USED
Mantel-Haenszel Procedure)
DIF analyses evaluate fairness of individual
test items comparing reference group (whites or
males) to focal group (ethnic or gender minority)
performance for people of equal aptitude DIF
is relevant for selecting/removing test items to
minimize bias DIF was not used in this test
reduction effort because was unnecessary to go to
item level for reducing test time. However, SOW
specified DIF analysis and is of interest to
explain crossover analysis results
6
DIF ANALYSES (Contd) Example VA and WK
analyses VA good balance of reference and
focus group items and only 2 critical
items Favored Favored Reference v Focus Sig
DIF Items Reference Focus Critical White v
Hispanic 17 of 25 7 9
0 White v Black 17 of 25 8 9
0 Male v Female 21 or 25 9 11
2 2/0 WK poor gender ethnic balance
of items but was removed Favored Favored Refe
rence v Focus Sig DIF Items Reference Focus
Critical White v Hispanic 17 of 25
12 5 6 4/2 White v Black 20
of 25 16 4 12 8/4 Male v
Female 16 of 25 11 5 2
2/0