AFOQT Phase VI

1
AFOQT Phase VI Reduction of AFOQT Administration
Time Summary of Results Presented by Dr. Wayne
Shore Dr. Bruce Gould 23 August 2001
2
(No Transcript)
3
AFOQT FACTOR ANALYSIS

• ? 5 FACTORS COMPRISED OF 16 SUBTESTS
• - Verbal WK (.88), VA (.81), RC (.81)
• - Quantitative MK(.82), AR(.75), DI(.64),
  SR(.52), GS(.52)
• - Spatial/Visualization HF(.82), RB(.71),
  EM(.53), BC(.50)
• - Pilot AI(.88), IC(.66), MC(.56)
• - Perceptual Speed TR (.85), BC (.62), SR
  (.56)
• 90 OF RELIABLE VARIANCE ACCOUNTED FOR
• FACTOR STRUCTURE MAINTAINED IF UNDERLINED
  SUBTESTS REMOVED NO SIG LOSS IN FACTOR
  PREDICTION

4
EFFECTS ON AFOQT COMPOSITE RELIABILITY
AFOQT OPERATIONAL AND PROPOSED COMPOSITE
RELIABILITIES (Rxx) (USING WHERRY-GAYLORD
PROCEDURE) COMPOSITE OPERATIONAL PROPOSED Verba
l .93 .91 Quantitative .94 .92 Academic
Aptitude .96 .94 Pilot .96 NP .92 NPAR .94
Nav-Tech .97 .95
5
RELIABILITY EFFECTS (Contd)
INTERCORRELATIONS BETWEEN OLD NEW
COMPOSITES Verbal .970 Quant .974 Acad
Apt .981 Pilot .965 Nav-Tech .977 NO
SIGNIFICANT RELIABILITY OR CORRELATION LOSS BY
REMOVING RC, DI, MC, EM, SR EFFECTS ON
PREDICTION OF CRITERIA SHOULD BE MARGINAL
6
PILOT COMPOSITE (P) VALIDITY AFTER SUBTEST
REDUCTION
CURRENT P SUBTESTS VA, MC, EM, SR, IC,
BC, TR, AI NEW PILOT (NP) COMPOSITE REMOVES
UNDERLINED SUBTESTS AR IS POSSIBLE ADDITION TO
NP (NP-AR) VALIDITIES (corrected) Criteria--
Ground School (GS), T-37 check ride T-38 check
ride Sum of GS, T-37 T-38 UNIT WEIGHTED
REGRESSION
WEIGHTED COMPOSITE GS T-37 T-38 SUM
GS T-37 T-38 OPERATIONAL -
.3686 - -
- - - NP .3320 .3619
.2279 .3883 .3409 .3828 .2477 NP
AR .3528 .3658 .2296 .3967 .3717
.3944 .2540 NO NAV-TECH VALIDITIES YET NO
OTS OR AFROTC DATA EXPECTED
7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
DIFFERENTIAL ITEM FUNCTIONING (DIF) (USED
Mantel-Haenszel Procedure)
DIF analyses evaluate fairness of individual
test items comparing reference group (whites or
males) to focal group (ethnic or gender minority)
performance for people of equal aptitude DIF
is relevant for selecting/removing test items to
minimize bias DIF was not used in this test
reduction effort because was unnecessary to go to
item level for reducing test time. However, SOW
specified DIF analysis and is of interest to
explain crossover analysis results
20
DIF ANALYSES (Contd) Example VA and WK
analyses VA good balance of reference and
focus group items and only 2 critical
items Favored Favored Reference v Focus Sig
DIF Items Reference Focus Critical White v
Hispanic 17 of 25 7 9
0 White v Black 17 of 25 8 9
0 Male v Female 21 or 25 9 11
2 2/0 WK poor gender ethnic balance
of items but was removed Favored Favored Refe
rence v Focus Sig DIF Items Reference Focus
Critical White v Hispanic 17 of 25
12 5 6 4/2 White v Black 20
of 25 16 4 12 8/4 Male v
Female 16 of 25 11 5 2
2/0
21
(No Transcript)