Whole Person Assessment Institute for Selection

1
Whole Person AssessmentInstitute for
Selection Classification

• Navy Personnel Research, Studies, and Technology
• Amanda Lords, EdD

2
Current Selection and Classification Process

• ASVAB battery (verbal, math, technical)
• Designed to predict success in training
• AFQT, high school diploma, and medical screen
• Background check
• 7-10 minute classifier interview
• Focused on critical-to-fill positions

3
WPA Components

• 1st Watch
• ORCA
• ASVABValidation
• RIDE
• JOIN
• SYRUS
• NCAPS

4
1st Watch Objective
Understand what affects Sailorsacross first term
of enlistment

• Identify recruits/Sailors at risk
• Identify intervention points
• Provide tools for interventions

• Better quality Sailors
• Better prepared Sailors

5
Conceptual Model for 1st Watch
Person-Environment Fit
Outcome Variables
Individual Variables
Experiential Variables
Moderating Variables
6
First Watch Data Collection Method
Use New Instruments and Existing Data
with Longitudinal and Cross-Sectional Samples
Begin RTC
End RTC
End A School
End 1st Term

• Database Links
• PRIDE
• CETARS
• EMF

7
OfficeR Competency Assessment(ORCA)

• NSTC partnered with NPRST
• OBOA requested 1st Watch type effort for
  Officer Accessions
• Person-Job Fit model that explores links between
  specific competencies, personality, and job
  performance
• No single JO job
• Demands vary by community and rank
• Focus on entry-level Junior Officers and job
  requirements common across communities
• First phase of researchresults used to develop
  later phases of research

8
ORCA Objectives

• Evaluate Junior Officer training
• Develop a unified set of measurement tools
• Identify competencies of Junior Officers
• Link competencies, training, and job performance
• Do all sources provide comparable training to
  equally prepare new Junior Officers?
• How can we improve accessions process in
  recruiting, selecting, training potential
  officers to optimize job performance?

9
ORCA Conceptual Model
Individual
Job Requirements

• Knowledge
• Skills
• Abilities
• Personality

• Domains
• Tasks

Job Performance
Training
What are they required to do and how well are
they doing it?
What do they have coming in?
What and how do we train them?
10
ORCA Research Design
Begin Initial Officer Training
End Initial Officer Training
Focus Groups
Fleet Assignment
11
ORCA Technical Development

• Web versions of New Officer and Training
  Graduation May
• Hosted on NPRST servers
• Fully editable administrative functions
• Branching for command specific questions
• Develop on online reporting engine
• Provides near real-time access to data
• Programmed statistical procedures

12
RIDE Rating IDentification Engine

• Designed for use by recruiting and PSD Great
  Lakes classifiers
• Best match for recruits based on
• Qualifications
• Available training seats
• RIDE classification algorithm
• Provides greater flexibility in Sailor
  entry-level classification
• Optimizes training success

13
Fleet RIDE

• Fleet Sailor-centric variant of RIDE
• Provides RIDE-similar capabilities for
• Career counselors
• Enlisted community managers (ECMs)
• Individual Sailors for reclassification and
  career guidance
• Transforms RIDE processes and technologies for
  use in support of Perform to Serve (PTS)
• Contributes to the balance of Navy rating skill
  mix

14
RIDE/Fleet RIDE
15
JOIN Job Opportunities in the Navy

• Pictorially-based Navy rating preference tool

16
JOIN Job Opportunities in the Navy

• There is no structured means for deriving
  preferences from naïve applicants
• Selection and classification decisions based on
  applicant preferences and interests lead to less
  buyers remorse
• Expect fewer administrative and disciplinary
  problems

17
Operational Status

• RIDE/Fleet RIDE and JOIN are key foundational
  components of Sea Warrior spiral one
• Fleet RIDE/JOIN are in use at over 1600 commands
  with over 2500 users (career counselors,
  community managers, etc.)
• Fleet RIDE and JOIN are available on Navy
  Knowledge Online (NKO)

18
SYRUS-Multitasking

• Sailors must be increasingly capable of
  performing broad-sweeping duties in wide-ranging,
  often hostile environments
• The Navy requires methods to accurately assess
  individuals multitasking ability
• SYRUS tests include measures of
• Memory (Sternberg test)
• Mathematical operations
• Visual monitoring
• Auditory response

19
SYRUS Testing Variables
Mathematical operations
Memory
Visual monitoring
Auditory stimuli

• Physiological
• Skin temperature
• Skin conductance
• Respiration
• Heart rate
• Blood pressure

• Non-cognitive
• Adaptability
• Anxiety
• Goal orientation
• Perceived workload

Demographic Sex, Age, Race
Cognitive ASVAB, SAT, or ACT
20
Personality Testing

• Personality testing not typically used for
  classification
• Historically used to detect psychopathology
• Long and cumbersome administration
• Validation
• Limited to small groups
• Not based on on-the-job performance
• Proprietary have to pay for every
  administration lack control over information
  obtained

21
Personality Test Types
Psychiatric Screen Based on non-normal
functioning Deselects/Identifies Precludes
individuals from service or identifies
individuals who may have psychiatric
difficulties Risk Misclassification of
individuals who would have otherwise provided
valuable service
Performance Prediction Based on on-the-job
performance Sorts In conjunction with ASVAB
classifies individuals into jobs for which they
are likely to succeed Risk Minimal, individuals
would be cognitively qualified (ASVAB scores)
NCAPS was developed to predict performance - not
psychopathology
22
Navy Computer Adaptive Personality Scales (NCAPS)

• Increase job performance
• Decrease unwanted attrition
• Enhance job/career satisfaction

23
Navy Computer Adaptive Personality Scales (NCAPS)

• Computer-based
• Scored instantly
• Easy administration
• Adaptive item presentation
• Requires fewer questions less time to
  administer
• Better differentiation among trait levels more
  accurate
• Forced choice format more resistant to faking

1st of its kind
Innovative
Web-enabled
24
NCAPS Item Presentation

• Traditional
• I always do the work that is expected of me
• This describes me all of the time
• This describes me most of the time
• This describes me some of the time
• This describes me rarely
• This doesnt describe me

Adaptive I always do the work that is expected of
me I like to set goals that force me to perform
at a level higher than what Ive done in the past
25
NCAPS Traits
Assesses traits required for success across most
Navy jobs
Attention to Detail
Adaptability/ Flexibility
Dutifulness/Integrity
Achievement
Dependability
Self Reliance
Stress Tolerance
Willingness to Learn
Social Orientation
Vigilance
Leadership Orientation
Self Control
Depth of Thought
26
NCAPS Predictive Validation

• Center for Naval Aviation Technical Training
• AT, AW, AC, AD, AS, AME, PR, AMH, ABE, ABF, ABH,
  AO,
• Center for EOD and Diving
• EOD, Divers
• Center for Surface Combat Systems Support
• FC, ET, STG, MN, GM, TM, OS, BM, SM, QM
• Center for Naval Engineering
• DC, EM, EN, GSE, GSM, HT, IC, MM, MR
• Center for Anti Terrorism and Navy Security
  Forces
• MA (possibly VBSS)
• RTC Dive Motivator program (TBD)

27
NCAPS Concurrent Validation

• Large scale concurrent validation planned
• Test instructors of all ratings reporting to the
  Center for Naval Leadership in (MAY)
• Concurrent validation of representative ratings
• Predictive validation of instructor performance
• Test NCAPS at ratings group symposiums
• Begin discussions with targeted communities this
  summer to determine best method of obtaining
  large samples

28
NCAPS Validation

• Validation using multiple data sources

Cognitive
On-the-Job Performance
Moral
NCAPS
Career/Job Satisfaction
Physical Readiness
Training Performance
29
NCAPS Validation

• Cognitive
• ASVAB/AFQT
• Relationship between NCAPS and ASVAB
• Combination required for success

Cognitive
Moral
On-the-Job Performance
NCAPS
Physical Readiness
Career/Job Satisfaction
Training Performance
30
NCAPS Validation

• Moral
• Attrition Reasons

Cognitive
On-the-Job Performance
Moral
NCAPS
Career/Job Satisfaction
Physical Readiness
Training Performance
31
NCAPS Validation

• Physical Readiness
• Training
• Moderate, yet significant, correlations with RTC
  PRT performance
• Fleet

Cognitive
On-the-Job Performance
Moral
NCAPS
Physical Readiness
Career/Job Satisfaction
Training Performance
32
NCAPS Validation

• Training Performance
• Time-to-train
• Academic grades
• Small, yet significant, correlations with RTC
  academic performance
• Sim/lab grades
• Final grades
• Number of setbacks
• Attrition reason
• Graduation status
• Performance by instructional method
  (traditional/computer based)

Cognitive
On-the-Job Performance
Moral
NCAPS
Career/Job Satisfaction
Physical Readiness
Training Performance
33
NCAPS Validation

• Career/Job Satisfaction
• Reclassification
• Reenlistment
• 1st Watch Measures
• Initial analyses in progress
• Orca Measures

Cognitive
Moral
On-the-Job Performance
NCAPS
Career/Job Satisfaction
Physical Readiness
Training Performance
34
NCAPS Validation

• On-the-Job Performance
• EMF
• Supervisor evals
• Community-specific criterion

Cognitive
Moral
On-the-Job Performance
NCAPS
Career/Job Satisfaction
Physical Readiness
Training Performance
35
NCAPS Validation

• Results will provide
• Global trait levels required for success
• Job family-specific trait levels required for
  success

Cognitive
On-the-Job Performance
Moral
NCAPS
Career/Job Satisfaction
Physical Readiness
Training Performance
36
Whole Person Assessment Components
37
Projects Points of Contact
1st Watch
ORCA
Dr. Rorie Harris [phone] rorie.harris_at_navy.m
il
Dr. Christina Underhill [phone] christina.un
derhill_at_navy.mil
NCAPS
SYRUS
Dr. Andy Jones [phone] andy.jones_at_navy.mil
Dr. Amanda Lords [phone] amanda.lords_at_navy.m
il
NPRST Institute for Selection Classification
Director
Dr. Jacqueline Mottern [phone]
jacqueline.mottern_at_navy.mil