Physical Abilities Screening Testing and Measurement Deborah L' Gebhardt, Ph'D', FACSM Human Perform

1
Physical Abilities Screening Testing and
Measurement Deborah L. Gebhardt, Ph.D.,
FACSMHuman Performance Systems, Inc.5000
Sunnyside Avenue, Suite 203Beltsville, Maryland
20705

• Email hpsdgebhardt_at_erols.com
• Phone [phone]

2
Overview of Topics

• Job Analysis of Physical Tests
• Test Development (basic ability, simulation)
• Test Validation
• Passing Score Development

3
Why Validation is Needed

• Need to complete academy
• Ability to perform job tasks
• Protect Public (failure to train)
• Federal statutes

4
Job Analysis ? Observations
Interviews ? Essential Task Identification ? Abi
lities Identification ? Physiological/Biomechani
cal Analysis ? Ergonomic Parameters
Identification ? Environmental/Working
Conditions
5
Physiological/Biomechanical Analysis
? Oxygen uptake requirements to carry 25 lbs.
of equipment in hostage situation ? Work
performed (kcal) chasing suspect through back
yards in residential area ? Forces required
to separate 2 people
6
Job Analysis ? Observations
Interviews ? Essential Task Identification ? Abi
lities Identification ? Physiological/Biomechani
cal Analysis ? Environmental/Working
Conditions ? Ergonomic Parameters
Identification
7
Ergonomic Parameters ? Heights
Weight ? Distances ? Forces Torques
? Acceleration ? Body Positions Equipment
Design ? Anthropometric Measurements
8
Task Inventory ? Questions form basis for
test development ? Information from site
visits observations ? Information not
obtained in site visits ? Confirm
innformation from few incumbents
9
Task Inventory - Essential Tasks ?
Demographic information (age, sex, race,
rank, job tenure, assignment, location) ?
Tasks rated on frequency, importance, time
spent ? Supplemental questions (ergonomic,
environment)
10
Task Inventory - Physical Abilities
? Muscular strength ? Muscular endurance
? Anaerobic power ? Aerobic
capacity ? Flexibility ? Equilibrium
? Dexterity
11
Types of Tests

• Basic Ability
• (e.g., Sit-ups, 1.5 mile run)
• Job Simulation
• (e.g., pursuit, trigger pull)

12
Advantages of Basic Ability Tests 1. Assess
one ability 2. Safe to administer 3. Can be
used for multiple jobs 4. Greater
standardization 5. Practical 6. Easily
transported
13

• Examples of Tasks Linked to Physical Abilities
• Muscular Strength
• Lift or drag incapacitated person 50 feet.
• Muscular Endurance
• Restrain or subdue resistive individual.
• Flexibility
• Stoop/kneel to gather evidence from under dash of
  vehicle.
• Equilibrium
• Walk on narrow elevated catwalks.

14
Disadvantages of Basic Ability Tests 1. Test
does not resemble job.
15
Advantages of Work Sample Tests 1. Looks like
the job. 2. Can be developed directly from
essential job tasks. 3. Can obtain limited
measure of how person handles equipment.
16
Disadvantages of Work Sample Tests 1. May not
assess all abilities/cannot assess one
ability. 2. Cannot be used for multiple
jobs. 3. Difficult to monitor safe
behavior. 4. Limited in simulation of skilled
tasks. 5. Practical constraints in
set-up/storage. 6. Not easily
transportable. 7. Typically more costly.
17
Basic Test Development Considerations Regardless
of Test Type ? Test reliability ? Test
validity ? Use test with least adverse impact
? Safety ? Practical considerations
18
Test Development - Basic Ability ? Select
known tests (e.g, VO2, arm lift) ? Design new
tests (address basic physiological
components) ? Design hybrid tests (based on
task ability data )
19
Test Development - Worksample ? Use
frequently performed essential tasks ? Include
ergonomic parameters ? Physiological/biomechanica
l simulation ? Task sequence ? Number of
individuals who performs task
20
Examples of Test - Retest Reliability
For Basic Ability Tests For HPS Arm Lift
.87-94 Arm Endurance .76 Leg
Lift .89 Leg Endurance .77 Sit
Ups .94 Handgrip .91 Sit
Reach .94 Stabilometer .82 Step
Test .95 1.5 mile .87 Bike
Ergometer-VO2 .76
21
Examples of Test - Retest Results For Work
Samples From HPS Law Enforcement
Reliability Coefficient Police Pursuit
Run/Rescue .90 Trigger Pull
(dom) .79 Trigger Pull (nondom) .77 Wall
Climb .88 Chainlink Fence
Climb .94 Pursuit Handcuff .93 Vehicle
Exit Rescue .83
22
Test Transportability ? Criterion-relate
d validity evidence needed ? Selection procedure
must be valid ? Tests must be fair to
minorities ? Incumbents in new Job must
perform substantially same job as in original
job
23
HPS Job Similarity Index (JSI)
JSI C1 C2
(C1 U1) (C2
U2) ? C1, C2Common tasks between 2
agencies ? U1, U2 Unique tasks for each agency
24
Medical Screening Waivers Applicants
? Waiver if test pre-offer ? Clearance from
personal physician Incumbents ? Medical
self-report ? Medical exam for clearance
25
Types of Validity

• Content
• Criterion-Related
• Construct

26
Types of Data to Collect
? Test-retest ? Multiple Trials ? Test Scores
Split Times ? Criterion measures (job perf.
measure)
27
Types Of Criterion Measures ? Simulation
of Job Task(s) ? Supervisor/Peer
Evaluation ? Injury Loss Work Time Data
? Productivity Data
28
Standardization of Test Administration
Procedures ? Written set-up
administration procedures ? Test administrator
training ? Objective scoring procedures
? Calibration of equipment ? Maintenance of
test area ? Standardized applicant rotation
29
Types of Data Analysis ? Descriptive
statistics (mean, std) ? Correlations ? Regressi
on analysis ? Canoncial correlation CAUTION
Significance is not always meaningful. Are
the results practical?
30
Physical Performance Test Criterion Measure
Correlations Ability
?Pursuit Work ?Ratings, Acad, Test
Ratings
Trig. Pull Work Sample Trunk Pull
.44 .60 .56 Arm Endurance
.39 .60 .54 Sit Ups .49 .30 .48 Equili
brium .10 -.09 .01 Arm Lift
.56 .75 .72 Leg Endurance
.43 .54 .59 Trigger Pull (Ndom) .37
-- -- Pursuit Rescue -.46 -- --
31
Multiple R .84 for 4-Item Test Battery
? Pursuit Work Sample ? Arm Lift ? Leg
Endurance ? Sit-Ups y4sit-upsarm
liftleg endurance18work sample
32
Passing Scores ? There are No Magic
Bullets ? Hardest part of a project
33
EEOC says Passing scores ...should normally be
set so as to be reasonable and consistent with
normal expectation of acceptable proficiency
within the work force (Uniform Guidelines, 1978,
Section H)
34
Factors To Consider When Setting Passing
Scores ? Test Validity ? Adverse
Impact ? Safety ? Job Performance ? Productivi
ty ? Legal Requirements
35
Methods To Set Passing Scores
? Expectancy Tables ? Contingency
Tables ? Taylor Russell Tables ? Normative
Data ? Ergonomic Data
36
Passing Scores Criterion-Related Validity
Study What is needed? ? Predictor tests
(arm lift, pursuit) ? Criterion measure (job
performance) ? Expectancy tables ? Contingency
tables
37
Multiple R .84 for 4-Item Test Battery
? Pursuit Work Sample ? Arm Lift ? Leg
Endurance ? Sit-Ups y4sit-upsarm
liftleg endurance18work sample
38
Job Performance
Subgroup 1
Subgroup 2
Common Regression Line
Test Performance Differential Prediction Analysis
39
Types Of Test Scoring Models ? Multiple
Hurdle ? Compensatory
40
Expectancy Table Level Test Score
Incumbents Job Performance
0.0 100.0
10.6305 10 753.3
93.9 10.8990 20
855.5 90.2
11.0030 30 923.7
78.6 11.3770 40
961.0 68.0
11.6466 50 1003.3
56.6 11.9239 60
1025.9 47.5
11.9677 70 1039.5
35.3 12.2246 80
1045.2 24.6
12.3328 90 1050.0
11.5 12.2720
41
Acceptable Unacceptable
Job Performance Success (False
Rejection - 2) (True Acceptance - 18)
X X
X X X X X X X
X X X X X
X X
X X X X True
Rejection - 8 False Acceptance - 2
X X X X X
X X X X
X
Test Score
Unacceptable Acceptable
Passing Score
42
Comparison Test Battery Score to Satisfactory
and Unsatisfactory Job Performance
Minimum Test Incumbents Attaining
Incumbents Failing Pass Score Specific Test
Score Specific Test Score


Unsat. Sat. Unsat
Sat.
000.00 14 86
--- --- 666.0 ( 5)
10 90 100
0 754.3 (10) 8 92 82
18 827.0 (15) 5 95 67
33 855.5 (20) 5 95
57 43 923.7 (30) 3 97
42 58
43
Test Implementation Costs ? Develop test
v. transport test ? Pretest training
program ? Training equipment facility
44
Physical Performance Tests

• Provide workers who can perform job
• Reduce injuries loss work time BUT
• Must be validated
• Must have job-related passing score
• Must have standard implementation

45
Documentation ? Foundation for test
development ? Provides a history of
project ? Ensures institutional memory
? Serves as a legal document
46
Post Validation Test Data Collection
? Applicant database (e.g., tracking, adverse
impact) ? Applicant score comparison with
academy ? Applicant score comparison with job
performance
47
Information for Purchase of The Process of
Physical Fitness Standards Development
Contact Lisa McIntosh Email Address
lisa.mcintosh_at_wpafb.af.mil Phone
937-255-4842