Console Operator Selection

1
Console Operator Selection
Jolene Kramer JPS Vice President
Northwest Process Technology Alliance - Technical
Trainers Conference
Bellingham, Washington
September 16, 2004
2
A True Story Russias Nuclear MissileersThe
Rocketchiki
Inner sanctum The TOPOL command center
3
Assessment Testing for the Rocketchiki

• TATIANA ANISIMOVA (Psychologist) The first test
  is to define personality traits. And the second
  is not so much a test, rather to work out
  bio-rhythms. In assessing personal traits, and
  everyone knows it these days, it's been proved,
  on the basis of a person's attitude to color one
  can determine not only their emotional state, but
  also their qualities. As for bio-rhythms, a human
  being is influenced by various rhythms the
  rhythm of the sun's movement, lunar movement and
  so on. And these bio-rhythms, date of birth, the
  lunar phase at that time, the impact of the sun
  on the earth in that year, on that basis the
  bio-rhythms are constructed.

4
Overview

• Why the console operator job is unique
• Benefits of good selection practices
• What most companies are doing
• The latest scientific findings
• Recommended selection practices

5
Jobs Similar to the Console Operator
2004

• Jobs Today
• Air Traffic Controllers
• Refinery Console Operators
• Pipeline Controllers
• Water Treatment Plant Controller
• Nuclear Power Plant Operators
• Utility Power Load Coordinators
• Missile Launch Operator

6
Common Features to These Jobs

• Dynamic Process
• High Information Processing Demands
• Visual Displays
• Interface Separates Employee From the Process
• Long Training Times
• Grave Consequences of Error
• Stressful

7
What Makes Jobs Complex
Future Console Operator?
Level of Ability
Todays Console Operator
Past Board Operator
Number of Abilities
8
Variation in Job PerformanceAs a Function of
Complexity
Over 100 Times
Best
12 Times
Best
3 Times
Best
Worst
Worst
Worst
Low
Medium
High
Complexity of Job
9
Abilities Needed by Console Operators

• Selective Attention (vigilance)
• Problem Sensitivity (identifies problems early)
• Time Sharing (multitasking)
• Reasoning
• Oral Expression (communicates clearly)
• Oral Comprehension
• Resists making premature judgments
• Visualization
• 9. Speed of Closure (pattern recognition)

10
Benefits of Good Selection Practices

• Decreased attrition
• Better trainability of operators
• Fewer incidents
• More profitable operation

11
Relationship Between Selection Training
Infinite
Level of Training Required
No Training
High
Low
Aptitude
12
Levels of Console Operator Expertise
Level 2 System Users
40 - 50
13
Description of Selection Systems

• Minimal System

Drug Screen
Application
Interview

• Standard Approach

Cognitive Test
Interview
Reference Checks
Application
Drug Screen
Physical

• Deluxe Model

(Standard plus role plays, group exercise,
simulations)
14
What We Believe Companies are Doing

• Universal
• Application blank
• Interview
• Drug Screen
• Physical

• Common
• Test of gen. knowledge, reasoning, mech. ability
• Reference checks

• Less Frequent
• Simulations, role plays, group exercises

• Rare
• Personality tests
• Honesty tests
• Graphology tests

15
Recommended Selection TechniquesTests of
Cognitive Ability

• Should be part of any console operator selection
  system
• Probably your single best predictor of
  performance
• True correlation with performance is about r
  .56 (for all complex jobs)
• Paper Pencil (RBH, Bennet Mech. Reasoning)
• Simulation-based

16
Measurement of Key Cognitive Abilities
Traditional selection methods measure some of
Key abilities
17
Interviews

• Should also be a component of any CO selection
  systems
• Can assess a range of KSAOs, particularly
  interpersonal skills, personality traits, org fit
• In general, the more you structure the interview
  process the better
• Unstructured interviews, r .38
• Structured r .51 (in combination with cog tests
  r .63) VERY STRONG CORRELATION.

18
Ways to Add Structure to Interviews

• Develop questions linked to KSAOs identified in
  the job analysis
• Ask same questions of each applicant
• Develop what did you do and what would you
  do type questions as appropriate.
• Patterned or behavioral interview
• Situational interview
• Develop a scoring system for responses
• Train interviewers

19
Reference Checks

• Obtain signed permission statement on application
  to check references
• Limit access in your company to derogatory
  information
• Inquire about state laws limiting liability for
  good faith disclosures by former employers
• Correlation with performance about, r .26

20
Drug Screen

• Typically assess about 8 10 illicit substances
  including marijuana and cocaine
• Some studies find 10 or higher test positive
• Laboratory link between drug use and impaired
  performance
• Some empirical evidence linking drug use and
  negative work behaviors
• Unclear what is the correlation with performance

21
Controversial Tests

• Personality Tests
• There are some new paper and pencil tests
• Target a few broad personality domains
• Conscientious is predictive of job performance
• Faking may not be a big issue
• r .31 (MODERATE CORRELATION)

• Honesty (Integrity)Tests
• Several paper and pencil tests on market
• Can ask direct questions or indirect questions
• Popular with retail establishments to reduce
  theft
• Does show some validity with job performance
• r.34

22
Tests to Avoid

• Graphology Tests
• Personality assessment based upon a sample of
  hand writing
• Fairly common in use as a selection tool in
  France and Israel
• Studies indicate it has virtually no predictive
  value
• r .02 (NO CORRELATION)
• Polygraph Tests
• Legally allowed in just a few situations
• Best validity obtained when asking specific,
  pointed, questions
• False positives do occur and can brand people

23
Example of a Good Selection System
Subject Matter Expertise
24
SummaryFor Console Jobs

• Big payoffs to good selection systems
• A good system will assess cognitive,
  interpersonal, good citizenship, org fit
• Cognitive tests and structured interviews are
  basic (OUGHT TO BE USING THEM)

25
Some Things to Ponder

• What do you think is the value of the selection
  tests that are administered to Russias Nuclear
  Missileers ?
• Think about your own facility.
• What is the value of the assessment tests you are
  using to select console operators at your
  facility ?