Assessment

1
Assessment
2
Review

• Job analysis identifies criteria to aid selection
• e.g., task-oriented
• e.g., person-oriented
• Knowledge of criterion problem helps us choose
  best criteria
• Contamination
• Deficiency
• So we know what to measure we need to understand
  issues surrounding measurement of human traits

3
Measurement

• Measurement the process of assigning numbers to
  objects or events according to rules (Linn
  Gronlund, 1995).
• Psychological Measurement concerned with
  evaluating individual differences in
  psychological traits.
• Trait descriptive label applied to a group of
  behaviors (e.g., friendly intelligent)

4
Utilizing Individual Differences

• Psychologists assume that most traits are
  normally distributed in the population.
• e.g., height, intelligence, KSAs
• Psychologists study
• measuring these differences
• using these differences to predict performance

5
Key Terms

• Predictor (X) - any variable used to forecast a
  criterion
• Criterion (Y) variable we are trying to
  predict, typically performance

SAT Test
College GPA
IQ
Job Performance
6
Types of Assessment

• Biographical Information
• ask about persons past activities, interests,
  behaviors.
• Interviews
• structured vs. unstructured
• get info., but also behavior sample
• Work Samples ask them to do a representative
  task rate their performance
• Assessment Centers
• multiple representative tasks
• Letters of Recommendation
• Psychological Tests

7
Hiring the Best

• Job College Professor
• Tasks lecturing, leading discussion
• What KSAs are associated with each task?
• What are the best ways to measure KSAs?
• Can only collect 2 examples of each type of data
  to assess key competencies
• Biographical Data
• Interview questions
• Work samples
• Provide brief rationale for each

8
Types of Assessment

• Biographical Information
• Interviews
• Work Samples
• Letters of Recommendation
• Psychological Tests

9
Biographical Data

• Good questions are about events that are
• historical
• external
• discrete
• controllable (by the individual)
• verifiable
• equal access
• job relevant
• non-invasive
• (Mael, 1991)
• Rationale vs. empirical method

10
Biographical Data

• Strong criterion validity
• drug use, criminal history predicts dysfunctional
  police behavior (Sarchione et al., 1998)
• not redundant with personality (McManus Kelly,
  1999)
• Measurement issues
• Generalizability
• Faking
• Fairness
• Privacy concerns

11
Interviews

• Structured vs. Unstructured
• Info. gathering vs. interpersonal behavior sample
• Situational interview
• How would you handle a circumstance in which you
  needed the help of a person you did not like?
• Measurement issues
• structured has more criterion related validity
• value of unstructured?
• Illusion of validity
• Guidelines for structured interviews

12
Work Samples

• perform a task under standardized conditions
• historically were for blue collar jobs
• e.g. use of tools, demonstrate driving skills
• white collar examples
• speech interview for foreign worker, test of
  basic chemistry knowledge,
• Measurement issues
• high criterion validity if skills are similar to
  job
• costly to administer
• work best with mechanical, rather than
  people-oriented tasks

13
Assessment Centers

• Realistic tasks done in groups
• Assessed by multiple of raters rating multiple
  domains
• Multiple methods
• in basket group exercise
• leaderless group exercise
• Strong criterion validity (e.g., teachers,
  police)
• overall scores predict job performance
• Measurement issues
• costly to administer
• different ratings on a task too highly correlated
• dimension ratings not correlated strongly across
  tasks
• fix? focus on behavior checklists and rater
  training

14
Drug Testing

• opinion?
• People are more accepting of it if job involves
  risks to others (Paronto, et al., 2002)
• Measurement issues
• reliability is very high, but not perfect
• Validity?
• Normands, Salyards, Mahoney (1990)
• over 5000 postal service applicants
• those who tested positive had 59 higher
  absenteeism, 47 more likely to be fired
• no differences in injury or accidents

15
Letters of Recommendation

• ever written a letter of recommendation for
  someone?
• worst criterion validity of all commonly used
  assessment tools
• some use for screening extremely bad candidates
• Measurement issues
• restriction of range
• writer bias/investment

16
Psychological Test Characteristics

• Group vs. individual
• Objective vs. open-ended
• Paper pencil vs. performance
• Power vs. speed

17
Psychological Test Types

• Ability Tests
• Cognitive ability
• Psychomotor ability
• Knowledge and skill or achievement
• Integrity
• Personality
• Emotional Intelligence
• Vocational interest

18
Integrity Tests

• Designed to predict whether employee will engage
  in counterproductive work behavior (CWB)
• overt vs. personality (covert)
• Better at predicting general CWB and performance
  than theft (r .30 -.40)
• Measurement issues
• difficult to measure criteria!
• proprietary issues
• legal and privacy issues
• faking

19
Personality Tests

• measures predispositions toward particular
  feelings and behaviors
• not all tests are based on past research
• many have shown incremental validity
• e.g., predict when controlling for IQ
• Measurement issues
• job relevance
• not easily/often faked or a problem if faked
  (e.g., job faking too)

20
The Big 5

• Short version of Big 5

21
The Big Five Inventory

• Openness
• Highs imaginative, creative, and to seek out
  cultural and educational experiences.
• Lows more down-to-earth, less interest in art
  more practical.
• Conscientiousness
• Highs methodical, well organized and dutiful.
• Lows less careful, less focused more likely to
  be distracted
• Extraversion
• Highs energetic and seek out the company of
  others.
• Lows (introverts) tend to be more quiet and
  reserved.
• Agreeableness
• Highs tend to be trusting, friendly and
  cooperative.
• Lows tend to be more aggressive and less
  cooperative
• Neuroticism
• Highs prone to insecurity and emotional
  distress.
• Lows more relaxed, less emotional and less prone
  to distress.

22
Big 5

• Conscientiousness show consistent correlation
  with job performance across jobs, across cultures
• Emotional stability and agreeableness predict
  performance in customer service jobs

23
Intelligence Tests

• Have greatest validity
• Often very easy and inexpensive to use
• Wonderlic Personnel Test
• 50 items
• 12 minute time limit
• Sample questions
• Interpreting scores?
• Scores vary as a function of race and ethnicity
• Ethical issues?
• Face validity?

24
(No Transcript)
25
Determining Test Utility

• Goal of testing is to make decisions about
  individuals on the basis of the amount of a given
  trait they possess.
• A test should give us a true picture of a
  persons traits
• Test Score True Score Error

26
Reliability and Validity

• Reliability
• Test-retest
• Parallel (Alternate) forms
• Internal Consistency
• Validity
• Face
• Content
• Criterion-related
• Construct-related

27
Selecting a Predictor

• Validity
• Fairness
• Applicability
• Cost