M21 : Assessment in the Workplace

1
M21 Assessment in the Workplace

• Selection Validation, Job Analysis and Job
  Analysis Techniques

2
Overview

• Job Analysis
• What is it?
• Why do it ?
• Job Analysis Techniques
• description
• advantages limitations
• Selection Validation
• classical model (local criterion-related)
• content related validation (global procedures,
  job element method)
• Synthetic validation
• Bayesian Model

3
Purpose of a selection process

• Predicting future job performance
• gather job relevant information about the
  candidate
• discriminate between candidates in their ability
  to do the job
• assess person-environment fit
• PR exercise - selling the organisation to the
  candidate
• Empowerment of employees
• Demonstrate adherence to employment legislation

4

• To determine whether one person will be better
  in job x than another we must have criteria
  relevant to job x on which to judge their
  suitability.
• Job Analysis provides a way of determining what
  KSAs are important (essential/desirable)
• Job Analysis...
• any systematic procedure for obtaining detailed
• and objective information about a job

5
Other functions of Job Analysis

• Not just for selection ..
• Enables problems to be solved on the basis of
  data evidence, rather than subjective
  speculation or history
• Gives insight into why problems may occur/have
  occurred
• Rapid developments in technology have resulted in
  constant changes in work (individual group
  level)
• Can identify learning and training needs of
  current employees

JA
6
Choosing a Job Analysis Method

• Aim is to analyse the job in such a way that
  the functions, outputs or dynamics between
  specific tasks are not artificially distorted.
• Threats to Accuracy
• Time Determined Changes
• Person Determined Changes
• Situation Determined Changes

7
Dimensions along which JA techniques differ

• Orientation (worker- vs. task-oriented)
• Quantification (qualitative vs. qualitative data)
• Structure (open-ended vs. closed)
• Packaging (off-the-shelf vs. DIY)
• Sophistication (e.g. PAQ vs Observation)
• Proximity to Jobs (remote from- vs close to/part
  of job being studied)
• Applicability (broad vs narrow)
• Sensitivity (adaptable or inflexible)
• What kind of information is needed ?
• What resources are available ?

8
JA Methods

• Observation
• Self-Description (diaries)
• Interviews
• Critical Incident Technique
• Repertory Grid
• Checklists/Inventories
• Hierarchical Task Analysis
• Position Analysis Questionnaire

9
Observation

• Type of job?
• What to record?
• Advantages/Disadvantages
• (s) yield numerical ratings potential for
  accurate, objective record
• (-s) labour intensive, Hawthorne effect
  a-priori categorisation

10
Self-Description (Diaries)

• Suitable for most jobs
• What to record?
• Advantages/Disadvantages
• s Potential for accurate record inside
  workings
• -s Objectivity? Drop-out rate A-priori
  categorisation

11
Interviews

• Suitable for all jobs
• Identifying stakeholders
• Interview Structure
• Advantages/Disadvantages
• s flexible, less intrusive
• - s impression management, requires
  familiarity with the role, accuracy rests on
  interviewees recall interviewer-interviewee
  interaction

12
Interviews II Critical Incident
Technique(Flanagan, 1954)

• Incident has to be observable, so that inferences
  can be made about the persons response
• The incident must be critical
• Decide a critical incident
• Determine what was the background to the incident
• What did the person do that was very
  effective/very ineffective
• Why does that response make it very
  effective/very ineffective
• Advantages/Disadvantages
• s flexibility, psychological fidelity, amount
  of information
• -s can only be used where incumbent/job exist
  where to stop?!

13
Hierarchical Task Analysis(Annett Duncan, 1971)

• Principles
• tasks can be defined in terms of objectives/end
  products
• aims to identify the plan (schema) the job holder
  uses to achieve the objective
• the plan is organised into a hierarchical series
  of sub-plans and sub-goals
• the behaviour that is observed when analysing a
  task operations each sub-plan has an
  operation (and sub-operations)
• the analysis will yield task statements
  (activity verb, performance standard and
  conditions/context of operation)
• Advantages
• skilled, semi-skilled manufacturing type jobs
  with only a limited number of right ways
• can only be used for jobs in existence what
  level of detail to go to?

14
Position Analysis Questionnaire

• Information input
• Mental processes
• Work output
• Relationships with other persons
• Job context
• Other job characteristics
• Advantages/Disadvantages
• s job doesnt have to exist (or have
  incumbent) numerical data, minimal resources
  required.
• -s limitations of quantitative approach.

15
Selection Validation

• How to determine whether the KSAs identified by
  the job analysis (and assessed by particular
  selection techniques) are appropriate
• N.B. Face, Content, Criterion and Construct
  Validity
• 1. Classical (Local Criterion Related) Model
• conduct JA
• choose criterion
• choose selection techniques
• pilot selection process
• data analysis (selection data as predictors of
  x, differential validity ?)
• revise selection process (if necessary)

16
Problems with classical model

• Criterion data unsuitable and/or unavailable
• Inadequate number of employees performing the
  same job
• Restriction of range (through pre-selection)
• Extraneous (individual/organisational) variables
  distorting predictive validity
• Takes too long to obtain results (jobs change,
  selection situation changes, organisational roles
  change)
• Number of additional problems if use concurrent
  validation (current employees) as opposed to
  predictive validation (applicants) - attrition,
  varying skill/experience, future jobs?

17
Alternatives

• Content Related Validation
• Global Procedures what differentiates one job
  from another job
• Job Element Method only those elements that
  differentiate between best and worst employees
• Synthetic Validation
• detailed JA - identify job elements and weights
• for each element analysis empirical study of
  selection technique intended to measure it
• determine validity of selection process by
  combining validity coefficients identified x
  weighting Primoff (1958) j-coefficient
• Bayesian Model (Schmitt Hunter, 1977)
• previous validation estimate variability in
  error of validity coefficients reported.