Criterionrelated psychological assessment in executive and workplace coaching

1
Criterion-related psychological assessment in
executive and workplace coaching

• Dr Elizabeth Allworth
• Allworth Juniper Organisational Psychologists
• Dr Barbara Griffin
• The University of Sydney

2
Overview

• Criterion-related validity
• Models of job performance
• Performance in the context of change
• Predictors of performance in the changing job
  context

3
Reflecting on the context

• Imagine your client is a major pharmaceutical
  company, The Good Health Company
• CEO asks you to assess the capabilities of the
  companys executive team in preparation for a
  development program
• Why? What underlies this interest?
• Where would you start? What would you assess?
• What outcomes might you expect? Why would the
  company buy your assessment proposal?

4
Criterion-related psychological assessment

• Assessment must be linked to some criterion or
  outcome
• First, decide what the assessment should predict.
• Then, look for the best measure.

5
Criteria

• Organisational outcomes
• Effectiveness (growth, profits etc)
• Tenure talent retention
• Productivity (sales results, production figures
  etc.)
• Satisfactory performance
• Job performance (ratings from managers
  multi-rater feedback)
• Successful training outcomes
• Career advancement
• Job satisfaction

6
Implications for coaches

• Start from the perspective of the organisation
  and the individuals job requirements
• Understand the relationships between predictors
  (e.g. measures of cognitive ability, personality
  etc) and job performance
• Select assessments that are predictive of the
  criteria of interest

7
The performance domain

• Imagine you are now coaching the Sales Director
  in The Good Health Company.
• What do you think this persons job requires of
  them?
• What knowledge, skills, abilities and other
  attributes (KSAOs) would the Sales Director need
  for the role?

8
The performance domain

• Traditionally, job performance thought to be a
  unitary or global entity
• More recently, theorists have tried to
  differentiate between different aspects of the
  performance domain
• Today, performance domain considered to be
  multi-dimensional and dynamic

9
Campbell (1990)

• Job-specific task proficiency
• Non job-specific task proficiency
• Written oral communication
• Demonstrating effort
• Personal discipline
• Facilitation of peer and team performance
• Supervision/leadership
• Management/administration

10
Borman Motowidlo (1993)

• Task performance
• Core technical activities
• Usually explicit in a job description
• Contextual performance
• Helpful cooperative constructive behaviours
  that management values
• Organisational citizenship
• Seldom included in job descriptions

11
Benefits of Borman Motowidlos model for
coaching

• Easily understood by coachees
• A framework for developmental needs
• Applicable across all jobs
• Contextual skills likely to be more transferable
  across jobs
• Contextual skills potentially more malleable

12
Performance in the context of change

• Imagine now, The Good Health Company is about to
  be acquired by the multinational, SuperHealth.
• The Sales Directors role will change.
• What KSAOs are likely to be important in this
  context of change?

13
Adaptive performance

• Allworth Hesketh (1999) expanded Borman
  Motowidlos (1993) model to include Adaptive
  Performance
• Accounts for changing nature of work
• Refers to skills and behaviours relevant to a
  changing job context e.g. cognitive ability,
  self-efficacy for change, change coping skills
  etc.

14
Pulakos et al. (2000)

• Adaptive performance consists of
• Solving novel and complex problems
• New learning
• Interpersonal adaptation
• Cross-cultural adaptation
• Handling crises
• Coping with stress
• Handling ambiguity and uncertainty
• Physically oriented adaptability

15
Adaptive performance re-analysed

• Griffin and Hesketh (2003) divided adaptive
  performance into three categories of behaviour
• Reactive behaviour - changing or modifying
  oneself to better suit the new environment
• Proactive behaviour - initiating actions to
  change the environment
• Tolerant behaviour - continuing to function
  despite changed circumstances.

16
Coaching and adaptive performance

• Important for career success in the current
  climate
• Enables coachees to understand the types of
  change and new demands likely to be imposed in
  the future
• A guide to the process of coping with change and
  the types of behaviours necessary

17
Criterion-related validity

• The extent to which performance on a
  psychological test or measure predicts
  performance on the job
• e.g. High scorers should perform better on the
  job than lower scorers

18
Criterion-related validity

• First, determine the criteria using a model of
  job performance or results of a job analysis.
• Choose the tests that best predict the criteria
  of interest e.g.
• ability/aptitude cognitive ability tests
• personality
• interests
• values etc

19
Predictors of overall performance

• Cognitive ability
• Personality
• Conscientiousness
• Other Big Five factors

20
Cognitive ability

• Generalisable validity across jobs
• Measures of general cognitive ability usually
  refer to numerical, verbal and abstract reasoning
• Measures of learning and problem-solving ability

21
Cognitive ability job complexity

• General cognitive ability a better predictor of
  performance in higher level jobs
• Speed and accuracy measures better predictors in
  lower level jobs

22
Cognitive ability and stage of learning

• Cognitive ability important in early stages of a
  job
• Speed and accuracy important in later
  stages....if tasks can be done automatically

23
Personality - The Big Five

• Conscientiousness
• Neuroticism (Emotional Stability)
• Agreeableness
• Extroversion
• Openness to Experience

24
Personality - The Big Five

• Conscientiousness Emotional Stability
  predictive across most jobs
• Extroversion Agreeableness predicts management
  and sales performance
• Openness to Experience predicts training outcomes
• (Barrick Mount, 1991 Tett, Jackson,
  Rothstein, 1991)

25
The Big Five and other criteria

• Conscientiousness predicts job satisfaction
• Emotional Stability predicts income and
  occupational status
• (Judge, Higgins, Thoresen Barrick, 1999)
• Conscientiousness and Extroversion related to job
  status (employed versus unemployed) one year
  after graduation.
• (De Fruyt and Mervielde,1999)

26
Personality Leadership (Judge, Bono, Ilies
Gerhardt, 2002)

• Relatively strong relationship between leadership
  and Big Five
• Extroversion the strongest
• Conscientiousness next most important
• Openness to Experience and Emotional Stability
  also important
• Differential validity leadership emergence
  (being perceived as a leader) and leadership
  effectiveness (being successful as a leader)

27
Leadership and the Big Five (Judge et al., 2002)
28
Predictors of task and contextual performance
29
Predictors of adaptive performance

• Cognitive ability
• Behaviour
• Motivation
• Personality

30
Cognitive ability

• Cognitive ability is a good predictor of adaptive
  performance. Why?
• Cognitive ability predicts performance best when
  tasks are complex or novel
• (Murphy, 1989)
• New learning as job requirements change
• Need to transfer learning from one situation to
  another

31
Behaviour

• Problem-focused coping strategies (Lazarus
  Folkman, 1984 Allworth and Hesketh, 1999)
• Related constructs e.g. resilience, emotional
  control, and anxiety, may also be relevant to
  coaching

32
Motivation

• Self-efficacy for change (Allworth Hesketh,
  1999)
• Role breadth self-efficacy (Parker, 1998).
• Develops from successful and varied experiences
  in organisations, support and reinforcement for
  taking on new roles.
• Individuals with high role breadth self-efficacy
  seem more willing and confident to undertake a
  wide range of roles.

33
Personality

• Proactive personality (Crant, 1995)
• Relationship between career success (career
  promotions, salary improvements and career
  satisfaction) and proactivity in bringing about
  change in ones life and career. (Seibert, Crant
  and Kraimer, 1999)

34
Personality

• Openness to experience consistently related to
  technical proficiency and the acquisition of job
  knowledge (Hough, 1992).
• Individuals who are more open to experience and
  less orderly and deliberate in their approach to
  tasks cope better (LePine, Colquitt and Erez,
  2000)

35
Conclusion

• Psychological assessment needs to reflect the
  requirements of jobs
• Models of performance help to define job
  requirements
• The context of change provides an additional
  dimension for assessment