Administration and Interpretation of the Managerial Inbasket Simulation

1
Administration and Interpretation of
theManagerial Inbasket Simulation

• Kenneth M. Nowack Ph.D.
• 3435 Ocean Park Blvd. Suite 214 ? Santa Monica,
  CA 90405
• [phone] ? [phone] Fax
• http//www.envisialearning.com

2
ENVISIA LEARNING

• Envisia Learning helps leaders create
  high-performing organizations through effective
  talent management by offering a comprehensive
  selection of tools and services for talent
  selection, engagement, development and retention.
  Our results-based tools and processes are built
  on decades of research conducted by licensed
  industrial / organizational psychologists.
• Kenneth M. Nowack, Ph.D. is a licensed
  psychologist (PSY 13758) who has over 20 years
  experience in the human resources field as both
  an internal and external consultant. Dr. Nowack
  received his doctorate degree in Counseling
  Psychology from the University of California, Los
  Angles and has published extensively in the areas
  of 360-degree feedback, assessment, health
  psychology and behavioral medicine.
• Ken is the author of the Emotional Intelligence
  View 360, Executive View 360, Manager View 360,
  Performance View 360, Career Profile Inventory,
  PeopleIndex and the StressScan assessments.
• Ken is a guest lecturer at the UCLA Anderson
  School of Management and also serves on Daniel
  Golemans Consortium for Research on Emotional
  Intelligence in Organizations.

3
APPROACHES TO PERSONNEL SELECTION WHICH ARE MOST
PREDICTIVE?

• AVERAGE VALIDITY
• .38 to .54
• .38 to .54
• .41 to .43
• .41 to .43
• .24 to .38
• .15 to .36
• .15 to .26
• .14 to .26
• .13 to .15
• .10 to .15
• .10 to .15
• .10 to .15
• .00 to .10

• A. WORK SAMPLE TESTS
• B. INTELLIGENCE TESTS
• C. ASSESSMENT CENTERS
• D. PEER RATINGS
• E. WORK HISTORY
• F. INTERVIEWS
• G. PERSONALITY TESTS
• H. REFERENCE CHECKS
• I. TRAINING RATINGS
• J. SELF RATINGS
• K. EDUCATION/GPA
• L. INTERESTS/VALUES
• M. AGE

4
Description of the Envisia Managerial Inbasket
Simulation

• The Envisia Inbasket is a work sample simulation
  that assesses the leadership, task and project
  management competencies of a typical manager.
  The participant is given a brief description of
  the situation surrounding a fictitious
  organization they are assigned to work for and
  asked to respond to 23 Inbasket memos. The
  Inbasket memos contain work situations, requests
  from customers and employees, personnel problems,
  and decisions that a typical manager might be
  asked to handle. The participants task is to go
  through as many of the 23 Inbasket memos within
  90 minutes, making appropriate decisions,
  answering letters and memos from internal and
  external customers, planning meetings, and
  solving problems.

5
Envisia Inbasket Simulation Management
Competencies

• Initiative
• Interpersonal Sensitivity
• Planning/Organizing
• Delegation
• Follow-up/Administrative Control
• Problem-Analysis
• Decisiveness
• Judgment

6
Reliability and Validity of the OPD Managerial
Inbasket Simulation

• Content for the OPD Inbasket Simulation was
  derived from job analyses of managerial positions
  in diverse industries
• Average inter-rater reliability across the eight
  competencies was .93 in one study
• Validity study 1 explored the association between
  Inbasket scores and job performance for 132
  aerospace managers (average correlation was .26,
  p lt .05)
• Validity study 2 explored assessment center
  performance and Inbasket scores for 72 utility
  managers. Overall scores were significantly
  correlated with assessor performance ratings (r
  .26, p lt .05)
• Validity study 3 explored assessment center
  performance and Inbasket scores for 144
  production supervisors. Overall Inbasket scores
  were significantly correlated with both assessor
  and supervisory performance ratings (average rs
  .29, all ps lt .05)

7
Administration of the EnvisiaManagerial Inbasket
Simulation

• Read the administration instructions to the
  participant aloud
• Emphasize that they will have only 90 minutes to
  complete the exercise
• At the completion, the participant will be asked
  to fill out the Participant Report Form found in
  the Inbasket materials
• Emphasize that they can not cancel the trip they
  will be asked to go on as described in the
  instructions
• Participants can take breaks during the exercise
  but not allowed additional time beyond 90 minutes
• Participants are asked to attach any written
  responses to Inbasket items to the appropriate
  memo and number them to facilitate the scoring
  process

8
Scoring the Envisia Managerial Inbasket
Simulation

• The competency-based scoring key is based on
  observed behaviors demonstrated on the Inbasket
  Simulation
• The objective scoring key provides overall scores
  for each of the eight Inbasket competencies
• Points are given for specific actions and
  decisions observed by the Inbasket participant
• Comments from the Participant Report Form are
  utilized during the Inbasket scoring process
• Scoring ranges and norms from diverse US
  companies are provides in the back of the
  Inbasket scoring key

9
Envisia Managerial Inbasket Reporter

• The Envisia Managerial Inbasket Online Reporter
  generates an individual feedback report
  summarizes strengths and development areas across
  the eight competencies
• The feedback results results are both graphic and
  narrative and are based on US norms with
  approximately 5,000 supervisors and managers in
  diverse industries
• The one-page graphic summary provides feedback in
  standard scores (t-scores) with a mean of 50 and
  standard deviation of 10

10
Inbasket Simulation Reporter Sample
Graphic Results
11
Scoring the Envisia Inbasket SimulationInitiative

• Initiative is defined as the ability to influence
  events to achieve specific outcomes through
  individual actions (i.e., originates action
  rather than waiting for direction from others).
  Individuals who take initiative tends to make
  plans, decisions, and solve problems without
  waiting for direction from others
• Individuals who scored high on this Inbasket
  competency made decisions, took actions, and
  delegated assignments to others indicating a
  willingness to frequently take initiative. These
  individuals did not express hesitation to act and
  make decisions on their own in specific
  situations. For example, those with high scores
  tended to request additional information about a
  problem or decide to hold a special meeting to
  resolve an important issue. Individuals with low
  scores did not tend to take such actions or make
  such decisions as frequently

12
Scoring the Envisia Managerial InbasketInitiative

• Were actions taken on specific Inbasket items to
  prevent future problems and issues from occurring
  again?
• Were specific actions taken or decisions made
  that were proactive, rather than, just responding
  to the issues and challenges presented?

13
Scoring the Envisia Inbasket SimulationInterperso
nal Sensitivity

• Interpersonal sensitivity is defined as the
  ability to take actions that indicate
  consideration for the feelings and needs of
  others. Some demonstrations of sensitive and
  empathetic behaviors include asking questions
  about work and non-work activities, expressing
  concern about problems, taking an interest in
  others, and making decisions that take into
  account the feelings of others
• Individuals who scored high on this Inbasket
  competency tended to respond in writing to others
  in a manner that expressed caring and empathy in
  the handling of specific interpersonal situations
  and problems. High scores suggest more frequent
  demonstration of written praise, compliments,
  positive feedback, and recognition than towards
  others than those with low scores. For example,
  individuals with high scores might have written a
  note of congratulations to an employee for
  exceptional performance or expressed sensitivity
  in not approving a vacation request at an
  inappropriate time

14
Scoring the Envisia Managerial InbasketInterperso
nal Sensitivity

• Were internal and external customers responded to
  in a timely manner?
• Were internal and external customers responded to
  in a sensitive and caring manner?
• Did the tone of the response to specific Inbasket
  items express a level of understanding, warmth,
  and empathy?

15
Scoring the Envisia Inbasket SimulationPlanning/O
rganizing

• Organizing, Planning, and Scheduling are defined
  as the ability to effectively schedule time and
  establish a course of action to accomplish
  specific goals for self or others. In general,
  organizing refers to longer-range plans and
  activities, whereas, planning and scheduling
  refers to the management of daily tasks and time
• Individuals who scored high in this Inbasket
  competency demonstrated the ability to
  effectively manage their time, organize their
  schedule, and plan for future activities. These
  individuals approached the in-basket simulation
  by prioritizing each item rather than attempting
  to tackle them in the order they were placed.
  Individuals with high scores tended to
  demonstrate the effective use of planning tools
  by utilizing the monthly calendar or preparing an
  action item list of meeting dates and phone calls
  to be made

16
Scoring the Envisia Managerial Inbasket
Planning/Organizing

• Were the Inbasket items prioritized or done in
  order presented (I.e., were the items grouped and
  addressed in some logical order and attention
  given to high priority items)?
• Was the calendar used to assist in scheduling and
  planning efforts?
• Was there awareness of of specific scheduling
  conflicts and problems (I.e., notes or comments
  suggesting scheduling problems, meeting dates,
  etc.)?

17
Scoring the Envisia Inbasket SimulationDelegation

• Delegation is defined as the ability to allocate
  necessary authority and resources to subordinates
  in order to accomplish a task, assignment, or
  project
• Individuals who scored high on this Inbasket
  competency demonstrated the ability to select the
  appropriate individual to delegate tasks,
  projects, and assignments. Individuals who
  scored high also demonstrated good judgement in
  determining what was to be delegated in specific
  situations (e.g., making specific decisions,
  researching pertinent information, etc.). For
  example, individuals with high scores tended to
  select the right subordinate to carry out
  appropriate assignments in their absence and
  specified clear actions to be taken in writing.
  Those with low scores tended to take action or
  make decisions themselves rather than delegate
  these to others on many in-basket items

18
Scoring the Envisia Managerial Inbasket Delegation

• Was there awareness of the need to delegate on
  specific Inbasket items?
• Were tasks/assignments delegated for appropriate
  reasons (e.g., attend a meeting or gather
  additional information)?
• Were tasks/assignments delegated to the
  appropriate individuals?
• Were tasks/assignments that were delegated clear,
  specific, measurable, and have appropriate
  authority?

19
Scoring the Envisia Inbasket SimulationAdministra
tive Control/Follow-Up

• Administrative Control is defined as the ability
  to develop procedures to track monitor
  activities, tasks, and delegated assignments on a
  timely manner
• Individuals who scored high on this Inbasket
  competency demonstrated the ability to monitor
  and follow-up on tasks, projects, and delegated
  assignments to others. For example, these
  individuals wrote notes on their calendars to
  check on the progress of an assignment or
  delegated a task to their subordinate with
  specific outcomes and progress to be reported on.
  Individuals with low scores tended to delegate a
  great deal of authority and responsibility but
  did not attempt to implement formal or informal
  feedback mechanisms to check the progress of the
  task or assignment very often

20
Scoring the Envisia Managerial Inbasket
Administrative Control

• Were arrangements made to follow-up on tasks,
  projects, assignments, and meetings related to
  specific Inbasket items?
• Were future dates and meetings scheduled to
  monitor and track delegated tasks, projects, and
  assignments (e.g., request written report or
  schedule a meeting upon return from the business
  trip)?

21
Scoring the Envisia Inbasket SimulationProblem-An
alysis

• Problem Analysis is defined as the ability to
  accurately define a problem, gather and analyze
  information relevant to the problem, and
  determine possible causes and solutions to the
  problem
• Individuals who scored high on this Inbasket
  competency demonstrated the ability to see
  relationships between in-basket items, and
  correctly identify incongruent dates, times, and
  meetings. For example, individuals with high
  scores might have written a note to someone
  pointing out a potential meeting conflict and
  asking to change the date and/or time in order to
  accommodate their schedule. Individuals who
  scored high might also have linked several
  in-basket items together that were logically
  connected to each other by the people involved or
  the specific problem mentioned. Individuals with
  low scores tended not to point out the various
  scheduling conflicts or relationships between
  various in-basket items

22
Scoring the Envisia Managerial Inbasket Problem
Analysis

• Was there awareness of the interrelationships
  between Inbasket problems?
• Was there recognition of the need for additional
  information about some of the Inbasket items
  before decisions were made?
• Was there recognition of the need to research and
  investigate specific incidents further before
  actions were taken?

23
Scoring the Envisia Inbasket SimulationDecisivene
ss

• Decisiveness is defined as the ability and
  willingness to make a decision, render judgments,
  or take actions when required
• Individuals who scored high on this Inbasket
  competency demonstrated the capacity to make
  quick and numerous decisions when presented with
  the opportunity. These individuals actually made
  a greater number of decisions than those with
  lower scores on this competency. Individuals who
  scored lower tended to ask for more information
  before making a decision or taking action whether
  or not it was appropriate to do so. A highly
  decisive individual generally is characterized as
  taking in a small to moderate amount of
  information and assimilating that data or
  information quickly. Such individuals tend to
  arrive at a single focused solution rather than
  multiple or prioritized solutions

24
Scoring the Envisia Managerial Inbasket
Decisiveness

• Were actions and decisions taken that required
  immediate attention (e.g., vacation requests,
  deadline dates, etc.)?
• Were actions and decisions deferred on specific
  Inbasket items requiring additional investigation
  or information?

25
Scoring the Envisia Inbasket SimulationJudgment

• Judgment is defined as the ability to make
  decisions of high quality and consider
  alternative courses of action based upon
  available information and logical assumptions
• Individuals who scored high on this Inbasket
  competency demonstrated the ability to correctly
  identify the highest quality decisions and
  actions given the information available to them.
  These individuals tended to take actions
  considered being most appropriate given the
  specific information available to them in the
  Inbasket exercise

26
Scoring the Envisia Managerial Inbasket Judgment

• Were major requests and issues requirements
  responded to (e.g., request for a department plan
  from the boss)?
• Were appropriate decisions made (e.g., meeting
  with the company Vice President rather than
  attending a strategic planning meeting scheduled
  at the same time)?
• Was correct action taken on specific Inbasket
  items (e.g., disapproving the vacation request)?

27
Inbasket SimulationDecision Style
Low Decisiveness
Flexible
Analytical
High Decisiveness
Decisive
Integrative
Low Problem Analysis
High Problem Analysis
28
SELECTED REFERENCES

• Wimer Nowack (1998). 13 Common mistakes in
  implementing multi-rater systems. Training and
  Development, 52, 69-79.
• Nowack, K. Wimer, S. (1997). Coaching for human
  performance. Training and Development, 51,
  28-32.
• Nowack, K. (1997). Congruence between self and
  other ratings and assessment center performance.
  Journal of Social Behavior Personality, 12,
  145-166
• Nowack, K. (1994). The secrets of succession.
  Training Development, 48, 49-54
• Nowack, K. (1993). 360-degree feedback The
  whole story. Training Development, 47, 69-72
• Nowack, K. (1992). Self-assessment and
  rater-assessment as a competency of management
  development. Human Resources Development
  Quarterly, 3, 141-155.
• Nowack, K. (1988). Approaches to validating
  assessment centers. Performance Instruction,
  11, 14-16