Conducting Command Climate Assessments Tips and Best Practices

1
Conducting Command Climate Assessments Tips and
Best Practices
Paul Rosenfeld Carol E. Newell Equal
Opportunity Advisors Symposium Memphis, TN June
17, 2003
2
Overview of Presentation

• Group Exercise
• How to Conduct a Command Assessment Tips and
  Best Practices
• Q A

3
Conducting Climate AssessmentsGroup Exercise

• Assemble into small groups
• Discuss how climate assessments have been
  conducted at commands where you have been
  stationed
• Why were the assessments initiated?
• Who was involved in conducting the assessment?
• What role did leadership play in the assessments?
  
• What role did you have in the assessments?
• How was the assessment done? What information
  was collected (surveys, focus groups, etc.)?
• What was done with the information collected?
• What was the outcome of the assessments?
• Present findings to attendees

4
What is a Command Climate Assessment?

• Way of looking at the health and functioning of
  a command or an organization
• Looks at factors such as
• Morale
• Communication
• Teamwork
• Diversity, EO, SH
• Navy policy (OPNAVINST 5354.1E) requires an
  assessment
• Preferably on assuming a command or w/in 6
  months.
• With appropriate follow-up assessments during
  command tenure

5
Why do a Command Climate Assessment?

• How do we know how we are doing?
• Leaders may not know what is going on
• Assessment establishes baselines of things going
  well and things needing improvement
• Signals commitment Command leadership is
  committed to establishing a positive climate
• Good command climate relates directly to
• Mission accomplishment, Productivity, morale,
  retention, readiness

6
Barriers to a Command Climate Assessment

• Often hard to do correctly
• May take significant time and resources
• Ignorance is bliss May uncover unpleasant issues
  that need to be dealt with
• May result in situations where you have to
  deliver bad news to Commanders
• Leaders may not want to hear about problems
• The messenger may be shot -- How to deal with it?

7
Assessment Models

• Navy Model
• (Recommended but not mandatory)
• Collect Command Demographics
• Administer the Survey
• Conduct Interviews and Observations
• Conduct Analysis and Data Integration
• Develop a Plan of Action
• Present Findings

• General Organizational Model
• Form the Assessment Team
• Develop Assessment Plan
• Obtain Commitment from Leaders
• Gather Data
• Policy and Procedure Documents
• Demographic Breakouts
• Survey
• Interviews and Focus Groups
• Observations
• Analyze Data
• Present and Report Findings
• Action Plan

8
Assessment Models

• Navy and Organizational Models
• How are they similar?
• Both have 6 major steps and use principle of
  triangulation to determine findings
• How do they differ?
• Organizational Model has more up-front steps
  that may also be useful when doing Navy command
  assessments

9
Assessment Models Triangulation
Triangulation Data from multiple measures
are used to offset error from any single data
source.
10
Command Assessments Tips and Best Practices

• Based on
• Navy assessment experiences
• Marine Corps assessment experiences
• Experiences as diversity researchers
• Review of organizational literature
• Book chapter on Evaluating Diversity Programs

11
Planning for a Command Assessment

• Form a Team
• Hard to do an assessment without lots of help
• Members should be diverse in terms of
• Skill set
• Demographic background (gender, race,
  paygrade/rank, etc.)
• Develop Assessment Plan
• Plan should include
• List of team members
• Dates of team milestones
• Member (or committee) responsibilities
• Obtain Leadership Commitment
• Meet with CO/XO early in process to gain support
• Update CO/XO on progress regularly

12
Planning for a Command Assessment (Cont.)

• Identify Measures of Success
• Include
• Quantitative or objective data (promotion,
  retention, and disciplinary statistics, etc.)
• Called hard or factual data
• Qualitative or perceptional data (survey data,
  interviews, focus groups, observations, etc.)
• Called soft or perceptual data
• Both quantitative and qualitative data sources
  are needed for a good assessment
• Remember Use more than one data source and look
  for consistent trends across sources

13
Planning for Command Assessment (Cont.)

• Set reasonable timetables
• Faster is not always better
• Avoid one size fits all mentality
• Need to custom-tailor assessments for the
  specific command

14
Gathering Assessment Data

• Divide team into committees with specific data
  collection tasks
• Policy Procedure documents/Past assessments
• Demographics
• Survey
• Interviews/Focus Groups/Observations
• Each committee should provide the CMEO manager
  with a brief summary report of their findings

15
Gather Assessment Data Documents

• Examples of potential data sources
• Policy statements, rules, and regulations
• Are they enforced?
• Are they posted throughout the organization?
• Are people aware of them
• Documentation of special emphasis celebrations or
  observances
• Examples Hispanic, African Amer., or Native
  Indian heritage, months.
• Frequency of EO/SH training
• Key Document Results of last Command Assessment
• The best comparison is often to yourself

16
Gather Assessment Data Demographics

• Provides a diversity snapshot of the
  organization
• Types of data to collect
• Promotion rates, advancement, retention rates,
  disciplinary actions etc., by key race/ethnic and
  gender breakout groups
• Data should be analyzed by the following breakout
  groups
• Race, gender, paygrade/rank, etc.
• Data gathered can be used to determine
• Are disparities present in administrative and/or
  disciplinary proceedings?
• Are career actions consistent and fair across all
  grades?
• Are job assignments, training opportunities, and
  special programs managed based on merit, fitness,
  capability?
• Are awards and recognition given fairly?

17
Demographics (contd)

• Use Excel spreadsheet tool to get a demographic
  snapshot
• Look for trends over time
• Beware of groups/breakouts with small numbers
• Good working rule Never draw conclusions on
  breakouts of less than 5 be cautious on
  breakouts of less than 10

18
Gather Assessment Data Command Survey

• Most common component of command assessment
• Often the survey IS the assessment
• Why?
• Danger Cant validate findings if only single
  data source is used

19
Types of Command Surveys

• MEOCS, SUEOCS, etc. (www.patrick.af.mil/DEOMI/DEOM
  I.HTM)
• DEOMI
• Standard survey, standard report
• Less work, less local control
• Must have at least 50 command participation
• Each survey has approx. 45-100 questions.
• CATWIN (www.catwin.nprdc.navy.mil)
• Standard survey items that can be modified or
  added to
• More work, more local control
• DO YOUR OWN SURVEY
• Lots more work, lots more control

20
Successful Command Surveys

• Involve leaders in survey development
• Determine topic areas in addition to those on the
  standard survey
• Write additional questions
• Administer to maximize response rate
• When conducting focus groups/interviews select
  sample subjects which represent the entire
  command and key subgroups.
• Analyze both survey data and comments for entire
  command and key subgroups

21
Survey Involve Leaders

• If not using MEOCS, top leadership will likely
  want to approve the survey items
• Buy-in can be gained if they are involved in
  survey development
• CO, XO, CMC and others as appropriate

22
Survey Determine AdditionalTopic Areas

• Focus on key areas of interest EO, Diversity,
  QOL
• Limit number of additional questions
• Determine additional information leadership wants
  to know
• Interviews, focus groups, and past surveys can
  provide useful questions

23
Survey Writing Additional Questions

• Keep the items simple and short
• Use items that ask specifics
• Avoid double negatives
• Use the everyday language of Sailors
• Split double-barreled items
• Example Is discrimination occurring and sexual
  harassment?
• Include both closed and open-ended items

24
Survey Administration Response Rate

• Leave enough time to get the job done right
• Dont rush the survey administration
• Encourage voluntary participation to maximize
  response rates
• Remember to distribute the survey to all members
  of a command or to a random sample, if
  appropriate
• Remember to distribute to enough minorities and
  women so that key demographic breakouts will be
  possible

25
Survey Get Representative Sample

• For command surveys, it is important to
  accurately represent the views of personnel who
  are minority groups at a command
• Black, Hispanic, Asian/Pacific Islander
• Women
• Officers

26
Survey Sampling

• If target population is less than 125, survey
  everyone
• If target population is between 125 and 500,
  survey either the entire population or use a
  random sample
• If target population is more than 500, random
  sampling should be used
• Use random sampling technique at right

27
Survey Analysis

• For MEOCS may need to consult with EOA to assist
  interpretation
• For CATWIN and locally generated surveys
• Review data for all respondents
• Review data by key demographic variables
• Look for disparities by race and gender Usually
  gt . 5 for averages (means) and 10 for
  percentages
• Caution Looking at overall command results can
  be misleading Why? Because there may be some
  groups who do not agree with overall command view
  due to negative personal experiences.
• Compare to past survey results if available
• Survey results should be validated through other
  means
• Focus groups, observations, interviews,
  demographics

28
Survey Open-ended comments

• Use comment information to help understand
  meaning of results found in rest of survey
• Dont overreact to comments
• Remember that comments often are NOT
  representative of how all survey respondents feel
• Comments often more negative than rest of survey
• Dont hand over comments to leaders unless you
  paraphrase them to ensure anonymity and remove
  mention of any specific individuals

29
Gather Assessment Data Interviews Focus Groups

• Individual interviews Top leaders
• Focus groups Cross-section of organization,
  command
• Base questions on
• Concerns about command climate or other relevant
  (e.g., EO, Diversity, QOL) issues
• Trends noted in other data gathering measures
• Interview/Focus Group data often helps explain
  results of survey

30
Gather Assessment Data Naturalistic Observations

• Naturalistic observations can help support or
  refute command climate findings from other data
  sources
• Use observational skills to see command behavior
  as it occurs
• Attendance at command functions and special
  emphasis events
• Seating arrangements at meals
• Inappropriate jokes and comments overheard in the
  workplace

31
Analyze Data/Presentation

• General Guidelines
• Analyze all the data for a more complete picture
• Look for major themes present in multiple sources
  
• Dont base conclusions on data from a single
  source
• Prioritize and simplify -- Dont sweat the small
  stuff
• Do not give leadership a stack of results
• Categorize findings in a way that tells a story
  and is easy to follow
• Give a balanced view
• Sort results into Good News and Areas for
  Improvement
• Include recommendations and follow-on actions

32
References

• In addition to materials used in conjunction with
  the CMEO Managers course, the following deal
  with diversity assessments in organizations
• Rosenfeld, P., Landis, D., Dalsky, D. (2003)
  Evaluating diversity programs. In J.E.
  Edwards, J.C. Scott, and N. S. Raju (Eds). The
  human resources program-evaluation handbook. (pp.
  343-362) Newbury Park, CA Sage.
• Hubbard, E. E. (1997). Measuring diversity
  results. Petaluma, CA Global Insight
  Publishing.
• Keller, J. M., Young, A., Riley, M. (1996).
  Evaluating diversity training 17 ready-to-use
  tools. San Diego, CA Preiffer Company.

33
(No Transcript)
34
The End