Title: Diversity Awareness Training Sanchez
1Diversity Awareness TrainingSanchez Medkik
- Hypothesis
- Nature of quasi-experimental design
- Measures used their validity
- Tests of Hypotheses
- Alternative explanations for results
- Learning Points
2Hypothesis
Diversity Awareness Training
Cultural Awareness
Differential Treatment of Culturally Different
Individuals
3Method
- Participants
- 125 supervisors/mgrs in a county government
- 125 Raters of supervisors/mg above
- Are no raters evaluating two supervisors?
Condition Number of Participants
Diversity Training 69
Control 56
4Method
- Design
- No random assignment to conditions
- Participants in experimental group were chosen
bec.. - they were one of 4 employees with the longest
tenure in each of their departments - had not received diversity awareness training
- Participants in control group were matched on
tenure with those in the experimental group - Control group Ps would have been eligible for
training - Did Ps in control group receive training before?
5Measures
- Pre-training performance ratings
- Relevant to training dimensions (e.g., Coworker
contact, communication skills) - Extracted for the year immediately before
training - 5-point rating scales ( of items not specified)
- Anchors used poor to excellent
- Issues
- Reliability not given
6Method
- Matched control and experimental groups on
tenure - Control variables
- Pre-training performance rating
- Demographics
- Gender, ethnicity, tenure, educational level
- Demographics of coworkers who rated Ps in control
experimental groups - Gender ethnicity
7Establishing equivalence
- Tested for mean differences between experimental
and control group on - Matching and Control variables
- Do not present appropriate statistical test
results for means but present sds - Present means for categorical variables(!)
- Present correlational information
8Means on Demographic Var
9Means on Continuous Control Variables
Pre-training Performance Control Experimental
Coworker contact 4.14 4.10
Communication 4.17 4.12
Diversity training is not significantly
correlated with any of these variables
10Training Outcome Measures
- Trainee reactions
- 6-items
- 5-point Likert rating scales
- Reliability.98
- Completed immediately after training
- Only completed by experimental group
- Usefulness of mean data
11Training Outcome Measures
- Cultural Awareness
- Correctly pair nine-terms with their meanings
- Completed 1 year after training
- Previously developed scale called CAI
- Reliability.75
12Training Outcome Measures
- Differential Treatment Ratings
- Coworkers ratings of how Ps treated those who
were culturally different from Ps - 1 year after training
- Previously developed discrimination scale
- 10 items rated on 5 point scale
- Reliability.98
13Means on Outcome Variables
Variable Control Experimental
Cultural Awareness 5.66 6.24
Differential Treatment 1.26 1.44
Diversity training is not significantly
correlated with any of these variables
14Validity of Rater Sample
- No differences between participant and rater
sample on - Proportions of men women
- Proportions of Whites VMs
- Correlation b/w post-training measures and
performance ratings, between supervisor and peer
performance ratings - Did raters know whether target was in the
experimental vs. control group? - Higher expectations
15Preliminary Analyses
- Significant correlations between (control)
demographic variables - Tenure Educational level -.26
- Ethnicity and Educational Level -.26
- Gender Ethnicity -.39
- Coding issues?
- 1male, 0female
- 1White, 0VM
- 1less than 5 years, 521 years or more tenure
- Educational level coding not provided
16Preliminary Analyses
- Significant correlations between pre-training
performance demographic variables - Coworker Contact Communication.50
- Coworker contact gender-.24
- Communication gender-.22
- Communication education.25
- Coding 1male, 0female
17Validity of Outcome Variable
- Trainee reactions not related to any variable
- Usefulness of trainee reactions
- Statistical Power issues
18Validity of Outcome Variable
- Significant correlations with b/w Cultural
awareness control variable - Coworker contact performance .27
- Criterion validity of outcome variable
- Ethnicity.30 1White, 0VM
- Education level.50
19Validity of Outcome Variable
- Significant correlations between Differential
treatment control variables - Gender.20 (1male, 0female)
- Rater ethnicity.30 (1White, 0VM OR 1VM
0White) - But no correlation b/w DT pre-training
performance rating - Implications for
- Using type of raters
- Criterion validity of differential treatment
ratings - Do supervisors have opportunity to notice
differential treatment?
20Hypothesis Testing
- Regression analyses to test for mediation effects
requires - Independent and dependent variable to be related
- Mediator variable to be related to both
independent dependent variables - Criteria not met for
- IVTraining
- MediatorCultural Awareness
- DVDifferential treatment
- BUT.forging ahead!
21Hypothesis Testing of fake data?
Step Predictor R2 ?R2 ß
1 Participant Ethnicity 13 13 -17
Rater Ethnicity 25
2 Training 19 06 25
3 Training 19 0 25
Cultural Awareness 12
3e Rater Ethnicity x Training 26 07 44
22Graph of Interaction
23Discussion
- Lack of support for hypotheses
- Diversity training did not have any effect on
social perception biases - Educational level participant ethnicity
predicted cultural awareness - Trainee reactions were positive(!)
- Uselessness of these types of measures
24Alternative Explanations
- Diffusion of treatment among controls
- Not supported by higher differential treatment
ratings given to trained participants - Selection bias
- Lack of differences on control variables
including pre-training performance ratings - Trainees held to higher standard by non-white
raters
25Alternative Explanations
- Qualitative analyses of interviews with non-white
raters of trainees - Possible backlash due to
- Lack of information re purpose of training
- Timing of post-test need for post-training
support - Pre-training beliefs feelings
- Usefulness of non-white raters who interact
w/diversity trainee
26Learning Points from Article
- Writing up unexpected results
- Presentation of statistical results
- Means vs. frequencies depends on type of variable
- Double check statistical results
- Discrepancy between correlational and regression
tables - Analyses should also be guided by hypotheses