Planned Contrasts and Data Management

1
Planned Contrasts and Data Management
Class 17
2
Topics Covered Today   1. Planned Contrasts   2.
Analysis of Residual Variance 3. Post-hoc
tests   4. Data Management a. Setting up data
files b. Cleaning data   5. Tuesday a.
Interactions in regression b. Hand out ANOVA
exercise
3
  What we won't cover from Keppel Chapter
11         1. Orthogonal comparisons
(pp.210-211) 2. Analysis of trend (pp.
211) 3. Multiple comparisons (pp. 211-
212) 4. Analysis of cell means (pp. 212-222)
4
Planned Contrast Function
1. Factorial ANOVA tests for orthogonal
(perpendicular) interactions.
2. Some studies predict non-orthogonal
interactions.
3. Planned contrast provides more predictive
power to confirm non-orthogonal contrasts of any
particular shape (wedge, arrow like above
or other).
5
Planned Contrast Execution (Conceptual)   1.
Must predict pattern of interaction before
gathering data.   Predict that Democratic women
will be most opposed to gun instruction in
school, compared to Democratic men, Republican
men, and Republican women.
6

• Convert Separate Factors into Single Factor
• 1. Two separate factors
• Political Party Democrat, GOP
• Gender Male, Female
•  
• Convert the two separate factors into a single
  factor
• genparty 1) Male Republican
• 2) Male Democrat
• 3) Female Republican
• 4) Female Democrat

7
Converting Multi-factors into Single Factor for
Planned Contrast
Gender
Converted into single factor with four
levels   GENPARTY   1 Male/Republican 5.00
2 Male/Democrat 4.50 3 Female/Republican
4.75 4 Female/Democrat 2.75
8
Planned Contrast Execution (Conceptual)   3.
Conduct one-way ANOVA, with new single variable
as predictor.   4. Assign weights to the four
levels, as follows   1) Male
Republican -1 2) Male Democrat -1 3)
Female Republican -1 4) Female Democrat 3  
Weights indicate which sub-groups are to be
compared. Weights must add up to zero
5. Planned contrast then limits comparison to
the indicated groups, but counts all subjects
in terms of degrees of freedom and computation of
error. This provides greater predictive power.
9
Graph of Gender X Political Party and Opposition
to Gun Usage in School
10
Univariate Analysis of Variance   DataSet1
Orthogonal Interaction
11
Planned Contrast, Page 1
12
Planned Contrast, Page 2
13
Quality Control for Planned Contrast
Issue Planned contrast can be a very liberal
test, confirming patterns that dont closely fit
with actual predictions.
Result of this 1, -1, -1, 3 planned contrast
is still significant How to assess the quality
of a significant planned contrast?
14
Analysis of Residual Variance
Logic of test Did (Between groups effect
Contrast effect) leave significant amount of
sytematic (non-random) variance unexplained?
If so, then the contrast did not do a good job.
It did not explain the outcome
fully. However, if whats left over (i.e.,
between contrast) is not significant, then the
contrast accounts for most of the treatment.
In this case, the contrast did do a good job.
15
Steps in Analysis of Residual Variance Test

• Get SPSS printout of planned contrast
• Get t of contrast, square it to get contrast F (t
  F )
• Compute SS contrast (SSc) Multiply contrast F by
  mean sq. w/n (MSw) of oneway. This results in SS
  contrast (SSc).
• Compute SS residuals (SSr) Get SS between (SSb)
  from oneway, and subtract SSc. (SSb SSc) SS
  residuals (SSr)
• Compute MS contrast (MSc) Divide SSr by df,
  which is (oneway df planned contrast df). This
  produces the MS contrast (MSc)
• Compute F residuals Divide MSc by MSw. MSc/MSw
  F residuals
• Compute df for F resid numerator df (df
  oneway df contrast see 5a, above), denominator
  df df within (from oneway).
• Check this F in F table from any stats book. If
  significant, contrast is not a good fit. If not
  significant, the contrast is a good fit.

16
Residuals Analysis Test

• 1. Get SPSS printout of planned contrast
• 2. F of contrast (Fcont) t2 t -3.39 t2
  11.49
• 3. SScontrast (SScont) F cont X MSw 11.49 X
  1.04 11.95
• 4. SSresiduals (SSres) SSbetween (SSb) 12.50
• SSb SScont 12.50 11.95 .55
• 5a. Contrast df df oneway df contrast 15
  -12 3
• 5b. MScontrast (MScont) SSres / contrast df
  .55/3 0.18
• F residuals (Fresid) Divide MScont by MSw
  0.18/1.04 .17
• DF for Fresid df contrast (see 5a, above), df
  within (3, 12)
• 8. F table at (3, 12) df, for criterion p lt .25
  F 1.56
• 9. Obtained Fresid lt 1.56, therefore residual is
  not significant, therefore contrast
  result is a good fit for data.

17
Post Hoc Effects   Do female democrats differ
from other groups?   1 Male/Republican 5.00
2 Male/Democrat 4.50 3 Female/Republican
4.75 4 Female/Democrat 2.75
Conduct three t tests? NO. Why not?
Capitalizes on chance
Solution Post hoc tests of multiple
comparisons.   Post hoc tests consider the
inflated likelihood of Type I error   Kent's
favoriteTukey test of multiple comparisons,
which is the most generous.   NOTE Post hoc
tests can be done on any multiple set of means,
not only on planned contrasts.
18
Conducting Post Hoc Tests   1. Recode data from
multiple factors into single factor, as per
planned contrast.   2. Run oneway ANOVA
statistic   3. Select "posthoc tests" option.
ONEWAY gunctrl BY genparty /CONTRAST -1 -1
-1 3 /STATISTICS DESCRIPTIVES /MISSING
ANALYSIS /POSTHOC TUKEY ALPHA(.05).
19
Post hoc Tests, Page 1
20
Post Hoc Tests, Page 2
21
Data Management Issues
Setting up data file   Checking accuracy of
data   Disposition of data     Why obsess on
these details? Murphy's Law   If something can
go wrong, it will go wrong, and at the worst
possible time.
22
Creating a Coding Master
1. Get survey copy   2. Assign variable
names   3. Assign variable values   4. Assign
missing values   5. Proof master for
accuracy   6. Make spare copy, keep in file
drawer
23
Coding Master
24
Cleaning Data Set   1. Exercise in delay of
gratification   2. Purpose Reduce random
error   3. Improve power of inferential stats.
25
Techniques for Cleaning Data
1. Print out data file and visually scan
it.   2. Run descriptives check means, min
and max values, valid cases   3. Run
frequencies   4. Run cross tabs check if
number of subs per condition looks OK   5.
Correlations Check if patterns of data move in
expected direction   6. Multiple data entry
Locate and correct disparities in data entry.
26
Complete Data Set
27
(No Transcript)
28
(No Transcript)
29
Using Cross Tabs to Check for Missing or
Erroneous Data Entry
Case A Expect equal cell sizes
Case B Impossible outcome
30

• Check a Sub Sample
•  
• 1. Determine acceptable error rate
•  
• Number of cases to randomly sample, by rate of
  acceptable
• error rate

31
Storing Data   Raw Data   1. Hold raw data in
secure place   2. File raw data by ID   3.
Hold raw date for at least 5 years post
publication     Automated Data   1. One
pristine source, one working file, one syntax
file   2. Back up, Back up, Back up
32
Sort Raw Data Records
33
COMMENT SYNTAX FILE GUN CONTROL STUDY SPRING
2007 COMMENT DATA MANAGEMENT IF (gender 1
party 1) genparty 1 . EXECUTE . IF (gender
1 party 2) genparty 2 . EXECUTE . IF
(gender 2 party 1) genparty 3 . EXECUTE
. IF (gender 2 party 2) genparty 4
. EXECUTE . COMMENT ANALYSES UNIANOVA
gunctrl BY gender party /METHOD SSTYPE(3)
/INTERCEPT INCLUDE /PRINT DESCRIPTIVE
/CRITERIA ALPHA(.05) /DESIGN gender party
genderparty . ONEWAY gunctrl BY genparty
/CONTRAST -1 -1 -1 3 /STATISTICS DESCRIPTIVES
/MISSING ANALYSIS /POSTHOC TUKEY ALPHA(.05).
34
Cleaning Data Set   1. Exercise in delay of
gratification   2. Purpose Reduce random
error   3. Improve power of inferential stats.