Schmidt

1
Schmidt Hunter Approach to r

• Bare Bones

2
Statistical Artifacts

• Extraneous factors that influence observed effect
• Sampling error
• Reliability
• Range restriction
• Computational error
• Dichotomization of variables
• addressed in the (bare-bones) analysis

3
Bare Bones r

• Find weighted mean and variance
• Note sample size weight.
• Note that for unit weights, the weighted variance
  estimator is the sample, not population, estimate.

4
Confidence Interval for Mean
There are k studies, with Ni observations.
This is not the only formula they use, but its
the best one IMHO.
5
Estimated Sampling Error Variance

• The variance of r

Estimated variance for a study.
Estimated sampling variance for a meta-analysis.
Note mean r is constant. This is the variance of
sampling error we expect if all the studies have
a common effect estimated by r-bar.
6
Variance of Rho
Classical Test Theory
Sampling Error
A definition
7
Estimated Variance of rho
Note that the variance of rho will be called
tau-squared by Hedges
-
To find the variance of infinite-sample
correlations, find the variance of r in the
meta-analysis and subtract expected sampling
error variance. Schmidt would be quick to add
that part of the estimated variance of
infinite-sample correlations is artifactual.
8
Credibility Interval
The credibility interval and the confidence
interval are quite different things. The CI is a
standard statistical estimate (intended to
contain rho, or average of rho). The CR is
intended to contain a percentage of the values of
a random variable infinite-sample effect sizes.
The SH value forgets that there is also
uncertainty in the mean value the two should be
added. There are Bayesian programs that will do
this there is also an approximation called the
prediction interval described in Borenstein et al.
9
Bare-Bones Example (1)
Study Ni r
1 200 .20
2 100 .20
3 150 .40
4 80 .40
Mean 132.5 .30
lt- Unit weighted mean
10
Bare-Bones Example (2)
r Ni rNi
.20 200 40
.20 100 20
.40 150 60
.40 80 32
sum 530 152
11
BB Example (3)
Recall unwighted or unit weighted mean .30. Why
are they different?
12
BB Example (4)
r Ni
.20 200 1.507
.20 100 .753
.40 150 1.922
.40 80 1.025
Sum 530 5.208
13
BB Example (5)
14
Interpretation

• Schmidt says this is a random-effects
  meta-analysis. It uses a sample of studies to
  represent a larger population of studies.

• People interpret the Credibility Interval, but
  typically do not recognize that it is poorly
  estimated.