Education Testing

1
Education Testing

• SAT, GRE, LSAT, MCAT

2
Individual versus Group Tests
3
Origins of Mental Testing in College Admissions

• Essay entrance exams on Latin, Greek, and Ancient
  History
• SAT introduced in 1920s
• Carl Brigham
• He concluded based on IQ scores that Asians and
  Jews less gifted than Anglo-Saxons

4
The Scholastic Assessment Test (SAT)

• SAT I (Reasoning)
• 3 hour test of mathematical and verbal skills
• SAT II (Subject Tests)
• 1 hour test of knowledge in a subject area and
  ability to apply the knowledge

5
Predictive Validity of SAT I

• Freshman GPA most frequently used criterion
• freshmen courses more similar and less variable
  than at any other year in college
• predictor and criterion data readily available
• freshmen grade averages highly correlated with
  cumulative grade averages

6
Predictive Validity of SAT I
All tables including this one from UC and the
SAT Predictive Validity and Differential Impact
of the SAT I and SAT II at the University of
California UC Office of the President
7
Predictive Validity of SAT I
8
Predictive Validity of SAT I
9
Predictive Validity of SAT I
10
Predictive Validity of SAT I
11
Predictive Validity of SAT I
12
Predictive Validity of SAT I
13
(No Transcript)
14
Graduate Record Exam (GRE)

• General Test Verbal, Quantitative, and
  Analytical Writing sections
• Subject Test
• Predictive Validity
• Meta-analysis (Ingram, 1980)
• Verbal scores and GPA 25
• Quantitative scores and GPA 12.5
• Yale study
• Postgraduate First Year GPA Verbal r .17,
  Quant r .29
• Veterinary students
• College GPA and Postgraduate GPA r .55
• Postgraduate GPA and Verbal r .41
• Postgraduate GPA and Quant r .47

15
Law School Admissions Test (LSAT)

• Reading comprehension, logical reasoning (½ of
  the items), analytical reasoning
• Electrons orbit around the nucleus of an atom the
  way the earth orbits around the sun. It is well
  known that gravity is a major force that
  determines the orbit of the earth. We may,
  therefore, expect that gravity is the main force
  that determines the orbit of an electron.
• The argument attempts to prove its cause by
• Applying well-known general laws to a specific
  case
• Appealing to well known specific cases to prove a
  general law about them
• Testing the conclusion by a definite experiment
• Appealing to an apparently similar case
• Stating its conclusions without giving any kind
  of reason to think that it might be true

16
Law School Admissions Test (LSAT)

• Writing sample
• 6.8 "never" use the writing sample, 9.9
  "always" use the sample 25.3 frequently" use
  the sample 32.7 "occasionally and 25.3
  reported "seldom
• Can take the test up to 3 times in a two-year
  period

17
Medical College Admissions Test (MCAT)

• Physical Sciences, Verbal Reasoning, Writing
  Sample, Biological Sciences
• Intellimetric
• Like GRE, may void score before seeing the
  results
• Can take it up to 3 times per year
• About 50 of those taking it use a prep service

18
Are Education Tests Biased?

• 1000 Specialists Surveyed (Snyderman Rothman,
  1988)
• 65 response rate
• Are tests biased against African Americans and
  lower socioeconomic levels?
• 2.10-2.25 (SD.8)
• 73 favored intelligence tests and aptitude tests
  in hiring
• 82 or more favored use of SAT, ACT, LSAT, GMAT,
  MCAT, or GRE

19
Views on Detecting Test Bias

• Test bias exists if there are mean differences
  (Larry P. vs. Riles, 793 F. 2d 969 (9th Cir.
  1984))

20
Views on Detecting Test Bias

• Test bias exists if test systematically predicts
  less accurately for individuals belonging to one
  group versus another
• slope bias
• SAT 1250 predicts 3.1 for majority and minority
  group, but SAT of 1000 predicts 2.6 for majority
  and 2.3 for minority
• intercept bias
• Slopes are the same, but SAT 1250 predicts 3.1
  for majority and 2.9 for minority
• prediction errors differ for majority and
  minority
• Spread of scores around regression line differs
• SAT 1250 predicts 3.1 .4 for majority and 3.1
  .9 for minority