Validity

1
Validity

• PSY 614
• Fall 2007
• Instructor Emily E. Bullock, Ph.D.

2
Validity

• Validity how well does a test measure what is
  purports to measure
• Validation process in which the test developer
  and test user may play a role to gather and
  evaluate validity evidence

3
Types of Validity

• Face Validity
• Content Validity
• Construct Validity
• Criterion-Related Validity

4
Face Validity

• Face validity the extent to which a test
  measures what it appears to measure
• Has received little respect in the study of
  validity
• Likely more of an issue to the test taker than
  the test user or developer

5
Face Validity-an example of personality
assessment

• Which of the following activities would you most
  enjoy?
• A. painting a picture alone
• B. going to a big party at which you know few
  people
• C. going to a small party where you will know
  nearly everyone
• D. engaging in an intellectual discussion with a
  group of 3 well-informed people

6
Face Validity-an example of personality
assessment
7
Content Validity

• Content Validity describes a judgment concerning
  how adequately a test samples behavior
  representative of the universe of behavior the
  test was designed to sample
• There is no exact statistical measure of content
  validity. It is typically determined by expert
  judgment of test content.

8
Determining Content Validity

• Typical procedure (pg158 of text)
• Describe content domain
• Determine areas of the content domain that are
  measured by each test item
• Compare the structure of the test with the
  structure of the content domain
• Lawshe technique of determining content validity
• Content validity in employment settings
• What type of information does a measure of
  content validity not seem to regularly take into
  account?

9
Construct Validity

• Construct validity judgment about the
  appropriateness of inferences drawn from test
  scores regarding individual standings on a
  variable called a construct.
• Construct represent ideas constructed by
  scientists to help summarize a group of related
  phenomena or objects (see text pg 163)
• Two essential properties
• Abstract summaries of some regularity in nature
• Related to or connected with concrete, observable
  entities or events

10
Determining Construct Validity

• Evidence of homogeneity The test is homogeneous,
  measuring a single construct
• Test score increase or decrease as a function of
  age or the passage of time as theoretically
  predicted
• Test scores obtained subsequent to some event or
  to the mere passage of time differ from pretest
  scores as theoretically predicted
• Method of contrasted groups Test scores obtained
  by people from distinct groups vary as predicted
  by theory
• Test scores correlate with scores on other tests
  in accordance with what would be predicted from a
  theory
• ? Convergent validity ? Factor Analysis
• ? Discriminate validity

11
Criterion-Related Validity

• Criterion-Related Validity a judgment regarding
  how adequately a test score can be used to infer
  an individual's most probably standing on some
  measure of interest
• Criterion the measure of interest the standard
  against which a test or a test score is evaluated
• Criterion Contamination a situation where the
  criterion measure itself has been based, at least
  in part, on predictor measures

12
Two Types of Criterion-Related Validity

• Concurrent Validity test scores are obtained at
  about the same time that the criterion measures
  are obtained
• Predictive Validity test scores obtained at one
  time and the criterion measure obtained a future
  time after some intervening event has taken place

13
More on Criterion-Related Validity

• Validity coefficient
• Incremental validity degree to which an
  additional predictor explains something about the
  criterion measure not explained by predictors
  already in use

14
More on Criterion-Related ValidityDecision Theory

• Base Rate extent to which a particular attribute
  exists in the population
• Hit Rate proportion of people a test accurately
  identifies as possessing or exhibiting a
  particular attribute
• Miss Rate proportion of people the test fails to
  identify as having or not having a particular
  attribute
• False Positive miss wherein the test predicted
  that the test taker did posses the particular
  attribute
• False Negative miss wherein the test predicted
  that the test taker did not possess the
  particular attribute