KNR 295

1
Measurement Theory Construct Validity

• Chapter 3

2
Measurement validity construct validity
3
Construct Validity
4
Construct Validity

• Determined by Operationalization

5
Construct Validity

• So construct validity assesses how well your
  procedures/measures match your ideas/theories
• General construct
• Specific operationalization

6
Construct Validity

• Two views
• Definitionalist
• The construct,the whole construct, and nothing
  but the construct
• Impossible!
• Relativist
• Define your construct
• Explain how and why what youre doing measures
  the construct in question
• Produce some evidence (whether we do this depends
  on the kind of research in question)

US
7
Construct Validity

• Translation validity
• Face validity
• Content validity
• Criterion-related validity
• Predictive validity
• Concurrent validity
• Convergent validity
• Discriminant validity

8
Construct Validity

• Translation validity vs. Criterion-related
  validity
• Translation validity assesses whether the
  operationalization matches what you know of the
  construct
• Criterion validity actually measures this
  assessment (uses other measures to assess the
  construct validity)

What we do is translation validity arguing
about the construct validity but not measuring it
directly
9
Construct Validity

• Translation validity
• Face validity
• Does it look like you got it right?
• Ask othersmore objective
• Content validity
• Good definition of the construct
• Good match between your measure and the
  definition
• E.G. Fitness program does it abide by ACSM
  guidelines?

(e.g. constructBody fat. Measuresum of
skinfolds)
10
Construct Validity

• Criterion-related validity all involve some
  direct test of CV
• Predictive validity
• Does it predict what it ought to?
• E.G. Does sum of skinfolds predict cardiovascular
  disease?
• Concurrent validity
• Can your measure discriminate between 2 similar
  groups?
• E.G. Measure sum of skinfolds of males and
  females females should be higher than males(?)

11
Construct Validity

• Criterion-related validity
• Convergent validity
• Correlation between this operationalization and
  other similar ones
• E.G. Sum of skinfolds andBMI, underwater
  weighing, cadaver dissection
• Discriminant validity
• This operationalization is different from other
  stuff that is not supposed to measure the same
  thing
• E.G. Sum of skinfolds vs. age, vs. weight, vs.
  gender, and so on.
• Note this is why these items are included in
  popular equations converting sum of skinfolds to
  BF

12
Construct validity

• Convergent vs. Discriminant Validity

13
Construct validity

• Convergent vs. Discriminant Validity
• Note once youve done this, you still need
  translation validity to establish that the
  measures are what you purport them to be

14
Threats to Construct Validity

• The laundry list
• Use this to ask the right questions about the
  studies you critique
• so to begin

15
Threats to Construct Validity

• Inadequate preoperational explication of
  constructs
• Construct not defined carefully enough

16
Threats to Construct Validity

• Mono-operation bias (independent variable)
• Only one example of the construct
• E.G. only one training programbut there are many
  out there
• Mono-method bias (dependent variable)
• Only one example of the construct
• E.G. only one strength measure for a program that
  trained the whole body

17
Threats to Construct Validity

• Interaction of different treatments
• Use one or more control groups to isolate cause
• E.G. You want to show that strength training
  improves self-esteembut it could have been as a
  result of meeting you 3 times per week, not
  strength trainingso use a control group to
  compare results

18
Threats to Construct Validity

• Interaction of testing and treatment
• Imagine I was interested in whether research
  methods improved reasoning skills
• If I tested you every week on some IQ tests,
  these become part of the treatment, and impair
  construct validity (you might be getting better
  because of the test, not because of being in
  research methods)

19
Threats to Construct Validity

• Restricted generalizability across constructs
• Unintended consequences
• E.G. Finnish epidemiological study
• divide people into groups according to level of
  smoking cholesterol
• Take half of each group and assign to fitness
  nutrition program, half untreated
• All in fitness and nutrition program reduced
  smoking and improved fitness measures
• BUT they also had increased all cause
  mortality!!! (may be an apocryphal story)

20
Threats to Construct Validity

• Confounding constructs and levels of constructs
• Im finding out whether aerobics or strength
  training has the greater impact on muscle tone
• Ok, but what amount of strength training? How
  often are they doing aerobics?
• The labels are not in sufficient detail
• Could be that only the particular versions of
  these programs that you used will produce the
  results you found

21
Threats to Construct Validity

• Social threats to construct validity
• Hypothesis guessing
• I never collect data using people that have
  completed my motor learning class
• Evaluation apprehension
• Experimental booth in Bangor
• Experimenter expectancies
• I love my research

22
Reliability and Levels of Measurement

• Well leave those till we deal with conclusion
  validity
• They are really the concern of a good stats course