Measures

1
Measures
Introduction to the Principles and Practice of
Clinical Research

• Audrey Thurm, Ph.D. Pediatric and Developmental
  Neuropsychiatry, NIMH

With thanks to Daniel Pine, Erin McClure F.
Xavier Castellanos David Rubinow
2
The Uncertainty Principle

• The more precisely the POSITION is determined,
  the less precisely the MOMENTUM is known.
• 
  Werner Heisenberg

3
Outline Operational Questions

• 1. What is a measure/variable?
• 2. How do you measure?
• 3. How do you test measures (Reliability/Validity)
  ?
• 4. How do you select measures?
• 5. What factors influence the measures?
• 6. When is the measure obtained and how often?

4
What is a measure?

• Something that varies in an observable,
  quantifiable fashion
• Measure true value error
• Error random error non-random (biased) error
• How to minimize random error? Central limit
  theorem http//www.stat.sc.edu/west/javahtml/CLT.
  html
• http//www.math.uah.edu/stat/sample/sample5.ht
  ml

5
Types of error (random non-random)

• Non-random error (bias) ? Type I error
  (false positives)
• Would you vote for this moderately intelligent
  candidate?
• Random error ? Type II error (low power)
• What will you do on November 2nd?

6
Central Limit Theorem in ActionSum of All Rolled
Dice
Two Dice
One Die
Three Dice
Four Dice
7
Types of Scales(What You Measure With)

• Nominal or Categorical
• Classification or set of categories mutually
  exclusive and collectively exhaustive e.g.,
  gender sick vs. healthy
• Ordinal
• Mutually exclusive classes that form an ordered
  series rank order e.g., grades on a statistics
  test seriousness of a tumor
• Interval
• Ordered series of ranks with equal intervals
  between any two pairs of adjacent classes e.g.,
  temperature
• Ratio
• An interval scale with a true zero point origin
  e.g., weight

8
Categorical/Nominal Measures

• hanging chad only one corner of the almost
  completely punched-out piece of paper is still
  connected to the ballot
• swinging chad 2 corners remain attached,
  resembling a door
• tri-chad 3 corners remain attached and only one
  corner is flapping outward, but some pushed-away
  space is evident
• pregnant chad the rectangle has a bulge and
  seems to have been lightly punched, but all four
  corners are still attached
• dimpled chad the space has a slight
  indentation, but the corners remain connected

Gannett News Service
9
Categorical/Nominal Measures

• Classification or set of categories mutually
  exclusive and collectively exhaustive
• Statistical operation Counting
• Caseness
• Criteria lifetime vs. current symptoms
• Comorbid conditions
• Subthreshold non-cases
• Fundamental process in epidemiology

10
Ordinal Scales

• e.g., Rating Scales, IQ
• Subjectivity
• Floor/Ceiling effects common

11
Implications of Scale Type

• Determine which statistical operations are
  permissible
• Parametric
• Non-parametric
• Within subject comparisons
• Between subject comparisons

12
How Do You Test Measures?

• Reliability
• The consistency with which a measure assesses a
  given trait i.e., agreement between two measures
  obtained by the same or maximally similar methods
• Validity
• The extent to which a measure actually measures a
  trait i.e., agreement between two measures
  obtained by maximally different methods

13
Types of Reliability

• Test-retest
• Temporal stability of a test from one measurement
  session to another
• Internal-consistency reliability
• Also known as reliability of components average
  of the intercorrelations of single test items
  these coefficients go up as the number of test
  items increase

14
On the methods and theory of reliabilityJ J
Bartko W T Carpenter J Nerv Ment Dis 1976,
163307-314

• Unsuitable methods
• Percent agreement
• Chi-square
• Correlation
• Suitable methods
• Kappa
• Intraclass correlation coefficient (ICC)

15
Diagnostic Reliability - Agreement
Expected Frequencies
Bartko Carpenter J Nerv Ment Dis 1976,
163307-314
16
Types of Validity

• Face (Content) Validity
• Right items performance (or response) free of
  influence of irrelevant variables
• Criterion-related Validity
• Comparison with independent, direct measures
• Construct Validity
• Measurement of the theoretical construct

17
How Do You Select Measures?

• What is the variable of interest?
• What are the dependent and independent variable
  domains?
• Disorder dependent variables
• e.g., symptoms, side effects, biochemical indices
• Disorder independent variables
• e.g., life events

18
Measure Selection

• What are the variable characteristics?
• Severity vs. frequency, vs. both?
• Absolute value vs. change from baseline
• Symptom vs. syndrome
• Current vs. past
• What are the study characteristics?
• Cross-sectional vs. longitudinal
• Treatment (baseline and outcome)

19
Analog Scales

• Symptom specific
• Ease of completion
• Sensitive
• Replicable
• Ease of data entry and analysis

10 cm
Anchor point
Anchor Point
20
Effect of Endpoint Labels
Schwartz et al., 1991 Public Op Q 570-582
21
Effects of Response Alternatives
Schwartz et al., 1985, Public Opinion Q 49
388-395
22
Assessing the Measure

• Examine data that tests the instrument
• Range
• Sensitivity
• Proportion of true cases identified as true
• (minimizes false negatives)
• Specificity
• Proportion of false cases identified as false
• (minimizes false positives)

23
Screening Measures
Table describes screening test outcomes
Disease present
Disease absent
Group (a) True Positive
Group (b) False Positive
Positive result
Group (c) False Negative
Group (d) True Negative
Negative result
24
Assessing the Measure (II)

• How long and how often should the measure be
  applied?

25
What Factors Influence the Measure?

• Performance variables
• Skill and care
• Practice, floor ceiling effects
• Typographical errors
• Test conditions
• Insensitivity in range of interest
• Infinite unknown factors ? RANDOMIZE

26
Measures - Summary

• Choose your measures carefully
• Know their weaknesses
• Reliability before validity
• Beware of biased error above all
• Central Limit Theorem

27
Useful on-line Statistics Primer

• Hopkins, W. G. (1997). A new view of
  statistics Internet Society for Sport Science
• http//www.uq.oz.au/hmrburge/stats/index.html
• (book) Rosenthal Rosnow (1991). Essentials of
  Behavioral Research.