Validity (continued) Reliability

1
Validity (continued) Reliability
2
External Validity

• How far can you generalize your results beyond
  your experiment?
• 3 types
• Population
• Environmental
• Temporal

3
Threats to External Validity

• Testing/Treatment Interaction
• Pretest may make reaction to the IV different
  from those not tested
• Selection/Treatment Interaction
• Effect is found only for a specific group of
  participants

Experimental setting alters participants behavior
4
Threats to External Validity

• Reactive Arrangements
• A la Hawthorne effects
• Demand characteristics
• Participant Characteristics
• Specific animal species

Experimental setting alters participants behavior
5
(No Transcript)
6
Threats to External Validity

• Reactive Arrangements
• A la Hawthorne effects
• Demand characteristics
• Participant Characteristics
• Specific animal species
• Experimental participant populations
• College students
• White male Americans

Experimental setting alters participants behavior
7
Is External Validity Necessary?
Why might we conduct research if were not trying
to predict real-life behavior in the real world?

• May want to find out if something can happen
• We may be predicting from the real world to the
  lab
• If we can show something happens in the labs
  unnatural setting, we may have more confidence in
  the phenomenon
• We may study something without a real-world
  analogy

8
To review

• Method-Related Concerns
• Internal Validity
• External Validity
• Measure-Related Concerns
• Construct Validity
• Reliability

9
Reliability

• The extent to which a test is consistent in its
  evaluation of the same individuals

10
Types of ReliabilityInter-rater reliability

• Assesses the degree to which different
  raters/observers give consistent estimates of the
  same phenomenon
• agreement
• Correlation between observers scores

11
Types of ReliabilityTest-retest

• Correlation between two observations (i.e., set
  of scores) on the same test administered to the
  same (or similar) sample on two different
  occasions
• Assumes no change in construct being measured
• Time between observations is crucial

12
Types of ReliabilityParallel-forms

• Correlation between two observations (i.e., set
  of scores) on parallel forms of a test
  administered to the same sample
• Requires generation of many items that measure
  the same construct
• Two forms of test can be used independently
• Assumes randomly divided halves are equivalent

13
Types of ReliabilityInternal consistency

• Single test administered to a sample on one
  occasion
• Assesses the consistency of the results for
  different items for the same construct within the
  measure
• Average inter-item correlation
• Average item-total correlation
• Split-half reliability
• Cronbachs Alpha

14
Relationship between reliability and validity