Experimental Design

1
Experimental Design

• Chapter 1
• Research Strategies and the Control of Nuisance
  Variables

2
Introduction

• Experimental design
• A plan for assigning subjects to experimental
  conditions and the statistical analysis
  associated with the plan
• Identifies the independent, dependent, and
  nuisance variables and indicates the way in which
  the randomization and statistical aspects of an
  experiment are to be carried out

3
Formulation of Plans for Collection and Analysis
of Data

• Acceptable research hypotheses
• Distinction between independent and dependent
  variables
• Selection of independent variable
• Quantitative independent variables
• Qualitative independent variables
• Selection of the dependent variable
• Multivariate designs
• Nuisance variables
• Bias

4
Research Strategies

• Experiments
• Quasi-experiments
• Surveys
• Case studies
• Naturalistic observation

5
Other Research Strategies

• Ex post facto studies
• Retrospective and prospective studies
• Longitudinal and cross-sectional studies
• Longitudinal-overlapping and time-lag studies
• Time-series and single-subject studies

6
Threats to Valid Inference Making

• Four categories of threats to valid inference
  making
• Statistical conclusion validity
• Internal validity
• Construct validity of causes and effects
• External validity

7
Threats to Statistical Conclusion Validity

1. Low statistical power
2. Violated assumptions of statistical tests
3. Fishing for significant results and the error
   rate problem
4. Reliability of measures
5. Reliability of treatment implementation
6. Random irrelevancies in the experimental setting
7. Random heterogeneity of respondents

8
Threats to Internal Validity

• History
• Events other than the treatment that occur
  between the time the treatment level is assigned
  and the DV is measured
• Maturation
• Processes not related to treatment that occur
  within the subject
• Testing
• Repeated testing may result in familiarity
• Instrumentation
• Changes in calibration of a measuring instrument

9
Threats to Internal Validity

• Statistical regression
• Occurs when the measurement of the DV is not
  perfectly reliable
• Selection
• Prior differences in subjects
• Mortality
• Loss of subjects
• Interactions with selection
• Selection-history effects or Selection-maturation
  effects

10
Threats to Internal Validity

• Ambiguity about the direction of causal influence
• Diffusion or imitation of treatments
• If the subjects in different levels can
  communicate, differences may be compromised
• Compensatory rivalry by respondents receiving
  less desirable treatments
• Resentful demoralization of respondents receiving
  less desirable treatments

11
Threats to External Validity

1. Interaction of testing and treatment
2. Interaction of selection and treatment
3. Interaction of setting and treatment
4. Interaction of history and treatment
5. Reactive arrangements
6. Multiple-treatment interference

12
Other Threats to Valid Inference Making

• Experimenter-expectancy effect
• Demand characteristics
• Subject-predisposition effects
• Cooperative-subject effect
• Screw you effect
• Evaluation apprehension
• Faithful subjects
• Placebo effect

13
Controlling Nuisance Variables

• General Approaches to Control
• Experimental control
• Statistical control
• Some Specific Approaches to Control
• Blind procedures
• Deception
• Unobtrusive experimentation
• Multiple researchers

14
Controlling Nuisance Variables

• Some Specific Approaches to Control
• Debriefing
• Experimenter-expectancy control groups
• Unrelated-experiment technique
• Quasi-control group
• Yoked control procedure

15
Ethical Treatment of Subjects

1. Responsibility of researcher
2. Informed consent
3. Protection from harm
4. Protection of rights
5. Research deception
6. Confidentiality
7. Debriefing
8. Accurate and reporting