Seth M' Noar, Ph'D'

1
Basics of Research Design

• Seth M. Noar, Ph.D.
• CJT665
• Department of Communication
• University of Kentucky

2
2 Broad Types of Research

• Surveys Observations that occur in natural
  settings and which involve a minimum amount of
  interference
• Often operate in real world settings
• But, causality is difficult to demonstrate
• Experiments Observational studies in which data
  are collected under conditions where variables
  are manipulated
• Often operate in an artificial setting
• But, causality is easier to demonstrate

3
2 Broad Types of Research (contd)

• These two types are complementary in many ways
  ideal program of research takes advantage of both
• Research design similar to statistics should
  choose the best and most appropriate research
  design
• However, resources and other factors may help to
  dictate what design is feasible
• Large randomized trials cost a lot of money
• Some perfect experiments are unethical to
  conduct

4
Experiments

• True experiment involves
• IV manipulated variable
• DV variable expected to be influenced by the
  manipulation of the IV
• Pool of participants who are randomly assignment
  to groups (e.g., control and experimental)
• Note Random assignment distinguishes
  experimental from quasi-experimental designs

5
Experimental Designs

• 5 Types of Designs
• Pretest-Posttest Control Group Design
• Posttest Only Control Group Design
• Solomon Four Group Design
• Factorial Designs
• Blocking Designs

6
Pretests

• Purpose of pretest Demonstrate that
  randomization worked that groups are equivalent
  at baseline
• Also, if groups are different in some way, we can
  statistically control for this since we have
  pretest data
• Only problem pretest may sensitize participants
  to the IV manipulation
• In some cases then, no pretest is a better design

7
(No Transcript)
8
(No Transcript)
9
Solomon 4 Group Design

• Combines the previous 2 designs together
• Can evaluate several effects within 1 experiment
• Allows one to examine effects of the IV
  manipulation (treatment)
• Allows one to see if the pretest had an effect
• Allows one to examine effects of the IV
  manipulation (treatment) above and beyond any
  pretest effects (interaction)
• Note one is essentially treating the pretest as
  an additional IV

10
(No Transcript)
11
Solomon as 2 x 2 Factorial
Pretest
No Pretest
Treatment
Treatment Main Effect
No Treatment
Pretest Main Effect
12
Factorial Designs

• Factorial Designs involve more than 1 IV
• For example, a 2 X 2 design
• Sensation Value Low, High
• Need for cognition Low, High
• If we wanted 50 subjects in each group, we would
  need 50 x 2 x 2 200 subjects

13
2 x 2 Factorial
High NFC
Low NFC
High SV
Low SV
14
Blocked / Mixed Designs

• Blocked or mixed designs are appropriate when we
  have factors we are interested in that we cannot
  manipulate, and / or sample size is very small
• Examples IQ, Gender, Race
• Key feature is this participants are sorted into
  categories (blocked) according to some variable
  and then are assigned to groups

15
(No Transcript)
16
Validity

• 2 major types of validity in experiments
• Internal validity extent to which one can make
  causal inferences about the IV-DV relationship
• External validity The extent to which one can
  generalize findings of the study to the
  population and to other contexts
• These are generally mutually exclusive, and have
  mutually exclusive threats

17
Threats to Internal Validity

• History An event takes place between pre and
  post test (MM example)
• Maturation Individuals mature in some way
  between pre and post test (Anti-marijuana
  example)
• Testing Scores on post test change because of
  exposure to pretest
• Instrumentation Scores on post test change
  because of unreliable measurement (not actual
  change). (This happens more with subjective
  measures).
• Statistical Regression Unreliability or
  measurement error results in scores away from the
  mean at time 1, and then back to the mean at time
  2

18
Threats to Internal Validity (contd)

• 6. Selection If posttest scores differ
  across groups, this may be due to selection
  factors that went into creating the groups
  (gender diffs w/ college students)
• 7. Experimental Mortality Differences that
  occur in the posttest that are due to
  differential attrition across groups
• 8. Selection Interactions If selection
  factors went into creating the groups, these
  specially selected groups may encounter
  differential history, maturation, testing, or
  other effects.
• Note These threats, when they occur, make it
  very difficult to tell if it was in fact the IV
  that lead to changes in the DV

19
Threats to External Validity

• Key issue here is generalizability extent to
  which findings can be expected to be replicated
  with other samples, at other times and places,
  and under other conditions
• Thus, non-random sampling, interactions between
  IVs and the context in which they are delivered,
  and other factors can affect external validity
• Often we do NOT have true random samples
• Best we can do is attempt to replicate our
  findings in many samples as they become available