Non-Experimental designs

1
Non-Experimental designs

• Psych 231 Research Methods in Psychology

2
Non-Experimental designs

• Sometimes you just cant perform a fully
  controlled experiment
• Because of the issue of interest
• Limited resources (not enough subjects,
  observations are too costly, etc).
• Surveys
• Correlational
• Quasi-Experiments
• Developmental designs
• Small-N designs
• This does NOT imply that they are bad designs
• Just remember the advantages and disadvantages of
  each

3
Developmental designs

• Used to study changes in behavior that occur as a
  function of age changes
• Age typically serves as a quasi-independent
  variable

• Three major types
• Cross-sectional
• Longitudinal
• Cohort-sequential

Video lecture (10 mins)
4
Developmental designs

• Cross-sectional design
• Groups are pre-defined on the basis of a
  pre-existing variable
• Study groups of individuals of different ages at
  the same time
• Use age to assign participants to group
• Age is subject variable treated as a
  between-subjects variable

5
Developmental designs

• Cross-sectional design

• Advantages
• Can gather data about different groups (i.e.,
  ages) at the same time
• Participants are not required to commit for an
  extended period of time

6
Developmental designs

• Cross-sectional design

• Disadvantages
• Individuals are not followed over time
• Cohort (or generation) effect individuals of
  different ages may be inherently different due to
  factors in the environment
• Are 5 year olds different from 15 year olds just
  because of age, or can factors present in their
  environment contribute to the differences?
• Imagine a 15yr old saying back when I was 5 I
  didnt have a Wii, my own cell phone, or a
  netbook
• Does not reveal development of any particular
  individuals
• Cannot infer causality due to lack of control

7
Developmental designs

• Longitudinal design

• Follow the same individual or group over time
• Age is treated as a within-subjects variable
• Rather than comparing groups, the same
  individuals are compared to themselves at
  different times
• Changes in dependent variable likely to reflect
  changes due to aging process
• Changes in performance are compared on an
  individual basis and overall

8
Longitudinal Designs

• Example
• Wisconsin Longitudinal Study (WLS)
• Began in 1957 and is still on-going (50 years)
• 10,317 men and women who graduated from Wisconsin
  high schools in 1957 (and randomly selected
  brothers and sisters, and spouses too)
• Originally studied plans for college after
  graduation
• Now it can be used as a test of aging and
  maturation
• Data collected in
• 1957, 1964, 1975, 1992,
• 2003, 2011

9
Developmental designs

• Longitudinal design

• Advantages
• Can see developmental changes clearly
• Can measure differences within individuals
• Avoid some cohort effects (participants are all
  from same generation, so changes are more likely
  to be due to aging)

10
Developmental designs

• Longitudinal design

• Disadvantages
• Can be very time-consuming
• Can have cross-generational effects
• Conclusions based on members of one generation
  may not apply to other generations
• Numerous threats to internal validity
• Attrition/mortality
• History
• Practice effects
• Improved performance over multiple tests may be
  due to practice taking the test
• Cannot determine causality

11
Developmental designs

• Cohort-sequential design

• Measure groups of participants as they age
• Example measure a group of 5 year olds, then the
  same group 10 years later, as well as another
  group of 5 year olds
• Age is both between and within subjects variable
• Combines elements of cross-sectional and
  longitudinal designs
• Addresses some of the concerns raised by other
  designs
• For example, allows to evaluate the contribution
  of cohort effects

12
Developmental designs

• Cohort-sequential design

Time of measurement
1975
1985
1995
Cohort A
1970s
Cohort B
1980s
Cohort C
1990s
13
Developmental designs

• Cohort-sequential design

• Advantages
• Get more information
• Can track developmental changes to individuals
• Can compare different ages at a single time
• Can measure generation effect
• Less time-consuming than longitudinal (maybe)
• Disadvantages
• Still time-consuming
• Need lots of groups of participants
• Still cannot make causal claims

14
Non-Experimental designs

• Sometimes you just cant perform a fully
  controlled experiment
• Because of the issue of interest
• Limited resources (not enough subjects,
  observations are too costly, etc).
• Surveys
• Correlational
• Quasi-Experiments
• Developmental designs
• Small-N designs
• This does NOT imply that they are bad designs
• Just remember the advantages and disadvantages of
  each

15
Small N designs

• What are they?
• Historically, these were the typical kind of
  design used until 1920s when there was a shift
  to using larger sample sizes
• Even today, in some sub-areas, using small N
  designs is common place
• (e.g., psychophysics, clinical settings, animal
  studies, expertise, etc.)

16
Small N designs

• In contrast to Large N-designs (comparing
  aggregated performance of large groups of
  participants)
• One or a few participants
• Data are typically not analyzed statistically
  rather rely on visual interpretation of the data

17
Small N designs
Steady state (baseline)

• Observations begin in the absence of treatment
  (BASELINE)
• Then treatment is implemented and changes in
  frequency, magnitude, or intensity of behavior
  are recorded

18
Small N designs
Transition steady state
Steady state (baseline)
Treatment introduced

• Baseline experiments the basic idea is to show
• when the IV occurs, you get the effect
• when the IV doesnt occur, you dont get the
  effect (reversibility)

19
Small N designs
Unstable
Stable

• Before introducing treatment (IV), baseline needs
  to be stable
• Measure level and trend
• Level how frequent (how intense) is behavior?
• Are all the data points high or low?
• Trend does behavior seem to increase (or
  decrease)
• Are data points flat or on a slope?

20
ABA design
Steady state (baseline)
Transition steady state
Reversibility

• ABA design (baseline, treatment, baseline)

• The reversibility is necessary, otherwise
• something else may have caused the effect
• other than the IV (e.g., history, maturation,
  etc.)

• There are other designs as well (e.g., ABAB see
  figure13.6 in your textbook)

21
Small N designs

• Advantages
• Focus on individual performance, not fooled by
  group averaging effects
• Focus is on big effects (small effects typically
  cant be seen without using large groups)
• Avoid some ethical problems e.g., with
  non-treatments
• Allows to look at unusual (and rare) types of
  subjects (e.g., case studies of amnesics, experts
  vs. novices)
• Often used to supplement large N studies, with
  more observations on fewer subjects

22
Small N designs

• Disadvantages
• Effects may be small relative to variability of
  situation so NEED more observation
• Some effects are by definition between subjects
• Treatment leads to a lasting change, so you dont
  get reversals
• Difficult to determine how generalizable the
  effects are

23
Small N designs

• Some researchers have argued that Small N designs
  are the best way to go.
• The goal of psychology is to describe behavior of
  an individual
• Looking at data collapsed over groups looks in
  the wrong place
• Need to look at the data at the level of the
  individual