Revised 1/26/09

1
Revised 1/26/09
Right click for full Screen or end show.
Left click to proceed,
Psychology 242, Dr. McKirnan
Lectures 3 Developing research questions and
hypotheses. Basic experimental design internal
validity.
2
Research questions, hypotheses designs

• How do social values affect science?
• Framing a research question Where do hypotheses
  come from?
• The scientific process How do we get from a
  theory to actual data (and back)?
• How do we create an Independent Variable?
• What are the basic types of experimental designs?

3
The big picture Introductory lectures
?

• How do social values affect science?
• Framing a research question Where do hypotheses
  come from?
• The scientific process How do we get from a
  theory to actual data (and back)?
• How do we create an Independent Variable?
• What are the basic types of experimental designs?

4
The role of values in the scientific process.
Research begins ends with value judgments
5
Values, theory and data in the scientific process.

• Social values help define a scientific problem
  or question.
• Norms, values ( data) determine what is credible
  / fundable.

• Theory is influenced by norms empirical
  background of field.

• Science hinges on clear, objectively stated
  hypotheses.
• Clear hypotheses lessen bias in interpreting
  results.

• Methods analyses are most objective, but
  fields vary in methodological rigor.

• The meaning of a finding is influenced by
  cultural social values or concerns.
• for science and, particularly, for society.

6
Values and science The internet and sexual risk,
1.

• Social values help define a scientific problem
  or question.
• Norms, values ( data) determine what is credible
  / fundable.

• Sexual internet use became a large issue for
  gay/bisexual men in the 2000s
• Internet use exploded, and quickly became a
  sexual venue for all social groups.
• Men who have sex with Men MSM seem particularly
  likely to be risky via internet sex.
• The HIV prevention field needed to understand how
  this worked.
• Politically HIV has made research on MSM a
  legitimate and important topic

7
Values and science Internet sexual risk, 2.

• Theory is influenced by norms empirical
  background of field.

• Several existing theories may help us explain the
  phenomenon
• There are a lot of data showing both rising C02
  concentrations and global warming.
• Models of fossil fuel use make human influence
  theoretically possible.

8
Values and science climate change, 1.

• Science hinges on clear, objectively stated
  hypotheses.
• Clear hypotheses lessen bias in interpreting
  results.

• Global warming emerged as a key research topic
  in the 2000s.
• Political liberals made climate change a key
  component of environmental activity
• Evidence from several fields made climate change
  a credible topic
• New technology monitoring equipment, computer
  models allowed scientists to better capture
  climate changes

9
The big picture Introductory lectures

• How do social values affect science?
• Framing a research question Where do hypotheses
  come from?
• The scientific process How do we get from a
  theory to actual data (and back)?
• How do we create an Independent Variable?
• What are the basic types of experimental designs?

?
10
Framing a research question Hypotheses

• 1. We specify a functional relationship between
  variables.
• Variables have multiple values
• Functional relationships

2. We cast the relationship in the form of a
prediction.
If X then Y

•  3. We ensure the problem is testable
• Feasible to study
• The theory / hypothesis can potentially be
  disconfirmed

11
Framing a research question, 2.

•  4. We operationally define our basic terms.
•  5. We express our study variables on a
  measurement scale
• Core dimensions
• qualitative v. quantitative
• continuous v. binary
• linear (raw numbers) v. non-linear (logs,
  exponents)
• Quantitative scale types
• grouping / categorical
• ordinal / rank order
• Interval
• ratio

12
Sources of research questions

• Practical / applied questions
• Describe or explain an important social process
• Evaluate an intervention or policy change
• Unanswered questions from previous research
• Clarify conflicting / unclear findings
• e.g., additional research on diet (week 2
  reading)
• e.g., who benefits from a medical procedure
• (week 7 reading on mammography)
• Generalizability of previous findings
• Across populations
• Do findings among men generalize to women
• (Week 5 reading on negative results)
• Across different predictors outcomes
• Do conceptual models of health behavior
  generalize to recycling and energy conservation?

13
Sources of research questions, 2
Testing theories

• Use existing theory to explain a new phenomenon
• Do sensation seeking / risk taking theories of
  drug use explain unsafe sex in adolescents or gay
  men?
• Test contrasting theories of a phenomenon
• How much is adolescent problem behavior
  controlled by psychological variables
  (depression) versus peer influence?
• Develop new / expanded theory
• Develop test social cognitive theory to explain
  peoples storage and use of speech style
  information
• Key issues Hypothetical constructs
• Operational definitions
• Statement of testable hypothesis

14
Question 1
A hypothetical construct is
A A specific prediction about the outcome of an
experiment B A little known band from Muncie
Indiana C A general ? process that underlies
our observations D A central element in a theory
15
Question 2
To be testable, a hypothesis
Must rest on operational definitions. A true B
false C I dont know
16
Question 3
To be testable, a hypothesis
Must potentially be found to be false. A
true B false C I dont know
17
The big picture Introductory lectures

• How do social values affect science?
• Framing a research question Where do hypotheses
  come from?
• The scientific process How do we get from a
  theory to actual data (and back)?
• How do we create an Independent Variable?
• What are the basic types of experimental designs?

?
18
The research process

• Phenomenon

Overall issue or question What controls
emotional states? Why are some people vulnerable
to depression?
Theory
Possible explanation How it works
statement Several theories may help explain the
phenomenon
Theory 1 Emotional stability requires secure
emotional attachments.
Theory 2 Some brains are genetically disposed to
serotonin depletion during stress
19
Research process, 2
Theory 1 Emotional attachment ? emotional
stability.
A theory can lead to several hypotheses
Hypothesis 2 Emotional support during stress ?
less depression
Hypothesis 1 Fewer parent child interactions ?
vulnerability to depression.
A given hypotheses can be tested in several ways
Methods 1 Survey measurement assess of
family meals per week, correlate it with
self-reported depression.
Methods 2 Experiment ½ have structured parent /
child interactions, ½ do not, induce stress to
both groups, assess depression
20
Research process, 3
Theory 1 Emotional attachment ? emotional
stability.
Hypothesis 2 (Non)Support stress ? depression
Hypothesis 1 Family interactions ? depression.
Some hypotheses are best tested in a measurement
approach, and some with experimental designs

• Best tested by a measurement study
• Family interactions are difficult to bring into
  the lab,
• Possible ethical problems.

Can be tested in an experiment Both support and
stress can be controlled or manipulated in the
lab.
21
Research process The Big Picture
Phenomenon Big picture question.
Theory 2 Alternate explanation, invoking other
hypothetical constructs.
Theory 1 Possible explanation, invoking one set
of hypothetical constructs.
Hypothesis 2 Another prediction that tests the
same theory.
Hypothesis 1 A prediction that logically flows
from and tests the theory.
Methods 1 Operationally define the variables
test the hypothesis.
Methods 2 An alternate operational definition
way of testing the hypothesis.
22
Question 1
A theory
A Leads to one specific hypothesis B May be
one of several ways to explain something C Is
not as important as simply collecting data D Is
what you make up to explain why you forgot your
boy/girl friends birthday E Is not really
affected by social or personal values
23
Types of Variables Experimental designs
Independent Variable Dependent Variable

-- typically categorical  -- typically
continuous -- experimentally imposed --
measured -- hypothetical cause -- effect --
defines contrast space
-- models phenomenon
?what is compared to what e.g., administration
of drug v. placebo
? What is being explained e.g., task performance

• Used in true experiments
• Manipulation control of IV enhances internal
  validity
• Participants randomly assigned to experimental v.
  control groups

24
Types of Variables Measurement designs
Predictor Criterion

-- typically continuous   -- typically
continuous -- measured -- measured --
hypothetical cause -- effect -- defines
contrast space
-- models phenomenon
?what is compared to what e.g., people who
report using v. not using drugs

• What is being explained
• e.g., academic performance

• correlational or observational designs
• designs increase external validity
• potentially complex study 
• participants assigned by individual differences

25
The big picture Introductory lectures

• How do social values affect science?
• Framing a research question Where do hypotheses
  come from?
• The scientific process How do we get from a
  theory to actual data (and back)?
• How do we create an Independent Variable?
• What are the basic types of experimental designs?

?
26
Creating independent variables IVs

• 1. Direct experimental manipulation or
  treatment
• Most typical of true experiments
• Maximum control over IV
• 2. Indirect manipulation via experimental or
  research conditions
• Less direct control over IV
• 3. Quasi-Independent variables forming groups
  using a measured variable.
• Experiments without complete control over
  variables
• Used in measurement studies

27
Direct experimental manipulations
1. Experiments typically create an IV via direct
experimental manipulation

• Drug or biomedical intervention
• Behavioral intervention or treatment
• System-wide treatment (e.g., policy change,
  school-based)

• Simple presence v. absence of the treatment
• Single v. multiple treatment doses
• Type of treatment

• Structure the IV
• vis-à-vis

• Core design issue

• What is the Contrast Group?

• Placebo
• Attention control
• Within rather than between Ss

28
Indirect experimental manipulations
2. Indirect manipulation via experimental or
research conditions

• Experimental induction
• e.g., stress, anxiety, relaxation
• Via exposure to experimental condition
• instructional set or description of experiment
• "stage managing" social events 
• Assess IV via manipulation check
• self-report, standard assessment (e.g., of
  stress)
• observer rating

29
Quasi-independent variables
3. Create a quasi-Independent variable using a
measured variable.

• Categorize participants on a standard scale
• Scores over / under an established cut point,
• e.g., 4 depression symptoms on a standard scale.
• Scores based on a frequency a distribution
• Median split top v. bottom half.
• Extreme scores top v. bottom 10 of scores.
• Simple self-identification
• e.g., Republican v. Democrat.
• Behavioral index
• Used any drug in previous year v. not.
• Voted in 2008 v. not.

Not a True IV Participants not randomly
assigned to groups.
30
Using a measured variable to create groups

• Administer depression scale, count the symptoms
  rated 2 or 3.

• Form groups based on a cut point e.g., gt 4
  symptoms quasi-clinical depression.
• Participants are assigned to groups based on
  their ratings, not random assignment.

Below is a list of different feelings. Circle
the number that shows how many days you felt each
of these over the PAST WEEK. Rarely or A Little
A moderate Most or all of none of of the
Time amount of the time the time the
time (less than 1 day) (1 or 2 days) (3 - 4
days) (5 - 7 days) I was bothered by things that
usually do 0 1 2 3 not bother me. I felt I could
not shake off the blues even 0 1 2 3 with help
from my friends or family. I had trouble keeping
my mind on what 0 1 2 3 I was doing. I felt
depressed. 0 1 2 3 I felt that everything I did
was an effort. 0 1 2 3 My sleep was
restless. 0 1 2 3 I was happy. 0 1 2 3 I enjoyed
life. 0 1 2 3 I felt sad. 0 1 2 3
31
Question 4
An independent variable
A Is measured on a continuous scale B Is
manipulated by the researcher C Is the outcome
of the experiment D Is the phenomenon you are
trying to explain. E Does not care about other
people
32
Question 5
An dependent variable
A Is typically measured on a binary scale B
Is manipulated by the researcher C Is the
putative cause in the theory D Is the
phenomenon you are trying to explain. E Is
over-concerned about other people
33
The big picture Introductory lectures

• How do social values affect science?
• Framing a research question Where do hypotheses
  come from?
• The scientific process How do we get from a
  theory to actual data (and back)?
• How do we create an Independent Variable?
• What are the basic types of experimental designs?

?
34
Overview Basic Designs

• Pre-experimental designs no control group

Post-Test Only Design
Pre- Post- Test Design

• True (or Quasi-)experimental designs with a
  control group

After only Control group design
Pre- Post- Group Comparisons
Multiple group comparison
35
Pre-experimental designs
Post-Test Only Design
Treatment
Measure
Group
Only 1 group - typically an existing group no
selection or assignment occurs.
Experimental intervention (Treatment) may or
may not be controlled by the researcher. Use
for naturally occurring or system-wide events
(e.g., group trauma, government policy change,
etc.).
Measurement may or may not be controlled by the
researcher.
Pre- Post- Test Design
Measure1
Treatment
Measure1
Group

• Only one group
• only group available?
• naturally occurring intervention?

Measurements given to all participants at
baseline follow-up
All participants get the same treatment, which
may or may not be controlled by the researcher.
36
Pre-experimental Designs (2)
Advantage of Post- Pre- Post- Designs

• Study naturally occurring intervention,

• e.g., test scores before and after some school
  change,
• rime rates after a policy change, etc.
• Having both Pre- and Post measures allows us to
  examine change.

• Disadvantage no control group many threats to
  internal validity
• Maturation Participants may be older / wiser by
  the post-test
• History Cultural or historical events may occur
  between pre- and post-test that change the
  participants
• Mortality Participants may non-randomly drop out
  of the study
• Regression to baseline Participants who are more
  extreme at baseline look less extreme over time
  as a statistical confound.
• Reactive Measurement Participants may change
  their scores due to being measured twice, not the
  experimental manipulation.

37
Experiments

• After only Control group design

Treatment
Measure
Group 1
Measure
Group 2
Control

• Adds a control group. Either
• Observed Groups
• Naturally occurring (e.g., Class 1. v. Class 2)
  or
• self-selected (sought therapy v. did not).
• Assigned Groups Randomly assign participants to
  experimental v. control group, or match
  participants to create equivalent groups.

Measure Dependent Variables(s) only at
follow-up. Use experimental or standard measures
(e.g., grades, census data, crime reports).

• Advantage Control group lessens confounds /
  threats to internal validity.
• Random assignment decreases threats to internal
  validity.
• Disadvantage Existing or self-selected groups
  may have confounds.
• No baseline or pre- measure available
• assess change?
• ceiling (or floor) effects?
• cannot assess equivalence of groups at baseline.

38
Basic Designs True experiments (2)

• Pre- Post- Group Comparisons (most common study
  design)

Group 1
Measure 1
Treatment
Measure 2
Control
Group 2
Measure 1
Measure2
Two groups Observed (quasi-experiment)
or Assigned (true experiment).
Only one group receives experimental intervention.

• Post-test follow-up of dependent variable(s)
• Simple outcome
• Change from baseline.

Baseline (pre-test) measure of study variables
and possible confounds.
Advantages Pre-measure assesses baseline level
of Dependent Variable -- allows researcher to
assess change -- can detect ceiling (or floor)
effects -- can use to assign participants to
groups via matching -- can assess baseline
equivalence of groups Disadvantage Highly
susceptible to confounds if using observed or
self-selected groups.
39
More Complex Experimental Designs

• Multiple group comparison

Measure2
Treatment 1
Treatment 2
Measure2
Control
Control

• 3 (or more) groups
• typically formed by Random assignment.

• 2 experimental groups, e.g.
• low v. high dose,
• exp. situation 1 v. 2, etc.,
• plus the control group.

• Compare
• Level 1 of independent variable from Level 2
• Either / both experimental groups from control
  grp.

• Advantage Test dose or context effects
• Drug doses, amounts of psychotherapy, levels of
  anxiety, etc. Increasing dose effect can be
  tested against no dose.
• Diverse conditions to test 2nd hypotheses or
  confounds, e.g., therapy delivered by same sex v.
  opposite sex therapist.
• Disadvantage
• More costly and complex.
• Potential ethical problem with a no dose (or
  very high dose) condition.

40
Overview of true experimental designs
Representative of the larger population? -
Selection  - Size of sample

• Groups equal at baseline?
• Existing groups or self-selection
• v.
• - Random assignment

Equality of procedures? - Information -
Expectancies - Quality of blinding

• Faithfulness of treatment?
• - Operational def.
• - Correct dose?
• Manipulation check

Groups really different at outcome? -
Statistical significance
Internal Validity Likelihood of chance results
External validity Random selection of sample
Internal validity Random Assignment
Internal validity Lack of confounds
External Validity Correct independent variable?
41
Lectures so far key terms

• Features of research Key terms
• Theory
• Hypothetical construct
• Hypothesis
• Variable
• Operational definition
• Internal external validity
• Independent v. Dependent variables
• Measurement v. experimental studies

42
Research flow
43
Basics of major forms of research.
External validity Internal validity
44
Key terms concepts

• Role of values social judgments in the research
  process
• Basic elements of science
• Hypothetical constructs
• Operational definitions
• Statement of testable hypothesis
• Predictive, potentially refutable
• Specify Variables in functional relationship
• Replication

• The hierarchy of phenomena, theory, hypotheses,
  methods

45
Key terms concepts, 2

• Measurement v. experimental methods
• Types of variables used
• Cause effect assumptions
• Creating variables
• Direct treatment dose or manipulation
• Indirect use of context (manipulation check)
• Using a measured variable (self-reports or
  status variable) to assign to groups

46
Overview, 3

• Experimental design key elements
• Control group v. non-controlled designs
• Threats to internal validity

• Maturation
• History
• Mortality
• Regression to baseline
• Reactive Measurement

• Pre-experimental designs
• Pre-post designs
• Multiple group comparisons.