Title: Experiment Basics: Variables
1Experiment Basics Variables
- Psych 231 Research Methods in Psychology
2Reminders
- Print out the Class experiment exercise (from the
Lab web page) and bring it to labs this week - Group project introduction sections due this week
- Dont forget to take the on-line quizzes
- quiz 5, chapter 4, was due yesterday
- quiz 6, chapter 6, is due Oct 1 (Tuesday)
- Journal Summary 1 is due in labs next week
3Exam 1
- Mean 78.3
- Median 78
- Range 59-92
- If you want to go over your exam set up a time to
see me
4Many kinds of Variables
- Independent variables (explanatory)
- Dependent variables (response)
- Extraneous variables
- Control variables
- Random variables
- Confound variables
5Many kinds of Variables
- Independent variables (explanatory)
- Dependent variables (response)
- Extraneous variables
- Control variables
- Random variables
- Confound variables
6Identifying potential problems
- These are things that you want to try to avoid by
careful selection of the levels of your IV (may
be issues for your DV as well).
- Demand characteristics
- Experimenter bias
- Reactivity
- Floor and ceiling effects (range effects)
7Demand characteristics
- Characteristics of the study that may give away
the purpose of the experiment - May influence how the participants behave in the
study - Examples
- Experiment title The effects of horror movies on
mood - Obvious manipulation Having participants see
lists of words and pictures and then later
testing to see if pictures or words are
remembered better - Biased or leading questions Dont you think its
bad to murder unborn children?
8Experimenter Bias
- Experimenter bias (expectancy effects)
- The experimenter may influence the results
(intentionally and unintentionally) - E.g., Clever Hans
- One solution is to keep the experimenter (as well
as the participants) blind as to what
conditions are being tested
9- Knowing that you are being measured
- Just being in an experimental setting, people
dont always respond the way that they normally
would. - Cooperative
- Defensive
- Non-cooperative
Reactivity
10Range effects
- Floor A value below which a response cannot be
made - As a result the effects of your IV (if there are
indeed any) cant be seen. - Imagine a task that is so difficult, that none of
your participants can do it.
- Ceiling When the dependent variable reaches a
level that cannot be exceeded - So while there may be an effect of the IV, that
effect cant be seen because everybody has maxed
out - Imagine a task that is so easy, that everybody
scores a 100 - To avoid floor and ceiling effects you want to
pick levels of your IV that result in middle
level performance in your DV
11Variables
- Independent variables (explanatory)
- Dependent variables (response)
- Extraneous variables
- Control variables
- Random variables
- Confound variables
12Dependent Variables
- The variables that are measured by the
experimenter - They are dependent on the independent variables
(if there is a relationship between the IV and DV
as the hypothesis predicts).
- Consider our class experiment
- Conceptual level Memory
- Operational level Free Recall test
- Present list of words, participants make a
judgment for each word - 15 sec. of filler (counting backwards by 3s)
- Measure the accuracy of recall
13Choosing your dependent variable
- How to measure your your construct
- Can the participant provide self-report?
- Introspection specially trained observers of
their own thought processes, method fell out of
favor in early 1900s - Rating scales strongly agree - agree -
undecided - disagree - strongly disagree - Is the dependent variable directly observable?
- Choice/decision
- Is the dependent variable indirectly observable?
- Physiological measures (e.g. GSR, heart rate)
- Behavioral measures (e.g. speed, accuracy)
14Measuring your dependent variables
- Scales of measurement
- Errors in measurement
15Measuring your dependent variables
- Scales of measurement
- Errors in measurement
16Measuring your dependent variables
- Scales of measurement - the correspondence
between the numbers representing the properties
that were measuring - The scale that you use will (partially) determine
what kinds of statistical analyses you can perform
17Scales of measurement
- Categorical variables (qualitative)
- Nominal scale
- Ordinal scale
- Quantitative variables
- Interval scale
- Ratio scale
18Scales of measurement
- Nominal Scale Consists of a set of categories
that have different names.
- Label and categorize observations,
- Do not make any quantitative distinctions between
observations. - Example
- Eye color
19Scales of measurement
- Categorical variables (qualitative)
- Nominal scale
- Ordinal scale
- Quantitative variables
- Interval scale
- Ratio scale
Categories
20Scales of measurement
- Ordinal Scale Consists of a set of categories
that are organized in an ordered sequence.
- Rank observations in terms of size or magnitude.
- Example
- T-shirt size
21Scales of measurement
- Categorical variables
- Nominal scale
- Ordinal scale
- Quantitative variables
- Interval scale
- Ratio scale
Categories
Categories with order
22Scales of measurement
- Interval Scale Consists of ordered categories
where all of the categories are intervals of
exactly the same size. - Example Fahrenheit temperature scale
- With an interval scale, equal differences between
numbers on the scale reflect equal differences in
magnitude. -
- However, Ratios of magnitudes are not meaningful.
20º
40º
20º increase
The amount of temperature increase is the same
60º
80º
20º increase
20º
40º
Not Twice as hot
23Scales of measurement
- Categorical variables
- Nominal scale
- Ordinal scale
- Quantitative variables
- Interval scale
- Ratio scale
Categories
Categories with order
Ordered Categories of same size
24Scales of measurement
- Ratio scale An interval scale with the
additional feature of an absolute zero point.
- Ratios of numbers DO reflect ratios of magnitude.
- It is easy to get ratio and interval scales
confused - Example Measuring your height with playing cards
25Scales of measurement
Ratio scale
8 cards high
26Scales of measurement
Interval scale
5 cards high
27Scales of measurement
Interval scale
Ratio scale
8 cards high
5 cards high
0 cards high means as tall as the table
0 cards high means no height
28Scales of measurement
- Categorical variables
- Nominal scale
- Ordinal scale
- Quantitative variables
- Interval scale
- Ratio scale
Categories
Categories with order
Ordered Categories of same size
Ordered Categories of same size with zero point
Best Scale?
- Given a choice, usually prefer highest level of
measurement possible
29Measuring your dependent variables
- Scales of measurement
- Errors in measurement
30Example Measuring intelligence?
- How do we measure the construct?
- How good is our measure?
- How does it compare to other measures of the
construct? - Is it a self-consistent measure?
Measuring the true score
31Errors in measurement
- In search of the true score
- Reliability
- Do you get the same value with multiple
measurements? - Validity
- Does your measure really measure the construct?
- Is there bias in our measurement? (systematic
error)
32Dartboard analogy
Bulls eye the true score
33Dartboard analogy
Bulls eye the true score Reliability
consistency Validity measuring what is intended
reliablevalid
reliable invalid
unreliable invalid
34Reliability
- True score measurement error
- A reliable measure will have a small amount of
error - Multiple kinds of reliability
35Reliability
- Test-restest reliability
- Test the same participants more than once
- Measurement from the same person at two different
times - Should be consistent across different
administrations
Reliable
Unreliable
36Reliability
- Internal consistency reliability
- Multiple items testing the same construct
- Extent to which scores on the items of a measure
correlate with each other - Cronbachs alpha (a)
- Split-half reliability
- Correlation of score on one half of the measure
with the other half (randomly determined)
37Reliability
- Inter-rater reliability
- At least 2 raters observe behavior
- Extent to which raters agree in their
observations - Are the raters consistent?
- Requires some training in judgment
500
456
38Validity
- Does your measure really measure what it is
supposed to measure? - There are many kinds of validity
39VALIDITY
CONSTRUCT
INTERNAL
EXTERNAL
CRITERION- ORIENTED
FACE
CONVERGENT
PREDICTIVE
DISCRIMINANT
CONCURRENT
Many kinds of Validity
40VALIDITY
CONSTRUCT
INTERNAL
EXTERNAL
CRITERION- ORIENTED
FACE
CONVERGENT
PREDICTIVE
DISCRIMINANT
CONCURRENT
Many kinds of Validity
41Face Validity
- At the surface level, does it look as if the
measure is testing the construct?
This guy seems smart to me, and he got a high
score on my IQ measure.
42Construct Validity
- Usually requires multiple studies, a large body
of evidence that supports the claim that the
measure really tests the construct
43Internal Validity
- The precision of the results
- Did the change in the DV result from the changes
in the IV or does it come from something else?
44Threats to internal validity
- Experimenter bias reactivity
- History an event happens the experiment
- Maturation participants get older (and other
changes) - Selection nonrandom selection may lead to
biases - Mortality (attrition) participants drop out or
cant continue - Regression to the mean extreme performance is
often followed by performance closer to the mean - The SI cover jinx
45External Validity
- Are experiments real life behavioral
situations, or does the process of control put
too much limitation on the way things really
work?
46External Validity
- Variable representativeness
- Relevant variables for the behavior studied along
which the sample may vary - Subject representativeness
- Characteristics of sample and target population
along these relevant variables - Setting representativeness
- Ecological validity - are the properties of the
research setting similar to those outside the lab
47Measuring your dependent variables
- Scales of measurement
- Errors in measurement
- Reliability Validity
- Sampling error
48Sampling
- Errors in measurement
- Sampling error
Everybody that the research is targeted to be
about
The subset of the population that actually
participates in the research
Sample
49Sampling
- Allows us to quantify the Sampling error
50Sampling
- Goals of good sampling
- Maximize Representativeness
- To what extent do the characteristics of those in
the sample reflect those in the population - Reduce Bias
- A systematic difference between those in the
sample and those in the population
- Key tool Random selection
51Sampling Methods
- Probability sampling
- Simple random sampling
- Systematic sampling
- Stratified sampling
- Non-probability sampling
- Convenience sampling
- Quota sampling
52Simple random sampling
- Every individual has a equal and independent
chance of being selected from the population
53Systematic sampling
- Selecting every nth person
54Cluster sampling
- Step 1 Identify groups (clusters)
- Step 2 randomly select from each group
55Convenience sampling
- Use the participants who are easy to get
56Quota sampling
- Step 1 identify the specific subgroups
- Step 2 take from each group until desired number
of individuals
57Variables
- Independent variables
- Dependent variables
- Measurement
- Scales of measurement
- Errors in measurement
- Extraneous variables
- Control variables
- Random variables
- Confound variables
58Extraneous Variables
- Control variables
- Holding things constant - Controls for excessive
random variability - Random variables may freely vary, to spread
variability equally across all experimental
conditions - Randomization
- A procedure that assures that each level of an
extraneous variable has an equal chance of
occurring in all conditions of observation. - Confound variables
- Variables that havent been accounted for
(manipulated, measured, randomized, controlled)
that can impact changes in the dependent
variable(s) - Co-varys with both the dependent AND an
independent variable
59Colors and words
- Divide into two groups
- men
- women
- Instructions Read aloud the COLOR that the
words are presented in. When done raise your
hand. - Women first. Men please close your eyes.
- Okay ready?
60Blue Green Red Purple Yellow Green Purple Blue Red
Yellow Blue Red Green
List 1
61- Okay, now it is the mens turn.
- Remember the instructions Read aloud the COLOR
that the words are presented in. When done raise
your hand. - Okay ready?
62Blue Green Red Purple Yellow Green Purple Blue Red
Yellow Blue Red Green
List 2
63Our results
- So why the difference between the results for men
versus women? - Is this support for a theory that proposes
- Women are good color identifiers, men are not
- Why or why not? Lets look at the two lists.
64List 2Men
List 1Women
Blue Green Red Purple Yellow Green Purple Blue Red
Yellow Blue Red Green
Blue Green Red Purple Yellow Green Purple Blue Red
Yellow Blue Red Green
Matched
Mis-Matched
65- What resulted in the performance difference?
- Our manipulated independent variable (men vs.
women) - The other variable match/mis-match?
- Because the two variables are perfectly
correlated we cant tell - This is the problem with confounds
Blue Green Red Purple Yellow Green Purple Blue Red
Yellow Blue Red Green
Blue Green Red Purple Yellow Green Purple Blue Red
Yellow Blue Red Green
66- What DIDNT result in the performance difference?
- Extraneous variables
- Control
- of words on the list
- The actual words that were printed
- Random
- Age of the men and women in the groups
- These are not confounds, because they dont
co-vary with the IV
Blue Green Red Purple Yellow Green Purple Blue Red
Yellow Blue Red Green
Blue Green Red Purple Yellow Green Purple Blue Red
Yellow Blue Red Green
67Debugging your study
- Pilot studies
- A trial run through
- Dont plan to publish these results, just try out
the methods - Manipulation checks
- An attempt to directly measure whether the IV
variable really affects the DV. - Look for correlations with other measures of the
desired effects.