Social Science Reasoning Using Statistics

1
Social Science Reasoning Using Statistics

• Psychology 138
• Spring 2005

2
Producing Data

• Scientific method
• Research methods
• Observation methods
• Experimental methods
• Quasi-experimental
• Variables
• Types
• Operational definitions
• Sampling
• Samples and populations
• Statistics and parameters
• Techniques

• Measurements
• Continuous and discrete
• Scales of measurement
• Instrument
• Validity
• Internal and external
• confounds
• Bias
• Reliability
• Experimental control

3
Scientific method

• Systematic observation (and experimentation) used
  to explain how and why events occur
• Ask the research question
• Identify variables and formulate the hypothesis
• Define your population
• Select a research methodology
• Collect your data from a sample
• Analyze your data
• Draw conclusions based on your data
• Repeat

4
Basic Research Methods

• Observational study
• Researcher observes and measures variables of
  interest to find relationships between the
  variables
• No attempt is made to manipulate or influence
  responses
• Experimental methodology
• One (or more) independent variable(s) is
  manipulated while changes are observed in another
  variable (dependent)
• Used to establish cause-and-effect relationships
  between variables
• Uses extensive methods of control to minimize
  extraneous sources of variability
• Quasi-experimental methodology
• One (or more) of the independent variables is a
  pre-existing characteristic (e.g., sex, age, etc.)

5
Different basic methods

• Precise control possible
• Precise measurement possible
• Theory testing possible
• Can make causal claims

• May see patterns of complex behaviors
• Good first step
• May learn about something unexpected

• Artificial situations may restrict
  generalization to real world
• Complex behaviors may be difficult to measure

• Shouldnt make causal claims
• Directionality of the relationship isnt known
• Threats to internal validity due to lack of
  control
• Sometimes the results are not reproducible

6
Measuring and Manipulating Variables

• Two levels of variables
• Conceptual level of the variables
• What the theory is about
• Operational level of the variables
• What is actually manipulated/measured in the
  research program

7
Variables

• Identify the things that were studying
• Variables
• Characteristics or conditions that change or has
  different values for different individuals (or
  situations)
• Independent (explanatory) variables
• The variable that does the explaining
• In an experiment it is the variable that is
  manipulated by the researcher
• Dependent (response) variable
• The variable that is observed for changes in
  order to assess the effect of the manipulation
• Typically it is the variable measured in an
  experiment

8
Variables

• Control variables
• Holding things constant - Controls for excessive
  random variability

• Random variables
• may freely vary, to spread variability equally
  across all experimental conditions

• Confound variables
• Other variables, that havent been accounted for
  (manipulated, measured, randomized, controlled)
  that can impact changes in the dependent
  variable(s)

9
Sampling
everybody that the research is targeted to be
about
the subset of the population that actually
participates in the research
10
Sampling

• Goals
• Reduce
• Sampling error - the difference between the
  population parameter and the sample statistic

• Maximize
• Representativeness - to what extent do the
  characteristics of those in the sample reflect
  those in the population
• Minimize
• Bias - a systematic difference between those in
  the sample and those in the population

11
Sampling Error

• Size of sampling error
• Amount of variability in the population
• Size of the sample
• Generally, the larger the sample, the smaller the
  sampling error

12
Sampling Methods

• Probability sampling

• Simple random sampling
• Systematic random sampling
• Stratified sampling
• Convenience sampling
• Quota sampling

• Non-probability sampling

13
Measurement

• How do we measure a variable?
• An instrument The tool that is used to measure
  the dependent variable

• Properties of our measurement?
• Units of measurement - whether the measurement
  has a minimum sized unit or not
• Continuous vs. discrete
• Scales of measurement - the correspondence
  between the numbers representing the properties
  that were measuring

14
Scales of measurement

• Categorical variables

• Nominal Scale Consists of a set of categories
  that have different names.

• Ordinal Scale Consists of a set of categories
  that are organized in an ordered sequence.

• Quantitative variables

• Interval Scale Consists of ordered categories
  where all of the categories are intervals of
  exactly the same size.

• Ratio scale An interval scale with the
  additional feature of an absolute zero point.

15
Scales of measurement
Interval scale
Ratio scale
8 cards high
5 cards high
0 cards high means as tall as the table
0 cards high means no height
16
Errors in measurement

• Reliability
• Do we get the same score with repeated
  measurements?
• Validity
• Does our measure really measure the construct?
• Is there bias in our measurement?
• Internal validity
• External validity

17
Dart board example
18
Internal Validity

• The precision of the results
• Did the change result from the changes in the DV
  or does it come from something else?
• Threats to internal validity
• History some event happens while youre doing
  the experiment
• Maturation - participants get older and have
  developmental changes
• Mortality participants die (or stop showing up)
  and cant continue, especially if they drop out
  more in some groups relative to others
• Interactions with selection some of these
  threats may interact, especially with selection
• Non-random selection to experimental conditions
  (groups)

19
External Validity

• Are experiments real life behavioral
  situations, or does the process of control put
  too much limitation on the way things really
  work?
• Will the same basic conclusions be supported with
  different operational definitions, different
  participants, different research settings?

20
Experimental Control

• Our goal
• to test the possibility of a relationship between
  the variability in our IV and how that affects
  our DV.
• Control is used to minimize excessive variability.

21
Sources of variability (noise)

• Sources of Total (T) Variability
• T NonRandomexp NonRandomother
  Random

22
Sources of variability (noise)

• Sources of Total (T) Variability
• T NonRandomexp NonRandomother
  Random

Constrain variability by carefully levels of IV
Eliminate counfounds
Use good measures

• Experimental procedures are used to reduce R and
  NRother so that we can detect NRexp.
• That is, so we can see the changes in the DV that
  are due to the changes in the independent
  variable(s).