Nonexperimental Research Methods

1
Nonexperimental Research Methods
2
Survey Research

• Want to know about someones opinions, attitudes,
  behaviors?
• Ask them!

3
Survey Research

• Descriptive Surveys
• Seek to describe elements from a population
• Analytic Surveys
• Seek to determine how variables are related

4
Survey Research

• Yes/No Questions
• Forced Alternative Questions
• Multiple Choice Questions
• Likert-type Scales
• Open-ended Questions

5
Tests and Inventories

• Designed to assess a specific attribute, ability,
  or characteristic of individuals

6
Tests and Inventories

• Construct Validity
• Are you measuring what you say youre measuring?
• Content Validity
• Interrater Reliability

7
Types of ReliabilityInter-rater reliability

• Assesses the degree to which different
  raters/observers give consistent estimates of the
  same phenomenon
• agreement
• Correlation between observers scores

8
Tests and Inventories

• Construct Validity
• Are you measuring what you say youre measuring?
• Content Validity
• Interrater Reliability
• Concurrent Validity

9
Tests and Inventories

• Construct Validity
• Are you measuring what you say youre measuring?
• Content Validity
• Interrater Reliability
• Concurrent Validity
• Criterion Validity

10
Reliability

• The extent to which a test is consistent in its
  evaluation of the same individuals

11
Types of ReliabilityTest-retest

• Correlation between two observations (i.e., set
  of scores) on the same test administered to the
  same (or similar) sample on two different
  occasions
• Assumes no change in construct being measured
• Time between observations is crucial

12
Types of ReliabilityParallel-forms

• Correlation between two observations (i.e., set
  of scores) on parallel forms of a test
  administered to the same sample
• Requires generation of many items that measure
  the same construct
• Two forms of test can be used independently
• Assumes randomly divided halves are equivalent

13
Types of ReliabilityInternal consistency

• Single test administered to a sample on one
  occasion
• Assesses the consistency of the results for
  different items for the same construct within the
  measure
• Average inter-item correlation
• Average item-total correlation
• Split-half reliability
• Cronbachs Alpha

14
Who do you survey?

• Population
• Complete set of individuals or events
• Sampling frame
• List from which you draw your sample
• Sample
• Group that you obtain to represent the population

15

• Survey participants provide you with one or more
  responses
• You use a statistic to summarize the responses
  from a group of people
• If you could measure the whole population, youd
  have a population parameter

16
The Sampling Distribution

• Imagine drawing three samples from the same
  populationcollecting the mean
• Means would differ slightly, but they should be
  pretty similar
• Imagine now drawing an infinite number of samples
  from the same population and computing an average
  from each one
• Those averages should be normally distributed

17
The Sampling Distribution

• Average of the sampling distribution should be
  the population parameter
• Standard deviation of the sampling distribution
  tells us how the samples would be distributed
  this is known as the standard error

18
The Sampling Distribution

• Standard deviation
• Spread of scores around the average of scores in
  a single sample
• Standard error
• Spread of averages around the average of averages
  in a sampling distribution

19
The Sampling Distribution

• Standard error is calculated from the standard
  deviation in your sample
• Also related to sample size
• Remember the 65, 95, 99 rule from a normal
  distribution?

20
The Sampling Distribution
21
The Sampling Distribution

• So, using your sample, you can estimate the
  average for the population and give odds that the
  actual population mean falls within a certain
  range with the confidence intervals

22
Who do you survey?

• Random sample or Random selection
• Every member of the population has an equal
  likelihood of being selected

23
Who do you survey?

• Random sample with replacement
• Chosen item or individual is returned to
  population, can be selected again
• Random sample without replacement
• Chosen item or individual is not returned to the
  population, can only be selected once
• Stratified random sampling
• Divide population into levels or subpopulations,
  random sample from them

24
Who do you survey?

• Nonrandom sampling methods
• Cannot rely on probability theory
• Can be accidental or purposive

25

• Heterogeneity sampling
• Sampling for diversity or variety
• Modal instance sampling
• Sampling for the most typical case
• Expert sampling
• Sample of people with known experience or
  expertise in some area
• Quota sampling
• Sample until you have a specific number from each
  subgroup in a population
• Snowball sampling
• Sample a few who fit your criteria, ask them to
  identify others who would also fit