URBP 204A QUANTITATIVE METHODS I Survey Research I

1
URBP 204A QUANTITATIVE METHODS ISurvey Research
I

• Gregory Newmark
• San Jose State University
• (This lecture is based on Chapters 7, 9 10 of
  Earl Babbies
• The Practice of Social Research, 10th Edition.
• All cartoons are from CAUSEweb.org by J.B.
  Landers.)

2
Populations and Samples
3
Populations and Samples

• Populations
• All the people in a specified group of people
• The population of Students at SJSU
• The population of Students in Urban Planning at
  SJSU
• The population of Students in 204A this semester
• Samples
• A portion of a larger population selected for
  study
• A 500 person Sample of Students at SJSU
• A 50 person Sample of Students in Urban Planning
• A 15 person Sample of Students in 204A this
  semester

4
Populations and Samples

• Ideally, research covers entire populations
• Medicine X always cures the common cold
• Financially, research is expensive
• We cant afford to test Medicine X on everyone
• Practically, we test samples of a population
• We can afford to test Medicine X on 1,000
  people
• Hopefully, those samples well represent the
  actual population
• For our results to be generalizable, our 1,000
  people should approximate the characteristics of
  everyone

5
Populations and Samples

• Sampling Error
• A measure of how well a sample approximates the
  characteristics of the larger population
• The difference between a sampling statistic
  (i.e., values in the sample) and a population
  parameter (i.e., values in the population)
• Low sampling error means higher precision
• Higher precision means more generalizability
• Valuable research has a high degree of
  generalizability

6
Sampling

• Say we want to sample the students at SJSU, how
  do we pick our sample?

7
Sampling

• Bias occurs when the sample is not representative
  of the population
• We sampled students exiting the gym and found
  that SJSU students have a high level of fitness
• We sampled students from 6 to 9 pm and found
  that most SJSU students work full time
• We sampled students at Burger King and found
  that very few SJSU students are vegetarian
• How can we avoid bias?

8
Probability Sampling

• Samples selected in accord with probability
  theory, typically using random selection
• All members of population have an equal chance of
  being selected for sample
• In theory, removes bias from sample

9
Probability Sampling

• In practice, several types of bias remain
  possible
• Non-response Bias
• Some people can not or will not answer the survey
• If their traits vary from responders, bias occurs
• Coverage Bias
• If members of population are not in sampling
  frame
• E.g. people without telephones in a telephone
  survey
• Selection Bias
• If members have a different probability of
  selection
• E.g. people with multiple lines in a telephone
  survey

10
Probability Sampling

• Samples never perfectly representative but we
  can estimate how close they are
• We exploit the fact that the sample averages are
  normally distributed
• Example
• Then we take the standard deviation of those
  averages which we call the standard error (SE)

11
Probability Sampling

• Calculate standard (sampling) error
• This is the Standard Deviation divided by the
  square root of the sample size (we use the SD of
  the sample rather than the population)
• We report our confidence level that the actual
  population parameter is within a given range from
  the sampling statistic

12
Probability Sampling
13
Probability Sampling

• Example Neighborhood of 10,000 households
• What is the mean household income?
• Population Parameter (true value for
  population)
• 50,000
• Survey incomes from a sample of 100 households
• Sampling Statistic (value for the sample)
• 49,000 with an SD of 10,000
• Calculate the standard error (SE)
• 1,000 10,000 / 10

14
Probability Sampling

• Example Neighborhood of 10,000 households
• Define our confidence level,
• 95 Confidence Level
• Define the associated confidence interval
• 95 Confidence Level is associated with 2
  standard errors
• Each standard error is 1,000
• Confidence interval would be from 47,000 to
  51,000
• We are 95 confident that the true mean income
  falls between 47,000 and 51,000
• Margin of Error half the range of the
  confidence interval
• We are 95 confident the true mean income is
  49,000 2,000

15
Probability Sampling

• Bad pun on margin of error

16
Probability Sampling

• Which faux movie is expressed as a confidence
  interval?

17
Probability Sampling

• Why is there no margin of error here?

18
Probability Sampling

• Why might the confidence interval be good to know?

19
Probability Sampling

• Sampling Designs
• Simple Random Sampling
• Everyone assigned a number, random cases selected
• Systematic Sampling Every 7th household. . .
• Every kth person chosen
• Sampling Ratio is the proportion of population
  selected
• Problem of periodicity (e.g. apartments, army
  units)
• Stratified Sampling Stratified by age group .
  .
• Groups units into homogenous strata before
  sampling
• Improves representativeness, in terms of strata
  
• Cluster Sampling Only certain schools
  selected
• Multistage approach
• Natural clusters selected first and then
  subsampled

20
Non-Probability Sampling

• Sampling error can not be statistically estimated
• Data only tells us about who was surveyed
• Approaches
• Judgment Samples used for targeting surveys
• Researcher decides whom to include
• Snowball Samples used for rare populations
• Respondents recommend other respondents
• Quota Samples common for market research
• Sample designed to include a designated number of
  people with certain specified characteristics
• Convenience Samples such as an in-class survey
• Sample includes whomever most easily accessed

21
Modes of Observation

• Surveys
• Experiments
• Qualitative Field Research
• Others ( for example, content analysis)

22
Surveys

• Strengths
• Describe characteristics of a large population
• Make large samples feasible
• More flexible can cover several topics
• Strong on reliability
• Weakness
• Least common denominator
• Life situation/ context not known
• Inflexible - can not change questions mid-way
• Respondent may form opinion at the moment
• Weak on validity

23
Surveys

• Design elements
• Questions and statements
• Open vs. Closed Ended Questions
• Avoid double barreled questions check the use
  of and
• Make items clear and cogent
• Be very specific, e.g. income last year based on
  tax return
• Respondent must be competent
• Dont ask a brain surgeon about rocket science
• Respondent must be willing to answer
• Are you having an affair?
• Avoid negative items and biased phrasing

24
Surveys

• Guidelines for survey interviewing
• Modest yet neat appearance
• Avoid voice inflections
• Be neutral
• Be polite
• Be familiar with the questions
• Dont add your words to the question
• Record responses exactly
• Gently probe for responses / clarifications

25
Surveys

• Guidelines for survey form design
• Not cluttered
• Professional look
• Clear instructions on how to choose responses
• Consider having introductory comments
• Use of contingency questions
• if yes to Question 7, ask Question 7a
• Use of matrix format
• Order of questions
• Do not bias due to the order
• In self- administered begin with interesting
  questions
• In interviews begin with uncomplicated questions
• Pretest the questionnaire

26
Surveys

• Self- administered Questionnaire
• Mail (home delivery or combination)
• Monitor the return
• Follow up mailings
• Questionnaire at a public gathering
• Telephone Surveys
• Unlisted number random digit dialing
• Advantages
• cheap and quick no dress code safety
• probe more sensitive areas
• more quality control possible as central location
• Disadvantages
• compete with bogus surveys easy to hang up
• answering machines cell phones

27
Surveys
28
Surveys