Randomisation

1
Randomisation

• Surveys should use some form of random sampling
  to obtain a representative sample
• Randomisation avoids subjective and other biases
• Randomisation allows the calculation of likely
  size of the sampling error estimates.

Simple Random Sampling(SRS)

• Label each population member from 1 to N and
  sample without replacement in a random way using
  a
• Lottery method
• Random Number tables
• Computer

2
Other types of sampling

• Stratified sampling
• Population is made up of different strata
• A sub sample is selected from each stratum using
  SRS
• The combined sub samples form your sample
• E.g. Looking at average net profit of companies
  listed on the NZSE. Using annual reports we can
  classify the companies into large, medium and
  small. These categories form our strata
• Cluster sampling
• Population is made up of distinct clusters
• Randomly select some clusters and then sample
  from each cluster
• E.g. Take a list of towns in NZ (pop. lt 20,000)
  and choose some at random
• In each selected town get a list of streets and
  choose some at random
• In each selected street get a list of the houses
  and choose some at random
• Ask each selected household some questions
• Systematic sampling
• Take every kth unit.
• Often used in biological sciences

3
Errors in surveys

• Sampling errors are
• Errors caused by the act of take the sample
• The difference between the sample values and the
  population value
• Bigger in smaller samples than in larger ones
• Unavoidable (the cost of taking a sample)
• Can be minimised (but never removed) by careful
  design of the sampling scheme
• E.g. TV3/CM Research Poll sample 1,500 people
  with a 3 margin of error
• The margin of error is an estimate of how large
  the sampling error is likely to be

4
Non-sampling errors

• Can be much larger than sampling errors
• Always present
• It is virtually impossible to determine how badly
  they will affect the results
• It is almost impossible to correct for
  non-sampling errors after the survey has been
  completed
• Therefore, we must try to minimise non-sampling
  errors when designing the survey
• That is we use a pilot survey (if possible) to
  expose the flaws or potential flaws in the survey
  design

5
Selection bias

• The sampling mechanism does not allow every unit
  in the population to be selected
• E.g. A telephone survey is a possible source of
  selection bias because the selection method
  excludes all those people not in the phone book

Self-Selection bias

• The sampling mechanism allows units to select
  themselves for the survey
• A suggestion box is an example of self-selection
  bias
• Many behavioural research studies, especially
  those concerning areas such as sexual habits have
  selection biases

6
Non-response bias

• Non-response occurs in three ways
• Non-coverage ? Selection bias
• Item non-response failure to answer a question
  on a survey
• Unit non-response failure to respond to the
  whole survey
• Item non-response usually occurs when questions
  are of a sensitive nature.
• E.g. Please state your income so we can see
  whether you owe more tax than you paid
• Unit non-response usually occurs when there is
  little or no incentive to respond.
• Mail-in surveys often suffer from unit
  non-response
• Telephone surveys

7
Question effects

• 18 Aug 1980, NY Times/CBS News Poll
• Do you think there should be an amendment to
  the constitution prohibiting abortions?
• 29 Yes, 62 No
• Later the sample people were asked
• Do you think there should be an amendment to
  the constitution protecting the life of the
  unborn child?
• 50 Yes, 39 No

8
Interviewer effects

• In 1968, one year after a major racial
  disturbance in Detroit, a sample of black
  residents were asked
• Do you personally feel that you can trust most
  white people, some white people or none at all?
• White interviewer 35 answered most
• Black interviewer 7 answered most
• Interviewers should be non-threatening.
• Studies have shown that women make the best
  interviewers

9
Behavioural considerations

• People may behave differently because they think
  they will get in trouble
• E.g. Official vote counts show 86.5 million
  people voted in the 1980 US presidential
  elections
• A Census Bureau survey of 64,000 households some
  weeks later estimated 93.1 million people voted.

Transfer of findings

• Is it valid to apply the findings of a survey
  conducted on one population to another?
• Generally not.
• E.g if a survey about the amount of fruit eaten
  is conducted in the US could we use the results
  for New Zealand?

10
Survey format

• Consider a subject you find embarrassing.
• Now suppose youre involved in a survey about
  that subject
• Will your answer to various questions differ if
  the survey uses
• A face to face interview
• A telephone poll
• questionnaires to be mailed back