Another poll ruins Federal Election suspense

1
Another poll ruins Federal Election suspense.

• On January 22, 2006 the research firm SES
  sampled 1,200 Canadians on their voting
  intentions in the upcoming Federal election.
  These were the results of the poll
• Conservative 36.4
• Liberal 30.1
• NDP 17.4
• Bloc Quebecois 10.6
• Green/Other 5.6
• The company claimed these estimates accurate to
  within /- 3 percentage points 19 times out of
  20.

2
The next day (January 23, 2006) several million
Canadian voters cast their votes as follows.

• Conservative 36.3
• Liberal 30.2
• NDP 17.5
• Bloc Quebecois 10.5
• Green/Other 5.5
• Q. How could SES Research, with a tiny sample of
  1200 predict with amazing accuracy the voting
  behavior of several million Canadiansand ruin
  much of the suspense on election night?
• A. With careful sampling techniques!!!

3
And a more recent poll ruins Provincial Election
suspense.

• On October 8, 2007 the research firm SES sampled
  800 Ontarians on their voting intentions in the
  upcoming Provincial election. The results of the
  poll are below (actual election results on
  October 10 in brackets)
• Conservative 30.5 (31.4)
• Liberal 42.6 (42.1)
• NDP 17.5 (17.1)
• Green/Other 9.4 (8.1)
• The company claimed that these estimates were
  accurate to within /- 5 percentage points 19
  times out of 20.and they were correct!!!

4
A short and painless history of sampling

• Sampling has developed in step with political
  polling elections allow researchers to test
  sampling designs.
• The infamous Literary Digest Presidential poll (a
  huge sample that was embarrassingly wrong in its
  Presidential prediction).
• Gallup from quota to probability sampling.
• Probability Sampling A sample will be
  representative of the population from which it is
  drawn if all members have a known (to the
  researcher), non-zero chance of selection.

5
(No Transcript)
6
(No Transcript)
7

• In addition to problems with sampling people who
  are convenient to the researcher, a
  researchers personal biases (conscious
  unconscious) may affect selection in ways that
  make the sample unrepresentative of the
  population.
• Sampling bias means those selected into the
  sample are NOT typical or representative of the
  larger population from which they have been
  chosenand usually the population that the
  researcher wants to generalize results to!

8
Basic Principles Concepts in Sampling.

• A sample is representative if sample
  characteristics closely resemble population
  characteristics.
• This can only happen if ALL members of the
  population have a known, non-zero chance of being
  selected into the sample (probability samples).
• EPSEM samples ensure that all members of the
  population have an equal-chance of being
  selected.
• Probability samples are always more
  representative than non-probability samples and
  allow researchers to estimate the margin of error
  (e.g. / - 3 percentage points 19 times out of
  20).

9
Concepts in Sampling.

• Element Sampling unit about which information
  is to be collected analyzed (unit of analysis).
• Population Set of all elements that exist at
  the time of study (spatially temporally
  defined).
• Sampling Unit An entity of entities considered
  for selection at some stage of the design. In
  single sampling designs the unit element are
  identical. In complex designs, there can be
  different units (e.g., 1st stage cities 2nd
  stage blocks 3rd stage household 4th
  stage adults in households).
• Sampling Frame The list(s) of sampling units
  from which a sample is to be selected.
  Multi-staged sampling designs will have multiple
  sampling frames.

10
Simple Random Sampling (SRS)

• Simple random sampling (SRS) is the basic
  probability design and is incorporated at some
  stage in ALL probability sampling designs.
• Each unit has an n / N chance or probability
  of being selected into the sample.where n
  the size of the sample and N the size of the
  population. (e.g., n 500 N 300,000
  chance of being selected is 500 / 300,000
  .0016)
• With SRS, you need an accurate and complete
  sampling frame.each element in the population of
  interest is listed once and only once!

11
Summary of Probability Sample Designs.

• Simple Random Sampling Assign a unique number
  to each sampling unit select sampling unit
  numbers using a random number table or generator.
• Systematic Random Sampling Determine the
  sampling interval select the first unit
  randomly, select remaining units using interval
  increments.
• Stratified Random Sampling Determine strata
  select from each stratum a random sample
  proportionate (or disproportionate) to the size
  of the stratum in the population of interest.
• Multi-stage Area Sampling Determine the number
  of levels or areas and from each level or area
  select randomly.

12
(No Transcript)
13
Systematic Sampling.

• The researcher selects every k element from the
  sampling frame after a random start. (e.g., you
  want to select a sample of 100 persons from a
  population of 10,000. After a random start
  between 1 and 100, you will select every one
  hundredth individual..(k N / n 10,000 / 100
  100).where k is the sampling interval N
  is the size of the population and n is the
  size of the sample.
• If your random starting number was 14, you will
  pick the 14th person on your list, followed by
  the 114th person, followed by the 214th person,
  and so on until you have drawn your sample of 100
  people.

14
(No Transcript)
15
Stratified Sampling.

• Stratified sampling can improve the
  representativeness of our sample by ensuring
  that different groups in the population are
  adequately represented in the sample.
• You draw a specified number / percentage of
  elements from subgroups in the populationknown
  as strata.
• Common strata age groups, education levels,
  ethnic groups, language groups, political
  affiliation, gender, occupational groups.
• Randomly select your sample within strata.

16
(No Transcript)
17
More on Stratified Sampling..

• For example, the student population of a college
  is 1000. Of these, 700 (70) are from Ontario,
  200 (20) are from other provinces, and 100
  (10)are from outside Canada.
• With stratified sampling we could ensure that in
  a sample of 100 students, we obtain 70 (70) from
  Ontario, 20 (20) from other provinces, and 10
  (10) from outside Canada by randomly selecting
  within these three strata.
• This is known as proportionate stratified
  sampling.

18
And even more on Stratified Sampling.

• For example, if a population of executives in a
  major company were 10,000, and 7000 were male and
  3000 were femalewe could divide the population
  into two strata listing all male and female
  executives.
• And then use proportionate or disproportionate
  stratified sampling to construct our sample.
  With proportionate stratified sampling our sample
  would be 70 male and 30 female with
  disproportionate stratified sampling our sample
  could be 50 male and 50 female.

19
Multi-stage or cluster sampling.

• This design assumes that any population can be
  regarded as comprising a hierarchy of sampling
  units (e.g., a university can be broken down into
  faculties, departments, sections and classes
  Canada can be broken down into provinces,
  counties or regions, cities, city blocks, and
  households).
• With cluster sampling we randomly sample down the
  hierarchy of sampling units in a population of
  interest.

20
More on multi-stage or cluster sampling.

• Compile a list of all cities in Ontario and
  randomly select cities from this list.
• Within each of the selected cities, compile a
  list of all residential city blocks and randomly
  select a number of residential city blocks.
• Within each of the selected residential city
  blocks, compile a list of all households and
  randomly select our sample of households.

21
And more on multi-stage or cluster sampling.

• Can save you a lot of money!
• Especially useful when it is difficult to put
  together an adequate sampling frame.
• Mechanics of cluster sampling are
  straightforward.
• On the downside, the selection of clusters
  depends on the goals of the study, population
  distribution, and elements to be studied.
• Generally produces the least representative of
  the probability designs..and hence the least
  accurate sample estimates.

22
Non-Probability Sample Designs.

• Chance of population element being selected into
  sample is unknownas is the representativeness.
• Cheaper, easier to implement designs, suited to
  preliminary studies of small populations.
• Purposive sampling sample is drawn based on the
  experience judgement of researcher(s).
  
• Quota sampling a non-probability analogue to
  stratified samplingresearcher selects a quota of
  respondents for the sample in a non-random way.
• Convenience sampling accidental samplingit is
  anything but random.avoid accidents.
• Snowball (Referral) sampling initial members of
  the sample used as informants to find other
  elements.

23
(No Transcript)
24
(No Transcript)
25
Measurement and Questionnaire Design

• A common challenge for all social research
  methods is accurate measurement.
• Questionnaires are the most common measurement
  instrument in the social sciences.not only in
  survey research but in experimental designs,
  focus groups, and participant observation.
• Conceptual definitions are linguistic or verbal
  definitions of concepts Operational definitions
  are the set of procedures, activities or
  questions that researchers develop to empirically
  measure concepts.

26
Conceptual Dimensions of Alienation

• Powerlessness Behaviors cannot determine
  outcomes.
• Meaninglessness Minimal standards for clarity
  in thinking are lacking.
• Normlessness Socially unapproved behaviors are
  required to get socially approved goals.
• Isolation Devaluing of goals beliefs that are
  highly valued.
• Self-estrangement Behaviors inherently
  unrewarding to you, and unrelated to self-image
  and/or self-development.

27
Operationalizing Alienation

• Seeman developed survey questions that measured
  the 5 conceptual dimensions of alienation. A
  series of questions were designed to measure
  powerlessness, meaninglessness, isolation,
  normlessness, and self-estrangement.
• Example To measure powerlessness, Seeman asked
  the following question Suppose your city was
  considering a bylaw that you believed to be
  unjust or harmful. What do you think you could
  do?

28
More on operationalizing alienation

• If you made an effort to change this bylaw, how
  likely is it that you would succeed?
• If such a case arose, how likely is it that you
  would try to do something else about the bylaw.
• Would you try to influence a politician?

29
Still more on operationalization

• Survey questions operationalize our
  concepts.transform abstract concepts into
  observations.
• Operationalization enables us to test hypotheses.
• Operationalization brings theoretical concepts
  into the real world where we can measure it,
  discover causes effects, test ideas
  hypotheses.

30
Measurement

• Measurement is the process where we map
  phenomena using numbers or values.
• Questionnaires, map social phenomena using
  numbers that correspond with responses to (the
  values and value labels in SPSS)
• What is your current religious affiliation?
  
• 1. Protestant
• 2. Catholic
• 3. Jewish
• 4. Muslim
• 5. Other

31
Types of Measurement

• Nominal Measurement Assign numbers to social
  categories.no special order
• 1 5 3 2 4
  (Range)
• 
  (Phenomenon)
• What religion are you? 1. Catholic
• 
  2. Protestant
• 
  3. Jewish
• 
  4. Muslim
• 
  5. Other

32
Types of Measurement

• Ordinal Measurement Measures social phenomenon
  that can be rank-ordered or sequenced in terms of
  more or less of the phenomenon being measured.
• 1 2 3 4 5
  (Range)
• 
  (Phenomenon)
• How often do you attend religious services?
  
• 1. Never
• 2. A few times a year
• 3. Once a month
• 4. Once a week
  
• 5. More than once a week

33
Types of Measurement

• Interval Measurement Measures social phenomenon
  on the basis of equal underlying intervals on the
  measurement scale.
• 0 1 2 3 4
  5 (Range)
• 
  (Phenomenon)
• Please indicate your overall level of
  approval for the government of Ontario.
  
• 0 -------- 1 -------- 2 -------- 3
  -------- 4 -------- 5
• Do not approve
  Total Approval
• .where the meaning of 0 is arbitrary or
  created by the researcher.

34
Types of Measurement

• Ratio Measurement Measures social phenomenon on
  the basis of equal underlying intervals on the
  measurement scale with a non-arbitrary zero
  point.
• 0 1 2 3 4
  (Range)
• 
  (Phenomenon)
• How many siblings do you have?
• ____
  Number of siblings

35
More on Measurement.

• Ratio-level variables contain the most
  information . So can always measure down from
  ratio to interval, ordinal, or nominal through
  recoding variables in SPSS.
• Nominal ordinal measures have less information
  so we cannot recode nominal or ordinal variables
  up to the interval or ratio level. However, by
  summing nominal or ordinal values from 2
  variables, you can create an index that
  increases the level of measurement for the
  variable measured by the index.