SII: Quantitative Methods and Surveys

1
SII Quantitative Methods and Surveys

• Tuesday 22nd January

2
Outline

• Introduction to Quantitative Methods
• The Hypodeductive Method
• Units of Analysis
• Sampling
• Variables (and questions)
• Analysis
• Univariate
• Bivariate (and multivariate)
• Strengths of Survey Research
• Weaknesses of Survey Research

3
Defining Terms

• Quantitative MethodsAre used to answer any
  counting related question How many? What
  proportion?
• Survey ResearchInvolves analysis of responses to
  a standardized questionnaire, containing a
  battery of questions.
• Primary (Data) AnalysisIs analysis conducted by
  the investigator(s) or institution that collected
  the data.
• Secondary (Data) AnalysisIs any further analysis
  of an existing dataset that produces results or
  conclusions other than those produced as a result
  of the first report on the inquiry. Often carried
  out by different people than those that collected
  the data.

4
Where quantitative data comes from

• Primary Sources
• Surveys
• Content analysis
• Observation studies
• Archival studies
• Other
• Secondary Sources
• Official statistics (surveys and other material)
• Archived academic surveys
• Other archived datasets
• A lot of secondary datasets are now available on
  the web and access is free for those in academic
  environments.
• You can browse some data at UK Data Archive
  www.data-archive.ac.uk/
• Economic and Social Data www.esds.ac.uk/
• Office for National Statistics
  www.statistics.gov.uk/

5
The typical hypodeductive structure of
quantitative research
PHASES
PROCESSES

• Theory
• Deduction
• Hypothesis
• Operationalization(research design)
• Data Collection
• Data-organization
• Data Analysis
• Interpretation
• Results
• Induction

From Corbetta (2003 59)
6
From Theory to Hypotheses

• Theory involves wide-ranging statements about the
  world. These are located at a high level of
  abstraction and generalization. They are often
  derived from empirical patterns and give rise to
  empirical forecasts.
• Quantitative analysis is usually involved in
  empirically testing particular hypotheses that
  are derived from theory.
• Hypotheses are general statements at a lower
  level of abstraction. They involve particular
  relationships (and directionality) between two
  (or more) concepts.
• Sometimes different theories will give rise to
  competing hypotheses. These can be empirically
  arbitrated. For example
• From gender role theory When married women earn
  more than their husbands they violate normal
  gender roles. They are therefore likely to
  increase the amount of housework that they do in
  order to compensate.
• From dependency theory The person who earnsthe
  most will have the most power. Therefore when
  women earn more than their husbands they have
  more power. And they will reduce the amount of
  housework that they do.

7
Unit of Analysis

• In order for any hypothesis to be tested
  empirically it needs to be located.
• Units of analysis are the things that are to be
  compared or analysed.
• Example If I want to investigate revolution I
  may use as my unit of analysis
• Individuals investigating who is more or less
  likely to become involved in revolutionary
  ferment
• Countries investigating what sort of society is
  more/less likely to become revolutionary
• Rebellions investigating which rebellions are
  more/less likely to develop into fully fledged
  revolutions
• Literature investigating which books are
  more/less likely to be published in revolutionary
  epochs
• etc.
• Corbetta (2003) lists the following types of unit
  of analysis the individual the aggregate of
  individuals the group-organization-institution
  the event and the cultural product.

8
A question of sampling

• Sometimes we can study every example of the thing
  that we are interested in the whole population.
• But most of the time this would be too difficult
  or time consuming.
• So we usually study just a sample of the cases
  that we are interested in.
• What is most important in selecting a sample is
  that it is representative of the population.
• When a sample is representative we can make
  inferences about the population based on the
  sample.

9
What is a Representative Sample?

• To be representative the sample should accurately
  reflect the range of possible responses/attitudes/
  behaviours of the whole population (n.b. this is
  not the population of the country, but the
  population of sociological interest).
• Since we may not know what that range is, we
  cannot know how to select a sample that is
  representative.
• Therefore the best that we can do is to ensure
  that every case (and so every attitude/behaviour)
  has an equal chance of being included into the
  sample.
• This is the Equal Probability of Selection Method
  (EPSM). And the sample that results is known as a
  Probability Sample.
• The central principle in a Probability Sample is
  random selection.

10
A Simple Random Sample
11
What is a Representative Sample?

• Some random samples are more complex than this
  involving clustering or stratifying. However
  these are still based on probabilities and so we
  can mathematically estimate the probability that
  any one case/person be selected.
• Not all samples are probability samples.
• Types of Non-Probability Sample
• Reliance on available subjects
• Purposive or judgemental sampling
• Snowball sampling
• Quota sampling
• Some of these samples may be relatively
  representative. However they are less likely to
  be representative of attitudes/characteristics
  that we did not foresee.
• Additionally, since non-probability samples do
  not involve an EPSM findings cannot be used to
  make inferences about the whole population.

12
Variables

• Quantitative Analysis involves the study of
  variables.
• Variables are attributes that vary across cases,
  and/or within a case over time.
• For example, gender, age, happiness, political
  association, occupation, number of students on a
  course
• The process of going from a concept to a variable
  involves operationalisation of the research
  question.
• When variables are produced in surveys they are
  often the product of closed questions.
• Closed questions are questions in which possible
  answers are given and the respondent selects from
  these all of the questions in last-weeks survey
  were closed as you did not have the option to
  write your own answers.

13
Types of variable

• Nominal (or categorical) i.e. ethnicity,
  religion, favourite colour, or

• What is your gender?
• ? Male
• ? Female

14
Types of variable
Thinking back over your first term at Warwick
(from the time you arrived here until Christmas
break), what do you think was the longest period
that you went WITHOUT having an alcoholic drink?
? Less than a day (i.e. you drank every
day) ? One or two days ? Three to six days
? Between one and two weeks ? Between two weeks
and a month ? More than a month, but
less than the full term ? The whole term
(i.e. you did not have a single drink)

• 2. Ordinal(categories are in order) i.e. social
  class, status, agreement/disagreement scale,
  or

15
Types of variable

• 3. Interval/Ratiomathematical operations
  possible. i.e. age, income, hours of work, or

How many alcoholic drinks have you consumed in
the last 7 days? ____________
16
Problems with survey questions

• Survey questions need to be
• Exhaustive that everyone fits into one category
• Exclusive so that everyone fits into only one
  category (unless specifically required to tick
  as many as apply).
• Unambiguous so that they mean the same to
  everyone and all responses are comparable.
• When one or more of these is violated, even if
  researchers are trying hard to be unbiased, the
  survey data will reflect not reality but the
  specific interpretations of each respondent, and
  since the researcher has no way of knowing what
  these are, she will have no way of knowing what
  she is analysing.
• For example How did you calculate your answer to
  the following How many alcoholic drinks have you
  consumed in the last 7 days? How did you work out
  what counted as one drink? Could you remember
  how many youd had?

17
Statistical Analysis

• Analysis can be
• Univariate involving just one variable at a
  time
• Bivariate involving two variables
• Multivariate involving three or more variables.
• Statistical Analysis can aim to
• Describe called descriptive statistics
• Make inferences from a sample to the population
  called inferential statistics
• Analyse relationships between variables called
  analytic statistics

The choice of statistical technique depends on
both the aims of the researcher and the types of
variable to be analysed.
18
Univariate Analysis

• We are interested in the form taken by the
  distribution of cases.
• This analysis is usually descriptive (although
  sometimes it involves inference).
• Where variables are categorical i.e. nominal or
  ordinal we use the mode and pictorial
  representations (such as pie-charts and
  bar-charts). We also give percentages of cases
  falling into each category.
• With interval level data we can go beyond the
  pictorial (although we will often start by
  looking at a chart called a histogram to get a
  feel of the data).
• We want to be able to summarise data as
  efficiently as possible so that we can see the
  wood for the trees.

19
Nominal or Ordinal variables can be represented
with pie-charts
The modal (most common) response was that
students went a maximum of 3 to 6 days without a
drink in their first term. Because this is an
ordinal variable we can add together the red and
light blue pieces of the pie and say that about
20 of you managed to go more than a whole
month.
20
Categorical Variables Dichotomous Variables.
When we are studying whether or not people have
done something there are only two possible
answers people have or have not done this thing
these are examples of dichotomous variables.
Describing dichotomous variables usually takes
the form of saying what proportion of people fall
into one of the two categories in this case,
those who have done the thing.
Sociology students (2007/8 and 2006/7) that had
ever consumed the following
21
A biased sample?

• To be representative the sample should truly
  reflect the range of possible responses/attitudes/
  behaviours of the whole population. So
• Question Since not every Warwick sociology
  student filled out a survey last week (as some
  were not in lecture), how representative do you
  think that the Survey of Warwick Sociology
  students was?
• Specifically, given that it asked questions about
  drink and drugs, do you think that the people who
  were in lecture last Tuesday were representative
  of those who were not here as well?
• One way of thinking about this is to ask whether
  there is likely to be differences between those
  people who do and who do not come to lecture in
  terms of how theyd have answered the questions.
• ? It may be that there is a correlation between
  taking drugs/drinking more and non-attendance at
  lecture. Therefore the sample of students may be
  biased, especially in relation to this topic.

22
Describing Interval-Ratio Variables
Interval-ratio variables have meaningful
response-categories. Their central tendency can
be described with a mean. And the amount of
variation from (or spread around) the mean, can
be described with the standard deviation. Interval
-ratio variables can be graphed with a histogram.
23
Univariate Analysis summarising data

• When we summarise data we look at
• Measures of location (or central tendency)
• Mean what people refer to as the arithmetical
  average
• Mode the most common value (or peak)
• Median if we place values in order, the middle
  one.
• Measures of dispersion
• The standard deviation based on the difference
  between (individual) data points and the
  (arithmetic) mean (actually the square root of
  the average of these).
• The range as in the everyday sense the
  largest value minus the smallest (i.e. height of
  tallest person minus height of shortest)
• The inter-quartile range one cuts off the
  highest and lowest 25 percent of data values and
  calculates difference between the new extremes
  (upper quartile minus lower quartile).

Note The standard deviation is the most commonly
used measure of dispersion because it provides
part of the solution to assessing sampling error
in random sample designs i.e. helps us to judge
how close our results (from a sample) are likely
to be to the underlying population
characteristics this is the essence of
statistical inference.
24
Bivariate and Multivariate Analysis

• is used to examine and specify relationships
  between two or more different variables.
• The relationships that are specified are
  probabilistic it is not that every man will do
  x and every women wont rather that men are
  more likely to do x.
• Furthermore quantitative analysis cannot explain
  why relationships exist, it can just show that
  they appear to and help to specify the associated
  social factors.
• Only things that can be measured and have been
  included in a statistical model can be found
  to be associated with one another.

25
Example of Bivariate Analysis Comparing Groups
Histograms showing responses to the question How
many alcoholic drinks have you consumed in the
last 7 days? by Gender
When you are comparing an interval-ratio variable
across groups you can compare their means and
medians
Male
Female
You can also look at differences in the shape of
the histograms here you can see that the female
histogram goes down much more steeply.
26
Example of Bivariate Analysis Comparing Groups
Bar chart showing EXPERIENCES AFTER DRINKING of
First Year Sociology students at Warwick, by
Gender (only students whove drunk alcohol since
at Warwick are included)
Where the two variables are both nominal we can
compare the proportion of cases (or people) who
fall into different groups. However just because
groups seem to differ, does not mean that there
is a relationship that would persist in the
population at large or one that is meaningful.
More statistical tests would be required to
evaluate this.
27
Example of Bivariate Analysis Comparing Groups
Comparison 2 Students from 2006/7 cohort and
from 2007/8 cohort.
Do you think that the two years are similar? Does
this give you more or less confidence that the
figures for this year are a true representation
of student life?
28
Juggling the numbers or doing research?

• Researchers who conduct quantitative analysis
  are responsible for making clear to readers the
  basis they make any claims. This requires
  specifying, among other things
• The number of cases in any category, or in any
  analysis
• Their sampling process
• The way in which questions and categories were
  constructed.

29
Strengths of Survey Research

• Useful in describing the characteristics of a
  large population.
• Make large samples feasible often relatively
  quick (and telephone/postal surveys can be
  conducted at a distance).
• Flexible many questions can be asked on a given
  topic.
• Relatively impersonal form of research can be
  good for asking sensitive questions that people
  are uncomfortable talking about.
• Easy to standardize interactions.
• Reliable (and replicable).
• Therefore relatively transparent methodology.

30
Weaknesses of Survey Research

• Can seldom deal with the context of social life.
• Inflexible - in that it requires that the
  researcher knows what to ask about before
  starting (and therefore poor for exploratory
  research).
• Subject to artificiality the product of
  respondents consciousness that they are being
  studied. This can be exacerbated where there is a
  power-relationship between the person studying
  and the person being studied.
• Weak on validity.
• Poor at answering questions where individual is
  not the unit of analysis.
• Usually inappropriate for historical research.
• Particularly weak at gathering at certain sorts
  of information
• Highly complex or expert knowledge
• Peoples past attitudes or behaviour
• Subconscious (especially macro-social) influences
• Attitudes (or at least embodied attitudes)
• Shameful or stigmatized behavior or attitudes
  (especially in face-to-face interview better in
  self-completion surveys)
• We will develop these criticisms more next week.