NIRSA External Review

1
Data Generation, Complexity and Synthesis
Prof. Rob Kitchin, NIRSA, NUI Maynooth
2
Introduction

• Generating data is about creating empirical
  evidence
• It is not a neutral, passive exercise
• It is generation, not collection creation not
  harvesting
• It is theoretically informed
• It involves assumptions about the world
  (ontology)
• It involves assumptions about how the world can
  be validly measured (epistemology)
• It involves assumptions about what academic
  research is for (ideology)
• It necessitates a series of informed choices
  about mechanisms to elicit useful data
  (methodology)
• It is a complicated process
• It involves synthesis the drawing together of
  lots of data to create an overview a
  theoretically informed, empirically grounded
  story
• It involves carefully thinking and planning, and
  sound execution it should not be an ad hoc
  process

3
Starting with questions

• Research is about answering questions
• There are lots of different types of questions
• What is A like? What is interacting in the urban
  environment like for disabled people?
• What does A mean? What do we mean by access for
  all' and barrier free environments'?
• Is A like B? Are the planning needs of disabled
  people the same as the able-bodied community?
• Is A different from B? Do planners and disabled
  people agree on how the urban environment should
  be designed?
• Is A better than B? Is the urban planning for
  disabled people better in Labour council areas
  than Conservative run areas?
• Are A and B related? Is there a relationship
  between the size of a town and the quality of
  urban planning for disabled people?

4
Starting with questions

• Does A affect B? Does poor design and low
  accessibility of an environment decrease usage by
  disabled people?
• Does A cause B? Do disabled people use this area
  more as a direct result of its re-development to
  make it more accessible?
• Is A located where B is min/max? Are special day
  care centres located in the areas of primary
  need?
• How are A and B minimised simultaneously? Can we
  maximise the accessibility of the environment
  whilst minimising the expense?
• Why does A support B? Is a disabled access
  project receiving funding from the state for
  political reasons rather than genuine commitment
  to the access needs of disabled people?

5
Framing questions

• How we set about answering our questions is not
  as simple as it at first might seem
• Answering questions is embedded in theoretical
  assumptions and choices
• Philosophical theory ideology, ontology,
  epistemology
• The specifics and mechanics of knowledge
  production
• Positivism, Phenomenology, Marxism, etc.
• Conceptual theory substantive focus (e.g.,
  governance - coalitions, regimes, regulation,
  actor-networks, etc.)
• Philosophical and conceptual theory are highly
  inter-related
• The rejection of theory e.g., collect data and
  let it speak for itself - is in itself a
  theoretical position empiricism

6
Framing questions

• Positivism
• Positivists argue that by carefully and
• objectively collecting data regarding social
• phenomena, we can determine laws
• to predict and explain human behaviour in
• terms of cause and effect.
•  
• Positivists reject normative and metaphysical
  questions that cannot be measured scientifically.
  
•  
•          Positivism differs from empiricism
  because it requires propositions to be verified
  (logical positivism) or hypotheses falsified
  (critical rationalism) rather than just simply
  presenting findings.
•  
•          Sources of primary data are
  closed-question questionnaires and surveys
•  

Empiricism   Empiricism refers to the school of
thought where facts are believed to speak for
themselves and require little theoretical
explanation.   Empiricists hold that science
should only be concerned with objects in the
world and seek factual content about them.
  Normative questions concerning the values and
intentions of people are excluded from study as
it is claimed we cannot scientifically measure
them.   A source of primary data is
closed-question questionnaires  
7
Framing questions

• Phenomenology
• Phenomenology rejects the scientific,
  quantitative approach of positivism.
• Instead phenomenology suggests that we
• concentrate upon understanding rather than
• explaining the world.
• The goal of phenomenology is to reconstruct the
• worlds of individuals, their actions, and the
• meaning of the phenomena in those worlds to
• understand individual behaviour, without drawing
• upon supposed theories.
•  
• A source of primary data are in-depth interviews
  with people who have experienced the phenomena in
  question  

Marxism Marxists suggest that society is
structured so as to perpetuate the production of
capital. Marxists are concerned with
investigation of the political and economic
structures that underlie and reproduce capitalist
modes of production and consumption.
           To do this Marxists suggest that we
need to consider how conditions might be under
different social conditions to highlight how
society operates.            A source of primary
data is observation but also re-examines
secondary data sources with analysis consisting
of determining the dialectical (how one affects
the other) relationship between societal
structures and individuals
8
Framing questions

• Thinking through philosophy
• Naturalist/Anti-naturalist?
• Value-free/Action-orientated?
• Objective/Situated?
• Truth or truths?
• Realist/Anti-realist?
• Structure/agency?

9
Answering questions

• Questions are answered through a four-stage
  process
• data generation (e.g. interviews, questionnaires,
  surveys)
• data analysis (e.g., statistical test, discourse
  analysis, etc)
• data synthesis (pulling and weaving together
  different analyses)
• data interpretation (what does the analysis tell
  you)
• These constitute methodology a set of ordered
  techniques
• Methodology must match ontological and
  epistemological assumptions
• It might be necessary to employ more than one
  methodology to provide a sufficient answer to a
  question

10
Example study of poverty

• Empiricism Facts about poverty would be
  collected and presented for interpretation by the
  reader. (e.g. Indices of poverty - social welfare
  recipient, housing tenure, etc.).
• Positivism Poverty is explained through testing
  a hypothesis by collecting and scientifically
  testing data related to poverty (e.g.
  statistically testing whether poverty is a
  function of educational attainment).
• Phenomenology To understand poverty it is
  suggested that we need to reconstruct the world
  of people who are poor (e.g. we need to try and
  see the world through the eyes of a poor people).
  This might be attempted by talking to them about
  their life experiences.
• Marxism Poverty is explained through the
  examination of how poor people are exploited for
  capital gain (e.g. Examining whether poor people
  are poor because it is in the interests of
  capital to retain unskilled, low wage jobs rather
  than distribute fully corporate profit).

11
Answering questions validly

• All good studies aim to be valid and reliable
• Validity concerns the soundness, legitimacy and
  relevance of a research theory and its
  investigation
• Validity relating to theory concerns the
  integrity of the theoretical constructs and ideas
  that support and provide the foundations for
  empirical research
• Validity relating to practice concern the
  soundness of the research strategies used and the
  integrity of the conclusions that can be drawn
  from a study

12
Practical validity

• Construct and analytical validity both relate to
  the methodological integrity of a study
• Construct validity concerns whether data
  generation techniques are sound, measuring the
  phenomenon they are supposed to without
  introducing error or bias they are telling you
  about what you want to know
• Analytical validity concerns whether the correct
  method of data analysis has been chosen, leading
  to results that represent the data truly.
  Clearly if you have chosen the wrong method of
  data analysis then you might end up drawing
  inappropriate conclusions
• Ecological and internal validity both relate to
  the integrity of the conclusions drawn from a
  study
• Ecological validity is concerned with the
  inferences that can be made from the results of a
  study
• Internal validity concerns whether the results
  from a study can be interpreted in different
  ways can different conclusions be drawn from the
  same results?

13
Answering questions reliably

• Reliability refers to repeatability or
  consistency of a finding
• Golledge and Stimson (1997) describe three kinds
  of reliability
• (1) quixotic reliability, where a single method
  of observation continually yields an unvarying
  measurement
• (2) diachronic reliability, which refers to the
  stability of an observation through time and
• (3) synchronic reliability, which refers to the
  similarity of observations within the same time
  period.

14
Answering questions ethically

• Research ethics are concerned with the extent to
  which the researcher is ethically and morally
  responsible to her/his participants, the research
  sponsors, the general public, and her/his own
  beliefs
• professional approach to research and focuses
  upon issues such as privacy, confidentiality, and
  anonymity
• Researcher should weigh carefully the potential
  benefits of a project against the negative costs
  to individual participants

15
Data synthesis

• Pulling and weaving together different analyses
• Different methods open up different aspects of a
  phenomenon methods are oligoptical in nature
  (partial views from selective locations) rather
  than panoptical (providing a full view from a
  gods-eye position)
• Different methods can offer results that are
  often paradoxical, contradictory, puzzling
• Is this because the phenomena are paradoxical,
  complex and messy or because of methodological
  issues?
• In synthesising analyses we need to think
  carefully about how data were generated and the
  validity in processing in different ways
• We need to think through any puzzling results and
  what paradoxes and contradictions might mean

16
(No Transcript)
17
(No Transcript)
18
Data interpretation

• What does the synthesis tell you?
• What insights have you gained?
• What is the answer to your question?
• relationships
• themes
• Effected by external and internal validity
• What are the potential effects of your
  interpretation?
• on theory
• on practice and policy
• What lessons can be learnt from your research
  praxis?
• how might methodology be tweaked, etc?