Mixing science and intuition: the process of synthesising data from a longitudinal mixed methods study of volunteering

1
Mixing science and intuition the process of
synthesising data from a longitudinal mixed
methods study of volunteering

• Rose Lindsey and Liz Metcalfe, University of
  Southampton
• Third Sector Research Centre
• ESRC grant no. ES/K003550/1

2
Presentation aim

• To explore the challenges encountered when
  combining longitudinal qualitative and
  quantitative secondary data to study volunteering
  across time
• Key challenge What do we mean when we talk about
  synthesising, integration, combining, mixing,
  interweaving, blending, merging? (Bryman, 2008)
• Do we think our methods of combining have
  actually worked?

3
Presentation outline

• Part 1 Designing the project
• Introduction to Continuity and Change project
• Discussion of the methodological challenges faced
  within the mixed-method research design
• Bringing different data sources and findings into
  dialogue
• Part 2 Challenges in practice
• Exploring the analytical challenges faced when
  putting the design into practice
• Working across methodological paradigms
• Understanding the effect of, and working across,
  time
• Design versus practice

4
Part 1 The Continuity and Change Project

• Aim To explore individual attitudes and
  behaviours towards volunteering, and individual
  views on the role and responsibility of the state
  towards provision for social need, across a
  period of thirty years.
• Design Concurrent use of longitudinal
  mixed-methods to analyse secondary data
• Time-frame 1981-2012, encompassing different
  periods of economic adversity and prosperity
• Project website http//longitudinalvolunteering.w
  ordpress.com

5
Choice of secondary data sets

• Qualitative data

• Quantitative data

• The Mass Observation Project
• Aim to capture experiences, thoughts and
  opinions of individuals
• A national panel of volunteers writing in
  response to themed questions or directives
  (1981 to present day)
• Longitudinal data following the same people
  through thirty years of their life-course

• British Household Panel Survey/Understanding
  Society
• Aim to understand individuals and households
  social and economic change
• A national panel of the British population and
  volunteers (1991 to 2012)
• Longitudinal data
• British Social Attitudes Survey
• Aim to track peoples changing social, political
  and moral attitudes
• A national survey of the British population (1983
  to 2012)
• Cross-sectional data

6
Why did we use mixed-methods?Enhancing strengths
and offsetting weaknesses
Our research design aimed to potentially offset
the respective weaknesses of these two analytical
methodologies by taking advantage of their joint
strengths to provide a completeness, and
comprehensive picture (Bryman, 2008, p.91)
Study strengths Study weaknesses
Qualitative Provides depth and nuance relating to the complex reasons why people behave in a certain way, or hold particular viewpoints Offers potential insights into how and why perspectives change or continue over time Enables insights into the connection between the life-course and routes into volunteering Not representative of the population Too much data
Quantitative Representative of volunteers within the population Can formally test how volunteering behaviour and attitudes change over time Offers potential insights into contextual re external events affect on change or continuity over time Insight into motivations and barriers are limited Limitations re understanding how individual time and the life-course affect volunteering
7
Multi-layered picture of volunteering behaviour
In-depth analysis of individual volunteers
Contextual social, economic and political events
over time
Behaviour and attitude analysis for volunteers
within the population
Sample size decreases
Focus on individuals increases

• In-depth analysis of individual volunteers

Contextual social, economic and political events
over time
8
Bringing secondary data sources, analyses and
findings into dialogue

• We aimed for three types of mixed-method
  dialogue
• across the lifetime of the project, described by
  Tashakkori and Teddlie (2008, p.104) as a
  continuous feedback loop, to enable an
  iterative research process
• some direct comparisons between qualitative and
  quantitative analyses where there was a fit
  between the data
• combining substantive findings so that the sum of
  our joint knowledge claims would be greater than
  our individual findings

Qualitative data
Substantive findings
Quantitative data
Project beginning
Project end
9
Study design challenges-sample fit

• The Mass Observation Project

• British Household Panel Survey/Understanding
  Society

• Sample restricted to available volunteering data
  (every other year between 1996 to 2011)
• N2067

• The Mass Observation Project (MOP)
• 15 directives (sets of questions) were selected
• N38
• 2 samples were taken to provide a range of ages
  and occupations
• Sample 1, n20 were writers from 1981 to 2012
• Sample 2, n18 were younger and wrote for shorter
  periods of time

British Social Attitudes Survey

• Questions of volunteering only asked a limited
  number of times
• Number of people each year mean (sd) 3393 (711.7)

10
How the three datasets complement each other,
temporally and thematically
11
Part 2 Challenges in practice

• Three main challenges were present throughout the
  project
• Working across methodological paradigms
• Understanding the effect of, and working across
  time
• Putting the design into practice

12
Working and communicating across methodological
paradigms

• Working across methodologies we encountered some
  challenges
• Differences in terminology
• Forming definitions
• Timings/speed of analysis
• Methodological standpoints differences in the
  types of questions that are being addressed
• Conceptions of time

13
How time fits together

• The way that these multiple perceptions of time
  interact and intersect (or not) was at the heart
  of the mixed methods effort for our research
  project.
• the flow of personal biographical, narrative,
  retrospective, life-course, individual time
• chronological time, moving from one year to the
  next
• contextual public/collective time related to
  chronological time

Hi, Im Sarah
Children
Retirement
Marriage
1981 1984 1987 1990 1993 1996
1999 2002 2005 2008 2011
2013
Recession
Recession
Double-dip recession
14
Multi-layered picture of volunteering behaviour
Biographical time
In-depth volunteer analysis
Behaviour and attitude analysis for volunteers
within the population
Sample size decreases
Focus on individuals increases
Chronological time
Contextual time
Changes in social, political and moral attitudes
over time
15
Design versus practice

• Longitudinal mixed-methods are more complicated
  than a single methodological approach
• Over-estimation of mixed methods It has not been
  possible to answer all of the designed research
  questions with the data chosen, the fit of the
  samples and the timing of the analysis
• Did we achieve our mixed method dialogue?
• Paradigm, background, and terminology differences
  make maintaining a mixed-method dialogue
  difficult
• How time fits together in practice time does not
  relate directly between different methodologies
• Has the project benefited from using
  mixed-methods?

16
References

• Bryman, C. (2008) Why do Researchers
  Integrate/Mesh/Blend/Mix/Merge/Fuse Quantitative
  and Qualitative research?, in M.M. Bergman (ed.)
  Advances in Mixed-Methods Research, London
  Sage. pp 87-100.
• Tashakkori, A. and Teddlie, C. B. (2008) Quality
  of Inferences in Mixed Methods Research Calling
  for an Integrative Framework in in M.M. Bergman
  (ed.) Advances in Mixed-Methods Research,
  London Sage. pp.101-119

17
Thank you for listening, any questions?

• Contact details
• R.Lindsey_at_soton.ac.uk
• E.Metcalfe_at_soton.ac.uk
• Project website
• http//longitudinalvolunteering.wordpress.co
  m/

18
Back-up slides
19
The challenges of analysing secondary data over
time

• Quantitative
• Variations in data collection process were
  difficult to uncover
• The questions that were asked limits the data
  available
• Data collected is set within the present time,
  only part of the life-course is recorded
• Qualitative
• Inconsistent descriptions of the life-course at
  different time-points
• Lack of awareness of the what is happening within
  the present time
• Accuracy of retrospective writings

20
Study design challenges

• The mixed-method design framed the study, and
  influenced how well the data sources fitted
  together. Compromises around the following
  choices needed to be made
• Choice of secondary data sources
• Choice of timing of analyses
• Choice of samples and how these substantively fit
  together
• Choice of samples and how these fit together
  across time (thematic and temporal bunching)

21
Concurrent mixed method design
Our research design aimed to potentially offset
the respective weaknesses of these two analytical
methodologies by taking advantage of their joint
strengths to provide a completeness, and
comprehensive picture (Bryman, 2008, p.91)
22
Cross-sectional or Longitudinal?Synchronic or
Diachronic?

• The length of chronological time being researched
  affects our perceptions and understandings of
  behaviour and attitudes
• Longitudinal/diachronic following a person
  through time
• Cross-sectional/synchronic A certain point in
  time

Hi, Im Sarah
Volunteered
Do not volunteer
Increasing age