Key issues for longitudinal research design: Lessons from the Growing Up in Scotland study

1
Key issues for longitudinal research
designLessons from the Growing Up in Scotland
study
Paul Bradshaw
2
Definitions of longitudinal research

• In the broadest sense longitudinal research
  involves the follow up of any set of entities in
  which changes can be observed over time.
• Individuals
• Households
• Institutions such as hospitals or schools
• Nations
• Most commonly focussed on individuals
• No recognised definition of what period or what
  number of follow-ups constitutes longitudinal
• Aim is usually to determine causes and processes
  which lead to change and/or to particular
  outcomes

3
The UK experience

• The UK has a long, and highly regarded, history
  of (particularly quantitative) longitudinal
  social enquiry
• A number of ongoing internationally renowned
  longitudinal studies providing information on the
  life histories of people as they move from birth
  to old age
• Current QNLR being undertaken in Britain
  includes
• Birth cohort studies
• Age cohort studies
• Family/Household panel studies
• Area studies
• Census-based studies

4
The challenges of longitudinal research

• The success of a longitudinal study depends on
  stable leadership from a committed principal
  investigator and a team of highly skilled
  researchers
• (Bynner et al, 2006)

Data collection
Data analysis
Sample
Ethics
Cost
Duration ( relevance)
5
The Sample

• Two main issues
• Sample design
• Non-response and attrition

6
Sample design considerations

• What is the population of interest?
• All vulnerable children?
• Sub-groups of interest? E.g. with particular
  characteristics age, family circumstances, area
  where they live, interventions received
• What do you want to be able to say about them?
• Use sample to generalise to population?
• Compare outcomes and experiences between children
  in different groups?
• How will you find/recruit them?
• Is there a sampling frame with the info you need?
• Will you need the help of an agency/organisation
  to recruit?

7
Sampling precision (1)

• When using a sample, the data produces
  estimates of what the real value may be in the
  population. The precision of these estimates is
  affected by
• Sample size
• Sample clustering
• Measuring sample precision
• Use confidence intervals and standard errors
• Confidence interval (CI)
• Typically 95 meaning 95 times out of 100 this
  interval will capture the true population value
  that we are trying to estimate
• Expressed as e.g. 64 (/- 6) suggesting the
  true value is somewhere between 58 and 70
• Differences will usually have to be of the
  magnitude of the CI for it to be statistically
  significant e.g. in example above, an increase
  or decrease of around 6 would be necessary.
• Smaller sample sizes produce larger CIs requiring
  greater change for statistically significant
  differences to be detected

8
Sampling precision (2)
Sample numbers required in each group to
demonstrate significant differences (with a power
of 0.8)
Size of difference between groups (percentage points) Size of difference between groups (percentage points) Size of difference between groups (percentage points) Size of difference between groups (percentage points)
Lower of the two percentages 5 7 10 15
10 687 374 200 101
25 1252 653 330 153
50 1566 797 389 170
75 1095 541 251 101
90 436 195 75 -
9
GUS response and attrition rates
No. cases issued No. cases achieved Response rate As of sw1 achieved
Birth cohort
Sweep 1 6583 5217 79 100
Sweep 2 5217 4512 86 86
Sweep 3 4665 4193 90 80
Sweep 4 4394 3994 91 77
Child cohort
Sweep 1 3605 2859 79 100
Sweep 2 2859 2500 87 87
Sweep 3 2599 2332 90 82
Sweep 4 2460 2200 89 77
10
GUS Sweep 5 response by age of mother
11
Non-contact and refusal rates
No. cases issued No. non- contact non-contact No. refused refused
Birth cohort
Sweep 2 5217 162 3 316 6
Sweep 3 4665 102 2 268 6
Sweep 4 4394 93 2 203 5
Child cohort
Sweep 2 2859 100 3 153 5
Sweep 3 2599 51 2 142 5
Sweep 4 2460 51 2 136 6
12
Sample maintenance strategies

• Keeping in touch
• Tracing
• Keeping respondents informed
• Valuing and appreciating respondents (including,
  but not restricted to, use of incentives)
• Boosts of key sub-groups of interest and on-going
  sample refreshment
• Making reasonable demands

13
Sample maintenance keeping in touch

• Potentially willing respondents can be lost by
  virtue of moving home.
• Important to establish effective procedures for
  obtaining updated contact information.
• Techniques include
• Collecting as much contact information as
  possible at first contact including
  telephone/mobile numbers and e-mail
• Information about stable contacts someone who
  knows the respondent and would know where they
  had moved to
• Regular mailings with return address
  undelivered items act as address checks ahead of
  fieldwork
• Updates from administrative and service databases

14
Sample maintenance keeping respondents informed

• In any research project, it is considered
  important to provide good information to
  respondents about the purpose and nature of
  research
• Respondents must understand the longitudinal
  nature of the study so that they also recognise
  why they are repeatedly visited and why they are
  irreplaceable
• Many studies adopt a branding approach a name
  and logo that clearly identifies the study and
  which is easily recognisable by respondents
• Websites can be extremely useful in providing
  more detailed information

15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
Sample maintenance valuing and appreciating
respondents

• If respondents are going to continue to take part
  in a longitudinal study they need to feel that
  their time and effort is valued and is
  worthwhile.
• This can be demonstrated through
• The interviewer
• Thank you letters sent after each interview
• Providing evidence that survey findings have been
  used
• Illustrating the impact they have had on
  government policy
• Use of incentives
• Non-financial rewards gifts such as pens,
  fridge magnets, calendars

20
Being part of NCDS

• Theres one thing Im guaranteed, all of my
  entire life, is one birthday card.
• Ive always liked being part of it, Ive always
  enjoyed being part of it because its different
  and Im quite proud of it really.
• I think I could almost put this in as a landmark
  in my life.
• I havent kidded on about anything in my life,
  warts and all, Ive been honest and Ive said it
  all.
• .. I think, you know, it does make you feel
  special in certain ways, like really..
• I havent ever seen anything negative in it
  actually, not anything negative at all..Ive
  always felt comfortable and I always know that if
  I dont want to answer a question I dont have
  to I think its just been fascinating

21
Recognising the importance of NCDS

• I can remember becoming part of the NCDS, I
  didnt understand it.... I think Ive just
  grown to understand it more as Ive got older and
  the importance of it and how its helping
  everybody really, thats what I think.
• Well, I think to a certain extent youve got to
  say its mainly for helping others, you know.
  Like you say its no benefit to me to do it, but
  then again its no skin off my nose not to do it,
  so. Its one of those things like, you know.I
  dont see a reason not to
• I think its interesting that theyre actually
  following all these people, right through their
  life and they find out comparisons. Aye, I
  think its really. thats why I take the bother
  to, aye, come. I want to be part of it because
  Ive been in it all my life.

22
Data collection what to ask and when

• Decisions on content of data collection require
  good planning important to ask the right
  questions at the right time
• Take a longitudinal view
• What stage are your subjects at?
• What stage will they reach?
• What situations, characteristics or contexts
  might you want to compare between those two
  stages?
• What might be significant at stage 1 when you are
  comparing outcomes at later stages?
• Decisions on content will usually be theory or
  hypothesis-based
• What if you miss something?

23
Data collection intervals between fieldwork

• How often do you follow-up your sample? This is
  dependent upon
• Respondent burden
• Developmental stages, processes or transitions
  that you are interested in
• Budget
• There is no recognised nor legitimate pattern
  its largely dependent upon the objectives and
  focus of the research

24
Follow-up intervals of selected UK child cohort
studies
Study Childs age in years Childs age in years Childs age in years Childs age in years Childs age in years Childs age in years Childs age in years Childs age in years Childs age in years Childs age in years Childs age in years Childs age in years
Study 0 1 2 3 4 5 6 7 8 10 11 16
NCDS (1958)
BCS (1970)
MCS (2000)
GUS BC (2005)
GUS CC (2005)
ALSPAC (1991)
25
GUS Sources of data
Sw1 Sw2 Sw3 Sw4 Sw5 Sw6
(2005/6) (2006/7) (2007/8) (2008/9) (2009/10) BC only (2010/11) BC only
Main carer Main carer Main carer Main carer Main carer Main carer
Partner
Child height weight Child height weight Child height weight
Cognitive assessmts Cognitive assessmts
Health records Health records Health records Health records Health records Health records

School records School records

26
Data analysis

• Benefits of longitudinal data
• Measuring change over time
• Flows into and out of states (e.g. poverty,
  unemployment, being looked after)
• The effects of change, or of state durations, on
  outcomes
• Impact of interventions
• Individual development
• Temporal ordering of events
• Improved control for omitted explanatory
  variables
• Improved control for the effects of previous
  states
• Exploring the effects of ageing and cohort
  membership

27
Data analysis

• Drawbacks of longitudinal data
• Data management
• Complex structure/relations
• Complex variable/samples
• Resultant file and variable management requires
  training and skills of good practice
• Software issues
• Complexity of methods
• Some methods only available via specific software
  packages

28
Longitudinal models

• Two main modelling approaches in social science
  research
• Event history analysis, time to an event
• (Also known as duration analysis survival
  analysis failure time duration economics
  hazard modelling)
• Panel data analysis
• Regression models suitable for repeated
  observations
• Time generally conceptualised as being discrete
• Extension of standard regression models
• Closely related to multilevel modelling
• (Simple methods can also be used)

29
Ethical issues (1)

• Confidentiality, data security and data access
• Safeguarding confidentiality through
• Restricting the data that are released
• Controlling the arrangements under which
  potentially disclosive data is released
• Data access/release
• Providing data for researchers to analyse on
  their desktop with the level of sensitive detail
  restricted
• Providing more sensitive data under a special
  licence
• Remote access where raw data is never released
  and analyses are run in house on behalf of
  researchers who receive edited outputs
• Safe settings where researchers are required to
  visit a protected site where access to the data
  is carefully managed.

30
Ethical issues (2)

• Informed consent
• What is informed consent?
• Respondents must understand that the project is
  longitudinal in nature and what that means for
  them
• They are free to withdraw at any time from the
  project as a whole or any aspect of it
• One-off or repeated consent?
• No universal practice, but
• Some form of repeated consent would normally be
  involved

31
Summary

• Longitudinal research in any methodological
  discipline presents a set of core challenges
• The key issue is to consider, in detail, how the
  fundamental temporal nature of the project
  affects the basic aspects of its design
  including
• The sample
• Data collection
• Data analysis
• Ethics
• Requires a more complex design, but a good design
  will produce tremendously valuable data

32
(No Transcript)