Interrupted Time Series: What, Why and How

1
Interrupted Time SeriesWhat, Why and How
An Example From Suicide Research

• Karen Smith

2
Acknowledgement

• Motivated by consultancy work with the Centre for
  Suicide Research, University of Oxford
• All analyses and graphs produced by Helen Bergen,
  Centre for Suicide Research

3

• Motivating example
• What is Interrupted Time Series?
• Why use it?
• Design issues
• Analysis issues
• Guidelines on use

4
Motivating Example

• Between 1997 and 1999 the analgesic co-proxamol
  was the single drug used most frequently for
  suicide by self-poisoning in England and Wales,
  with 766 over the 3 year period
• There is a relatively narrow margin between
  therapeutic and potentially lethal levels
• Death occurs largely because of the toxic effects
  of dextropropoxyphene on respiration and cardiac
  conduction
• MHRA conducted a review of the efficacy/safety
  profile
• Committee on Safety of Medicines advised
  withdrawal from use in the UK, the final date
  being 31 December 2007
• Patients who find it difficult to move to an
  alternative drug can still be prescribed
  co-proxamol

5
The Problem

• How to evaluate the impact of the announcement to
  withdraw co-proxamol on
• Prescribing of analgesics
• Mortality involving co-proxamol
• Mortality involving other analgesics
  (substitution of method is of concern)

6
Available Data

• Quarterly data on prescriptions of co-proxamol,
  cocodamol, codeine, codydramol, dihydrocodeine,
  NSAIDs, paracetamol and tramadol (from
  Prescription Statistics department of the
  Information Centre for Health and Social Care,
  England, and Prescribing Service Unit, Health
  Solutions Wales)
• Quarterly data on drug poisoning deaths
  (suicides, open verdicts and accidental
  poisonings) involving co-proxamol alone,
  cocodamol, codeine, codydramol, dihydrocodeine,
  NSAIDs, paracetamol and tramadol, based on death
  registrations in England and Wales (from ONS)
  single drug, with and without alcohol
• Quarterly data for overall drug poisoning deaths
  and for all deaths receiving suicide and
  undetermined verdicts

7
Simple Analysis

• Compare the proportion of deaths involving
  co-proxamol prior to the legislation with
  proportion following legislation
• Compare total number of poisoning deaths before
  and after legislation
• Time series plots of prescriptions and deaths
• Co-proxamol withdrawal has reduced suicide from
  drugs in Scotland, E. A. Sandilands D. N.
  Bateman, British Journal of Clinical
  Pharmacology, 2008.

8
Whats Wrong With This?

• Ignores any trends, both before and after change
  in legislation (or intervention in a more general
  setting)
• Ignores any possible cyclical effects
• Doesnt pick up on any discontinuity
• Variances around the means before and after the
  intervention may be different
• Effects may drift back toward the
  pre-intervention level and/or slope over time if
  the effect wears off
• Effects may be immediate or delayed
• Doesnt take account of any possible
  autocorrelation

9
A Solution Interrupted Time Series

• A special kind of time series in which we know
  the specific point in the series at which an
  intervention occurred
• Causal hypothesis is that observations after
  treatment will have a different level or slope
  from those before intervention the interruption
• Strong quasi-experimental alternative to
  randomised design if this is not feasible

10
Ramsay et al, 2003
11
The Model

• Use segmented regression analysis (Wagner et al,
  2002)
• Yt ß0 ß1 x timet ß2 x interventiont ß3 x
  time_after_interventiont et
• Yt is the outcome
• time indicates the number of quarters from the
  start of the series
• intervention is a dummy variable taking the
  values 0 in the pre-intervention segment and 1 in
  the post-intervention segment
• time_after_intervention is 0 in the
  pre-intervention segment and counts the quarters
  in the post-intervention segment at time t
• ß0 estimates the base level of the outcome at the
  beginning of the series
• ß1 estimates the base trend, i.e. the change in
  outcome per quarter in the pre-intervention
  segment
• ß2 estimates the change in level in the
  post-intervention segment
• ß3 estimates the change in trend in the
  post-intervention segment
• et estimates the error

12
Threats to Validity

• Forces other than the intervention under
  investigation influenced the dependent variable
• Could add a no-treatment time series from a
  control group
• Use qualitative or quantitative means to examine
  plausible effect-causing events
• Instrumentation how was data collected/recorded
• Selection did the composition of the
  experimental group change at the time of
  intervention?
• Poorly specified intervention point diffusion
• Choice of outcome usually have only routinely
  collected data
• Power, violated test assumptions, unreliability
  of measurements, reactivity etc.

13
Design Considerations

• Add a non-equivalent no-treatment control group
• Add non-equivalent dependent variables
• Intervention should not affect but would respond
  in the same way as primary variable to validity
  threat
• Remove intervention at a known time
• Add multiple replications
• Add switching replications

14
Problems

• Interventions implemented slowly and diffuse
• Effects may occur with unpredictable time delays
• Many data series much shorter than the 100
  observations recommended for analysis
• Difficult to locate or retrieve data
• Time intervals between each data point in archive
  may be longer than needed
• Missing data
• Undocumented definitional shifts

15
Applied to the Co-Proxamol Data

• 28 quarters in the pre-intervention period and 12
  in post-intervention
• Examined a number of common analgesics
• Prescriptions
• Deaths
• Examined overall suicides
• Some evidence of autocorrelation in the data,
  hence Cochrane-Orcutt autoregression used (Durbin
  Watson statistic of final models close to 2)

16
Prescriptions for analgesics dispensed in
England and Wales, 1998-2007
excluding liquids, suppositories, granules,
powders and effervescent preparations
17
Mortality in England and Wales from analgesic
poisoning (suicide and open verdicts), 1998-2007,
for persons aged 10 years and over (substances
taken alone, /- alcohol)

18
Prescriptions
Pre-intervention Pre-intervention Pre-intervention Pre-intervention Post-intervention Post-intervention Post-intervention Post-intervention
Base level, ß0 (SE) p Base trend, ß1 (SE) p Change in level, ß2 (SE) p Change in trend, ß3 (SE) p
Co-proxamol 3050.1 (139.9) lt0.001 -45.9 (7.7) lt0.001 -554.8 (74.9) lt0.001 -46.8 (16.8) 0.01
Cocodamol 1349 (12.4) lt0.001 34.1 (0.8) lt0.001 300.5 (53.6) lt0.001 30.7 (6.4) lt0.001
Codeine 204.8 (4.2) 0.007 9.6 (0.2) lt0.001 20.5 (11.6) 0.089 3.5 (1.2) 0.007
Codrydamol 1055.2 (6.1) lt0.001 -1.1 (0.3) 0.004 148.7 (35.8) lt0.001 -4.2 (4.1) 0.316
Dihydrocodeine 686.4 (31.1) lt0.001 -1.5 (1.4) 0.291 -29.7 (2.4) lt0.001 -0.8 (2.4) 0.731
NSAIDs 4652.4 (47.2) lt0.001 28.4 (3) lt0.001 -622.9 (70.2) lt0.001 -66.2 (8.5) lt0.001
Paracetamol 1493.4 (56.1) lt0.001 42.1 (3.0) lt0.001 232 (66.6) 0.001 23 (8.2) 0.01
Tramadol 47.2 (51.4) 0.365 31.4 (2.7) lt0.001 -41.9 (6.8) lt0.001 16.3 (5.2) 0.004
19
Prescriptions
Pre-intervention Pre-intervention Pre-intervention Pre-intervention Post-intervention Post-intervention Post-intervention Post-intervention
Base level, ß0 (SE) p Base trend, ß1 (SE) p Change in level, ß2 (SE) p Change in trend, ß3 (SE) p
Co-proxamol 3050.1 (139.9) lt0.001 -45.9 (7.7) lt0.001 -554.8 (74.9) lt0.001 -46.8 (16.8) 0.01
Cocodamol 1349 (12.4) lt0.001 34.1 (0.8) lt0.001 300.5 (53.6) lt0.001 30.7 (6.4) lt0.001
Codeine 204.8 (4.2) 0.007 9.6 (0.2) lt0.001 20.5 (11.6) 0.089 3.5 (1.2) 0.007
Codrydamol 1055.2 (6.1) lt0.001 -1.1 (0.3) 0.004 148.7 (35.8) lt0.001 -4.2 (4.1) 0.316
Dihydrocodeine 686.4 (31.1) lt0.001 -1.5 (1.4) 0.291 -29.7 (2.4) lt0.001 -0.8 (2.4) 0.731
NSAIDs 4652.4 (47.2) lt0.001 28.4 (3) lt0.001 -622.9 (70.2) lt0.001 -66.2 (8.5) lt0.001
Paracetamol 1493.4 (56.1) lt0.001 42.1 (3.0) lt0.001 232 (66.6) 0.001 23 (8.2) 0.01
Tramadol 47.2 (51.4) 0.365 31.4 (2.7) lt0.001 -41.9 (6.8) lt0.001 16.3 (5.2) 0.004
20
Prescriptions
Pre-intervention Pre-intervention Pre-intervention Pre-intervention Post-intervention Post-intervention Post-intervention Post-intervention
Base level, ß0 (SE) p Base trend, ß1 (SE) p Change in level, ß2 (SE) p Change in trend, ß3 (SE) p
Co-proxamol 3050.1 (139.9) lt0.001 -45.9 (7.7) lt0.001 -554.8 (74.9) lt0.001 -46.8 (16.8) 0.01
Cocodamol 1349 (12.4) lt0.001 34.1 (0.8) lt0.001 300.5 (53.6) lt0.001 30.7 (6.4) lt0.001
Codeine 204.8 (4.2) 0.007 9.6 (0.2) lt0.001 20.5 (11.6) 0.089 3.5 (1.2) 0.007
Codrydamol 1055.2 (6.1) lt0.001 -1.1 (0.3) 0.004 148.7 (35.8) lt0.001 -4.2 (4.1) 0.316
Dihydrocodeine 686.4 (31.1) lt0.001 -1.5 (1.4) 0.291 -29.7 (2.4) lt0.001 -0.8 (2.4) 0.731
NSAIDs 4652.4 (47.2) lt0.001 28.4 (3) lt0.001 -622.9 (70.2) lt0.001 -66.2 (8.5) lt0.001
Paracetamol 1493.4 (56.1) lt0.001 42.1 (3.0) lt0.001 232 (66.6) 0.001 23 (8.2) 0.01
Tramadol 47.2 (51.4) 0.365 31.4 (2.7) lt0.001 -41.9 (6.8) lt0.001 16.3 (5.2) 0.004
21
Suicides
Pre-intervention Pre-intervention Pre-intervention Pre-intervention Post-intervention Post-intervention Post-intervention Post-intervention
Base level, ß0 (SE) p Base trend, ß1 (SE) p Change in level, ß2 (SE) p Change in trend, ß3 (SE) p
Co-proxamol 81.0 (4.5) lt0.001 -1.194 (0.3) lt0.001 -28.3 (4.9) lt0.001 0.6 (0.6) 0.355
Other analgesics 51.3 (2.8) lt0.001 -0.3 (0.2) 0.095 6.4 (6.0) 0.297 -0.3 (0.6) 0.724
All drugs except co-proxamol and other analgesics 221.2 (7.3) lt0.001 -0.5 (0.5) 0.299 21.6 (12.8) 0.100 -5.4 (1.4) lt0.001
All drugs 353.7 (10.2) lt0.001 -2.0 (0.7) 0.008 0.004 (18.1) 1.000 -4.9 (1.7) 0.007
All causes 1319.0 (22.5) lt0.001 -4.8 (1.4) 0.002 12.8 (34.8) 0.716 -5.4 (4.1) 0.192
22
Estimating Absolute Effect

• The model may be used to estimate the absolute
  effect of the intervention. This is the
  difference between the estimated outcome at a
  certain time after the intervention and the
  outcome at that time if the intervention not
  taken place.
• For example, to estimate the effect of the
  intervention at the midpoint of the
  post-intervention period (when time 34.5 and
  time_after_intervention 6.5), we have
• Y34.5 ß0 ß1 x 34.5 without
  intervention
• Y34.5 ß0 ß1 x 34.5 ß2 ß3 x 6.5 with
  intervention
• Thus, the absolute effect of the intervention is
• ß2 ß3 x 6.5
• Standard errors calculated using method of Zhang
  et al
• s22 6.52 x s32 2 x 6.5 x s23
• Non-significant terms included due to correlation
  between slope and level terms

23
Results - Prescriptions
Estimates of absolute effect during 2005 to 2007 Estimates of absolute effect during 2005 to 2007 Estimates of absolute effect during 2005 to 2007 Estimates of absolute effect during 2005 to 2007
Mean quarterly estimated number pre announcement Mean quarterly number post announcement Mean quarterly change (95 CI)
Prescriptions (x1000)
Co-proxamol 1465.1 605.7 -859 (-1065 to -653)
Cocodamol 2524.7 3024.6 500 (459 to 540)
Codeine 534.6 578 43 (31 to 55)
Codrydamol 1018.2 1140.0 122 (99 to 145)
Dihydrocodeine 634.6 600.0 -35 (-68 to -2)
NSAIDs 5633.8 4581.0 -1053 (-1186 to -920)
Paracetamol 2947.0 3330.0 382 (268 to 497)
Tramadol 1130.1 1193.9 64 (-5 to 133)
24
Results - Deaths
Estimates of absolute effect during 2005 to 2007 Estimates of absolute effect during 2005 to 2007 Estimates of absolute effect during 2005 to 2007 Estimates of absolute effect during 2005 to 2007
Mean quarterly estimated number pre announcement Mean quarterly number post announcement Mean quarterly change (95 CI)
Suicides, Open
Co-proxamol 39 15 -24 (-37 to -12)
Other analgesics 39 44 5 (-5 to 15)
All drugs except co-proxamol and other analgesics 204 191 -13 (-34 to 8)
All drugs 283 252 -31 (-66 to 3)
All causes 1152 1130 -22 (-89 to 45)
25
Co-Proxamol Prescriptions

• Prescription data for England and Wales showed a
  steep reduction in prescribing of co-proxamol in
  the first two quarters of 2005, with further
  reductions thereafter.
• Regression analyses indicated a significant
  decrease in both level and slope in prescribing
  of co-proxamol - the number of prescriptions
  decreased by an average of 859 (95 confidence
  interval (CI) 653 to 1065) thousand per quarter
  in the post-intervention period.
• This equates to an overall decrease of
  approximately 59 in the three year
  post-intervention period, 2005 to 2007.

26
Other Analgesic Prescriptions

• There were also significant decreases in
  prescribing of NSAIDS of an average of 1053 (95
  CI 920 to 1186) thousand per quarter, equating
  to an approximate 19 decrease overall for 2005
  to 2007 and for dihydrocodeine of an average of
  35 (95 CI 2 to 68) thousand per quarter, or
  approximately 6 overall for 2005 to 2007.
• Prescriptions for the other analgesics increased
  significantly in the post-intervention period,
  apart from tramadol. Based on mean quarterly
  estimates this equated to percentage increases
  over the 2005 to 2007 period of approximately 20
  for cocodamol, 13 for paracetamol, 12 for
  codydramol, and 8 for codeine.

27
Co-Proxamol Deaths

• Marked reduction in suicide and open verdicts
  involving co-proxamol in the first quarter of
  2005, which persisted until the end of 2007.
• Prior to 2005 deaths due to co-proxamol alone
  were 19.5 (95 CI 16.9 to 22.2) of all drug
  poisoning suicides, whereas between 2005 and 2007
  they constituted just 6.4 (95 CI 5.2 to 7.5).
• Regression analyses indicated a significant
  decrease in both level and slope for deaths
  involving co-proxamol which received a suicide or
  open verdict - decreased by on average 24 (95 CI
  12 to 37) per quarter in the post-intervention
  period.
• This equates to an estimated overall decrease of
  295 (95 CI 251 to 338) deaths, approximately
  62, in the three year post-intervention period
  2005 to 2007.
• When accidental poisoning deaths involving
  co-proxamol were included, there was a mean
  quarterly decrease of 29 (95 CI 17 to 42)
  deaths, equating to an overall decrease of 349
  (95 CI 306 to 392) deaths, approximately 61,
  in the three year post-intervention period 2005
  to 2007.

28
Other Deaths

• There were no statistically significant changes
  in level or slope in the post-intervention period
  for deaths involving other analgesics (cocodamol,
  codeine, codydramol, dihydrocodeine, NSAIDs,
  paracetamol and tramadol) which received a
  suicide or open verdict (both including and
  excluding accidental deaths).
• There was a substantial though not statistically
  significant reduction during the
  post-intervention period in deaths (suicide and
  open verdicts) involving all drugs (including
  co-proxamol and other analgesics), with the mean
  quarterly change between 2005 and 2007 being -31
  (95 CI -66 to 3) deaths.
• The overall suicide rate (including open
  verdicts) during this period also decreased,
  though to a lesser extent, and the mean quarterly
  change of -22 (95 CI -89 to 45) deaths was not
  statistically significant.

29
Substitution of Method of Suicide

• Possible substitution of method must be
  considered in estimating the effect of changing
  availability of a specific method of suicide.
• Research evidence on failed suicide attempts
  suggests that its unusual for completely
  different method to be used.
• Withdrawal of co-proxamol was associated with
  changes in prescribing of other analgesics.
• Significant increases in prescribing of
  co-codamol, paracetamol, and codydramol occurred
  during 2005-2007.
• Analyses of suicide and open verdict deaths
  involving other analgesics combined indicated
  little evidence of substitution.

30
NSAIDs

• An abrupt reduction in prescribing of NSAIDs
  occurred shortly before the announcement of the
  withdrawal of co-proxamol due to concerns about
  Cox 2 inhibitors.
• However, NSAIDs are rarely a direct acute cause
  of death, especially by suicide.

31
Interpretation

• Following the announcement of the withdrawal of
  co-proxamol in January 2005 there was an
  immediate large reduction in prescriptions. This
  was associated with a 62 reduction in suicide
  deaths (including open verdicts), or an estimated
  295 fewer deaths.
• Inclusion of accidental deaths, some of which
  were likely to have been suicides increased the
  estimated reduction in number of deaths to
  approximately 349 over 3 years.
• Overall suicide and open verdict deaths decreased
  in England and Wales during 2005 to 2007. Thus
  underlying downward trends in suicide cannot
  explain the full extent of the decrease in
  co-proxamol related deaths following the MHRA
  announcement to withdraw co-proxamol.

32
Limitations

• Interrupted time series autoregression controls
  for baseline level and trend when estimating
  expected changes in the number of prescriptions
  (or deaths) due to the intervention.
• The estimates of the overall effect on
  prescriptions and mortality involved
  extrapolation, which is inevitably associated
  with uncertainty.
• The regression method assumes linear trends over
  time, and the co-proxamol prescribing data, in
  particular, had a poor fit, resulting in large
  standard errors in the post-intervention period.
• Estimates of the standard errors for absolute
  mean quarterly changes in number of prescriptions
  or deaths were determined exactly, including the
  covariance of level and slope terms.
• Estimates of percentage changes over the three
  year post-estimation period are point estimates
  and were not determined with standard error
  calculations.

33
Threats to Validity

• Co-proxamol prescription only drug, often given
  to elderly patients suffering arthritis
• Most poisoning suicides are in younger people, so
  co-proxamol use considered to be opportunistic
• Recording of suicide may be coroner dependent
• Open verdict may be given when there is lack of
  suicide note
• Prescriptions numbers were reducing as GPs tried
  to move patients to alternative analgesics prior
  to withdrawal so some diffusion of intervention

34
Design Considerations

• Post-hoc analysis
• Very difficult to identify suitable
  non-equivalent no treatment control group
• Inclusion of overall suicide rates goes some way
  towards examination of validity threat

35
Problems

• Diffusion of intervention but in this case prior
  to identified intervention point as demonstrated
  by decrease in prescriptions prior to
  announcement
• Data series rather shorter than the ideal of 100
  observations, but minimum of 12 before and after
  intervention point considered not unreasonable
• Quarterly figures give reasonable time intervals
• Theres no concrete evidence of a definitional
  shift in suicides in this time interval although
  this cannot be ruled out
• Open verdicts were included as a sensitivity
  check as one way of addressing potential missing
  data
• Almost impossible to consider impact of missing
  data on agent used in self-poisoning

36
Guidelines on Use

• Ramsay et al, 2003
• Quality criteria
• Intervention occurred independently of other
  changes over time
• Intervention was unlikely to affect data
  collection
• The primary outcome was assessed blindly or was
  measured objectively
• The primary outcome was reliable or was measured
  objectively
• The composition of the data set at each time
  point covered at least 80 of the total number of
  participants in the study
• The shape of the intervention effect was
  prespecified
• A rationale for the number and spacing of data
  points was described
• The study was analyzed appropriately using time
  series techniques

37
Findings of the Systematic Reviews

• Mass media review of 20 studies, Guideline
  dissemination and implementation review of 38
  studies
• Most studies had short time series
• Standard errors increased
• Reduced power
• Type I error increased
• Failure to detect autocorrelation or secular
  trends
• Over 65 analysed inappropriately
• Of the 37 re-analysed, 8 had significant
  pre-intervention trends
• Most were underpowered
• Rule of thumb with 10 pre- and 10
  post-intervention time points the study would
  have at least 80 power to detect a change in
  level of five standard deviations of the pre-data
  if the autocorrelation gt0.4
• Long pre-intervention phase increases power to
  detect secular trends

38
References

• Shadish, Cook and Campbell, 2002, Experimental
  and quasi-experimental designs for generalised
  causal inference, Houghton Mifflin.
• Ramsay CR, Matowe L, Grilli R, Grimshaw JM,
  Thomas RE. Interrupted time series designs in
  health technology assessment Lessons from two
  systematic reviews of behavior change strategies.
  Int.J.Technol.Assess.Health Care 200319613-23
• Wagner AK, Soumerai SB, Zhang F, Ross-Degnan D.
  Segmented regression analysis of interrupted time
  series studies in medication use research.
  J.Clin.Pharm.Ther. 200227299-309
• Zhang, F, Wagner, A, Soumerai, S. B., and
  Ross-Degnan, D. Estimating confidence intervals
  around relative changes in outcomes in segmented
  regression analyses of time series data. 15th
  Annual NESUG (NorthEast SAS Users Group Inc)
  Conference Last update 2002. http//www.nesug.inf
  o/Proceedings/nesug02/st/st005.pdf. Accessed 22
  October 2008.

39
Examples of Use

• Matowe, L, Ramsay, C. R., Grimshaw, J. M.,
  Gilbert F. J., MacLeod, M.-J. and Needham, G.
  Effects of mailed dissemination of the Royal
  College of Radiologists Guidelines on general
  practitioner referrals for radiography a time
  series analysis. Clinical Radiology 2002, 57,
  575-578
• Neustrom, M. W. and Norton, W. M. The impact of
  drunk driving legislation in Louisiana. Journal
  of Safety Research, 1993, 24, 107-121
• Ansari, F, Gray, K, Nathwani, D, Phillips, G,
  Ogston, S, Ramsay, C and Davey, P. Outcomes of an
  intervention to improve hospital antibiotic
  prescribing interrupted time series with
  segmented regression analysis. Journal of
  Antimicrobial Chemotherapy, 2003, 52, 842-848
• Morgan, O. W., Griffiths, C and Majeed, A.
  Interrupted time-series analysis of regulations
  to reduce paracetamol (acetaminophen) poisoning.
  PLoS Medicine, 2007, 4, 0654-0659

40

• K. Hawton, H. Bergen, S. Simkin, A. Brock, C.
  Griffiths, E. Romeri, K. L. Smith, N. Kapur, D.
  Gunnell (2009). Effect of withdrawal of
  co-proxamol on prescribing and deaths from drug
  poisoning in England and Wales time series
  analysis. BMJ, 338b2270