Systematic reviews of animal studies

1
Systematic reviews of animal studies

• Malcolm Macleod

2
Why do systematic reviews of animal studies?

• To summarise existing data
• To help design clinical trials
• To understand where evidence is lacking
• To understand the limitations of animal models

3
Basic requirements

• Understanding the model or knowing someone who
  does
• Clear a priori hypotheses
• A clear search strategy with inclusion and
  exclusion criteria
• Information and data management

4
(No Transcript)
5
The importance of hypotheses

• Observational research
• Susceptible to identification of statistically
  significant but biologically meaningless spurious
  associations
• If there isnt a question for which you wish to
  know the answer, why are you doing it?

6
Examples

• Hypothermia improves outcome in animal models of
  stroke
• The efficacy of hypothermia in animal models of
  stroke depends on the degree of cooling
• Evidence for the efficacy of NXY-059 is
  confounded by poor study quality
• The evidence for efficacy of stroke dugs in
  animals is confounded by publication bias

7
Search strategy

• What are you looking for
• Controlled studies testing the effect of
  hypothermia in an animal model of focal cerebral
  ischaemia brought about by occlusion of a
  cerebral artery, where outcome was measured as
  infarct size or neurobehavioural score.
• Exclusion criteria
• hypothermia was accomplished with use of a
  pharmacological agent that may also have an
  intrinsic neuroprotective property
• cooling was used to counteract (spontaneous)
  hyperthermia after MCAO
• brain cooling lasted lt10 min, for example to
  counteract heating in models of photochemically
  induced cerebral infarction
• data were presented in a way not suitable for use
  in a meta-analysis (e.g. no information on group
  size, mean or variance) or if
• mortality was the only outcome measure.

8
Example search strategy

• Studies of hypothermia in animal models of acute
  ischaemic stroke were identified from
• PubMed, EMBASE and BIOSIS up till December 31,
  200gt with the search strategy ltcerebralgt OR
  ltbraingt OR ltneurongt OR ltneuronalgt OR ltnervousgt
  AND ltischemiagt OR ltischaemiagt OR ltstrokegt AND
  lthypothermiagt OR lttemperaturegt (limit animals)
• hand searching of abstracts of scientific
  meetings of the International Society of Cerebral
  Blood Flow and Metabolism, the International
  Stroke Conference Joint (International)
  Conference on Stroke and Cerebral Circulation
  before 2000 and the European Stroke Conference
  during the same time period
• reference lists of identified publications and
• requests to authors of identified publications

9
Source selection

• Download search results to reference management
  system (eg RefMan)
• 2 investigators independently select sources
  against inclusion/ exclusion criteria
• May have to retrieve full text
• Discrepancies resolved by negotiation or in
  discussion with third investigator

10
Data extraction

• Publication meta-data

11
(No Transcript)
12
(No Transcript)
13
Data extraction

• Publication meta-data
• Outcome data

14
(No Transcript)
15
Data cleaning
16
Data analysis

• Standardised mean difference analysis
• Weighted mean difference analysis
• Normalised mean difference analysis
• Fixed Effects
• Random Effects
• Meta-regression

17
(No Transcript)
18
(No Transcript)
19
Testing significance

• Partitioning of heterogeneity
• Observed heterogeneity within group
  heterogeneity between group heterogeneity
• Test against chi squared distribution with n-1
  degrees of freedom
• Observational, so set high statistical bar

20
Data analysis
21
Demonstration
22
So, what can it do ?

• Describe a literature .

23
Estimate efficacy

• Hypothermia
• 101 papers
• 277 experiments
• 3353 animals

24
Describe efficacy in subgroups
25
Show potential sources of bias
NXY 059 9 publications 29 experiments 408
animals Improved outcome by 44 (35-53)
26
Illustrate publication bias
991 publications
27
Provide evidence to change practice

• Animals
• Sample size calculation
• Inclusion and exclusion criteria
• Randomization
• Allocation concealment
• Reporting of animals excluded from analysis
• Blinded assessment of outcome
• Reporting potential conflicts of interest and
  study funding

28
Further resources

• http//www.camarades.info
• http//www.camarades.info/index_files/papers.htm
• malcolm.macleod_at_ed.ac.uk
• The CAMARADES podcast
• http//www.camarades.info/index_files/podcasts/pod
  cast.xml