How to Read and Do a Cochrane

1
The Cochrane Collaboration Forum

• How to Read and Do a Cochrane
• Systematic Review
• Leura V International Breast Cancer Conference
  2004
• Presented by The Cochrane Breast Cancer Group
• Henderson I. Craig, Wilcken N, Ghersi D, Davis N,
  Parker S, Carrick S

2
Introduction

• I. Craig Henderson

3
What is The Cochrane Collaboration?

• An international organisation that aims to help
  people make well-informed decisions about
  healthcare by preparing, maintaining and
  promoting the accessibility of systematic reviews
  of the effects of health care interventions.

4
Overriding Principles

• Collaboration
• Building on the enthusiasm of individuals
• Avoiding duplication
• Minimising bias
• Keeping up to date
• Striving for relevance
• Promoting access
• Ensuring quality

5
Goals

• To ensure high quality, up-to-date systematic
  reviews
• To promote access to Cochrane reviews
• To develop an efficient, transparent
  organisational structure and management system
  for the Cochrane Collaboration
• To achieve sustainability of the Cochrane
  Collaboration

6
Structure of The Cochrane Collaboration
Collaborative Review Groups
Centres
Steering Group
Methods Groups
Fields
The Consumer Network
7
Cochrane Centres
8
Cochrane Centres

• Help organise and register review groups
• Facilitate collaboration among reviewers
• Provide training and consultation
• Establish liaisons
• Promote the Cochrane Collaboration
• Provide unique contribution

9
Collaborative Review Groups

• Currently 50 registered CRGs
• Focus
• Treatment of Disease or Health Problems
• Emphasis on Clinical Outcomes
• Emphasis on Rigorous Data

10
Some examples of CRGs

• Acute Respiratory Infections
• Breast cancer
• Dementia Cognitive Impairment
• Diabetes
• Effective Practice and Organisation of Care
• Eyes and vision
• Muskuloskeletal
• Pain and palliative care

• Neonatal
• Oral Health
• Infectious Diseases
• Injury
• Pregnancy Childbirth
• Renal
• Schizophrenia
• Stroke
• Tobacco Addiction
• Wounds

11
Review Group Products

• Systematic Reviews published on The Cochrane
  Library
• Specialised Registers

12
The Cochrane Breast Cancer Group

• The Cochrane Breast Cancer Group is committed to
  the development of evidence relating to the
  prevention, early detection and treatment of
  breast cancer at any stage
• 1 of only 5 CRGs in Australasia

13
Structure of the CBCG

• Editors
• Nicholas Wilcken
• Co-ordinating Editor, Australia
• Christine Brunswick, USA
• Mike Clarke, UK
• Patricia Ganz, USA
• Davina Ghersi, Australia
• I. Craig Henderson, USA
• Alessandro Liberati, Italy
• Sue Lockwood, Australia
• Alan Rodger, UK
• John Simes, Australia

• Editorial Base
• Davina Ghersi, Review Group Coordinator
• Sharon Parker, Assistant Review Group Coordinator
• Nicole Davis, Trials Search Coordinator

14
Achievements of the CBCG

• 22 completed reviews
• 24 protocols for reviews
• 332 registered members covering a wide range of
  disciplines including consumers
• Twice yearly newsletter
• Strong links with EBCTCG, ANZ BCTG, National
  Breast Cancer Centre, NSW Breast Cancer
  Institute, National breast Cancer Foundation, NSW
  Cancer Council

15
Achievements

• Some examples of guidelines/influential reviews/
• Australia
• NHMRC Guidelines for the management of ABC
• Canada
• Taxanes for metastatic breast cancer
• Central European Cooperative Oncology Group
  Consensus Guidelines
• Duration
• Chemo vs endo
• UK NHS/NICE/DOH
• Funding ongoing / updates of reviews

16
Why do a systematic review of any type, let alone
a Cochrane review?

• Nicholas Wilcken

17
Alternative strategies
How should we summarise the results of clinical
trials? Do we always need to rely on
Oxford overview-type reviews? What are the
pluses and minuses of other approaches?
18
Working out what works
Case reports, historical series, phase 2
trials, RCTs and systematic reviews/meta-analyses
are all aimed at finding the best treatments for
our patients
and are all aimed at improving cvs, selling more
drugs, etc
19
Working out what works
A non-randomised trial (Lind 47)
"Their cases were as similar as I could have
them.... They lay together in one place.....
and had one diet common to all....."
20
Working out what works

• Sailors with scurvy in 1747
• N 12
• - Cyder 2 - Sea Water
• 2 - Elixir Vitriol 2 - Oranges and
  Lemons
• 2 - Vinegar 2 - Nutmeg

21
Working out what works
"....the most sudden and visible good effects
were perceived from the oranges and lemons"
22
The place of systematic reviews
Most treatments tend to have modest
effects individual trials not always large enough
Ignoring such individual trials may introduce bias

• Hence the need for some kind of summary
• narrative reviews
• systematic reviews of literature
• individual patient data reviews

23
Types of reviews

• narrative reviews
• expert opinion

• systematic reviews of literature
• implies a specific search strategy
• should prospectively define the
• question
• pooling of data may be possible

• individual patient data reviews
• least bias
• greatest power

24
Types of systematic reviews

• Individual patient data reviews (the Oxford
  overview model)
• especially important for older (smaller)
• trials
• allow a degree of subset analysis
• take a long time, cost a lot of money

• Systematic reviews using summary data
• relatively cheap and quick
• immediate clinical relevance
• less statistically robust
• pooling of data not always appropriate

25
Un-Oxford overview without pooling of data
NBCC of Australia Lancet Onc June 2004
26
Methods

• prospectively written and peer-reviewed
• protocol
• systematic search strategy (specialised
• Cochrane register of RCTs and conference
  abstracts)
• duplicated, independent decisions on
• eligibility/duplicated data extraction

27
neoadjuvant
Patient numbers in reported trials
adjuvant
28
Tabulated results
29
Results

• 10 reported trials
• neoadjuvant 5 trials and 2,948 women
• 4/5 better CR (NS)
• adjuvant 5 trials and 9,211 women
• 5/5 better DFS (NS in 2)
• not correlated with ER status
• 2/5 better OS
• 15 unreported trials 18,000 women

30
Neo/adjuvant taxane review

• early/immature data but of immediate
• clinical relevance though statistically soft
• identifies pending data
• may influence clinical practice policy/funding
  decisions (eg ER status)

31
Un-Oxford (Cochrane) overview with pooling of data
32
Taxane v not - overall survival
HR 0.91 (0.84-0.98) for OS
33
Taxanes in MBC review

• possible modest benefit in overall survival
• all q3wk regimens so far
• subset analyses done, but probably not
• very reliable

34
What is a PMA

• A prospective meta-analysis (PMA) is a
  meta-analysis of RCTs identified, evaluated, and
  determined to be eligible for the meta-analysis
  before the results of any of those trials become
  known.

35
Cochrane reviews of the literature v IPD reviews

• Cons
• fewer zeros in p values
• pooling of data not always appropriate
• publication and other biases
• Pros
• relatively cheap and quick
• locate and organise the data
• fulfill a need for immediate clinical relevance
• bigger trials these days anyway
• hypothesis-generating for IPD reviews

36
First they do an on-line search
37
Practical Exercise

• The NHMRC have asked you to conduct a review to
  inform their clinical practice guidelines on the
  use of chemotherapy in metastatic breast cancer

38
Constructing an answerable question

• Population
• Intervention
• Comparator
• Outcomes

39

• Metastatic breast cancer

40
How to do a review
41
The review process
42
Whats in a protocol

• Background
• Objectives
• Criteria for considering studies for this review
• Types of studies
• Types of participants
• Types of interventions
• Types of outcome measures

43

• Search strategy for identification of studies
• Methods of the review
• Eg quality control, analysis

44
Perform review
Identify studies, apply eligibility criteria,
extract and analyse data
45
Example Objective

• To compare taxane containing chemotherapy
  regimens with regimens not containing a taxane in
  the management of women with metastatic breast
  cancer. This includes
• Regimen A plus taxane vs Regimen A
• Regimen A plus taxane vs Regimen B
• Single agent taxane vs Regimen C

46
Example Criteria

• Types of studies
• Properly randomised controlled clinical trials.

47
Example Criteria

• Types of participants
• Women with advanced breast cancer
• advanced breast cancer is defined as
  metastatic disease
• women with locoregional disease only will be
  excluded
• both newly diagnosed and recurrent cases
  will be included

48

• Types of interventions
• Intervention Group Any chemotherapy regimen
  containing a taxane.
• Comparator Any chemotherapy regimen not
  containing a taxane.

49

• Types of outcome measures
• Overall survival
• Progression-free survival (time to disease
  progression and/or death)
• Response (WHO, RECIST or individual protocol
  criteria)
• toxicity (WHO criteria or individual protocol
  based defintion)
• time to treatment failure (if reported)
• quality of life measures (trial specific
  instruments)

50
Search Strategy

• Search the specialised register of the Cochrane
  Breast Cancer Group.
• Using codes "advanced" and "chemotherapy"
• Search reference lists of other, related
  literature reviews
• A copy of the full article for each reference
  reporting a potentially eligible trial will be
  obtained.

51
Methods

• A. Assessing trials for eligibility
• The Selection Criteria will be applied
  to each trial.
• Trial publications will be used to assess the
  trial's eligibility with the results section (and
  any other area where results may appear) masked.
• If a trial has not been published, information
  will be obtained from the trial protocol or next
  best available resource.
• Where necessary, and possible, additional
  information will be sought from the principal
  investigator of the trial concerned.

52

• B. Quality Control and Peer Review
• Two reviewers will independently assess each
  potentially eligible trial for
• Inclusion in the review (according to the
• eligibility criteria)
• Quality
• A third reviewer will resolve any discrepancies
  regarding eligibility or quality.

53

• C. Analysis
• The most complete dataset feasible will be
  assembled.
• Results of eligible studies will be statistically
  synthesised (meta-analysis) if appropriate and
  possible.
• All analyses will be by intention to treat.
• Heterogeneity between trial results will be
  tested where appropriate.

54

• Sub-group analyses
• will depend on whether or not the required
  information is recorded in the trial
  publications. Possible sub-groups include
• Menopausal status (pre / peri / post)
• Hormone receptor status
• Newly diagnosed v recurrent disease
• Sensitivity analyses (e.g. including trials of
  borderline quality) and subgroup analyses will be
  performed if appropriate and possible.

55

• Assessing the Methodological Quality of the
  Included Studies
• Each study will be reviewed according to its
  design and by how the study was conducted to
  assess any bias. The checklist for quality of
  randomised controlled trials will include
• concealment of the allocation sequence
• generation of the allocation sequence
• comparability between groups at the baseline
• inclusion of all randomised participants in the
  analysis

56
Time to event outcomes
57
Why hazard ratios (HR)?

• Time to event outcomes (TTE) measure not only
  whether but when event happens
• e.g. measuring survival in cancer
• TTE most appropriately measured by HR
• odds ratios (ORs) measure number of events at
  single time point and apply only to that point
• HRs summarise the failure pattern over the entire
  survival experience

58
HRs in systematic review

• Collect individual patient data
• calculate log rank O-E and V directly from raw
  data
• Use published reports
• calculate or estimate HRs from summary statistics
  presented in published reports or tables
• Parmar et al. 1998. Statist Med 172815-34

59
(No Transcript)
60
What help do I get?
61
Doing a Cochrane review in breast cancer

• Contact Cochrane Breast Cancer Group
• Will advise and support
• Methodologic, clinical and consumer expertise
• Software
• Review Manager (RevMan)

62
Check out the Cochrane Library

• Since October 2002, all Australians have had free
  access to the Cochrane Library
• Go to http//www.update-software.com/clibng/clibl
  ogon.htm
• Log on anonymously.

63
(No Transcript)
64
(No Transcript)
65
(No Transcript)
66
Specialised Register

• Nicole Davis

67
The Breast Cancer Specialised Register

• What is it?
• A continuously growing database of randomised
  controlled trials (RCTs) and controlled clinical
  trials (CCTs) relevant to breast cancer.
• 4998 RCTs and CCTs on the register

68
Where do the references come from?

• A detailed search strategy has been developed to
  assist identification of published and
  unpublished trials relevant to breast cancer to
  be incorporated into the register.
• The search is conducted from a variety of sources
  including
• Medline
• Embase
• Physician data query
• The Cochrane Library
• Various on line trial registers
• Hand searching paper copies of journals/
  conference proceedings

69
Coding the references on the register

• All references are coded according to
• - The stage of breast cancer (e.g. early,
  advanced)
• - Type of intervention (e.g. chemotherapy,
  endotherapy)
• - Study type (RCT or CCT)

70
How can the register help you?

• The RCTs and CCTs are kept and made available
  for use in systematic reviews.
• Specifically designed searches can be performed
  on the register by the trial search coordinator,
  that will retrieve references relevant to your
  systematic review question.
• This resource is available to all reviewers by
  contacting the trial search coordinator.

71
Additional Cochrane Resources

• Cochrane reviewers handbook http//www.cochrane.or
  g/resources/handbook/index.htm
• Open learning materials
• http//www.cochrane-net.org/openlearning/
• RevMan homepage
• http//www.cc-ims.net/RevMan
• Cochrane Style Resource
• http//www.liv.ac.uk/lstm/ehcap/CSR/CSG.html
• Estimating Hazard Ratios (spreadsheet)
  Cochrane_at_ctc.usyd.edu.au

72
Non Cochrane resources

• RDDirect www.rddirect.org.uk
• bmjlearning www.bmjlearning.com
• Centre for health evidence www.cche.net
• CONSORT www.consort-statement.org
• QUOROM statement
• Lancet. 1999 Nov 27354(9193)1896-900
• MOOSE www.consort-statement.org/MOOSE.pdf