Randomised Controlled Trials in the Social Sciences: Challenges and Prospects

1
Randomised Controlled Trials in the Social
SciencesChallenges and Prospects
Assessing Nonpharmacological treatments Challenge
s and reporting guidelines for Randomised
Controlled Trials

• I . Boutron
• Dept of Epidemiology, Biostatistics and Clinical
  Research

2
Overview

• Specific methodological issues when assessing
  Nonpharmacological treatments
• Quality tools
• Reporting guidelines

3
Overview

• Specific methodological issues when assessing
  Nonpharmacological treatments
• Quality tools
• Reporting guidelines

4
Different types of NPT
Non-pharmacological treatments (NPT) take in many
different treatments
Participative interventions
Therapist-dependent interventions
Non implantable devices

• orthopedic braces or orthosis
• laser treatment

- rehabilitation - education

• surgery
• technical operation (arthroscopy,)

5
Assessing Nonpharmacological treatments

• Randomized controlled trials (RCT)
• Gold standard of therapeutic evaluation
• The design, conduct and reporting of RCTs aim at
  providing valid results
• Standard of the methodology of RCTs has been
  developed in the context of pharmacological
  treatments (PT)
• Specific methodological issues when assessing
  NPTs
• Blinding
• Placebo
• Complexity of treatments
• Care providers influence

6
Are NPT trials specific?Quality
Quality of trials assessing PT and NPT of hip and
knee osteoarthritis published in high impact
factor journals (1992-2002)

• PT (n60) versus NPT (n50)
• B. Comparison
• Oral drug, intra-venous, intra-muscular (n 46)
• Intra-articular injection (n 14)
• Surgery, arthroscopy, joint lavage (n 23)
• Rehabilitation, acupuncture, education(n 27)

Boutron, Tubach, Giraudeau, Ravaud Jama, 2003
7
Are NPT trials specific?The feasibility of
blinding
Boutron I, Tubach F, Giraudeau B, Ravaud P,
JCE, 2003
8
Are NPT trials specific?The success of blinding
9
Are NPT trials specific? Placebo
Moseley et al., NEJM 2002 Edworthy et al. J
Rheumatol 1999
10
Are NPT trials specific?Placebo

Paterson, C. et al. BMJ 2005
11
Are NPT trials specific? Placebo

• Externally clearly different placebo
• Physiotherapy for knee OA
• Placebo treatment consisted of sham
  ultrasonography
• Assessment of lifestyle advice
• Placebo treatment consisted of a syrup placebo

Bennell, Ann Rheum Dis, 2005 Spigt, JCE, 2005
12
Are NPT trials specific? The methods of blinding
Full blinding
Study cohort 145 non-pharmacological trials
(2004)

• Full blinding attempted
• externally identical placebo/control
• Unblinded care providers not involved in the
  subsequent patient care
• Example
• Transplantation of embryonic dopamine neurons
• four twist-drill holes made through the frontal
  bone after local anesthesia
• the dura mater was not penetrated

Boutron, Ravaud, Submitted Freed, NEJM, 2001
13
Are Non pharmacological trials specific? The
methods of blinding Partial blinding

• Partial blinding attempted (blind trial
  hypothesis)
• Externally clearly different placebo / control
• Manipulation of information
• Patients not informed of the existence of a
  placebo
• Patients not aware of the nature of the placebo
• Confidence in treatments sometimes tested

14
Are Non pharmacological trials specific? The
methods of blinding Partial blinding

• Zelen design or modified zelen design
• Usual care compared to a complex, physical
  therapy-based intervention for patello-femoral
  joint osteoarthritis of the knee
• 1) Researchers invited patients to participate in
  a cohort.
• 2) Randomization
• 3) Patients randomized to the intervention arm
  informed that they would receive the experimental
  treatment and signed a second consent form

Quilty, J Rheumatol. 2003
15
Are Non pharmacological trials specific? The
methods of blinding Outcome assessors

• Blinding of outcome assessors
• Centralized assessment (video, audiotape or
  photography )
• Outcome assessors blinded of study hypothesis
• To assess the analgesic effect of breast feeding
  in term neonates compared to mothers arms,
  pacifiers, placebo (i.e., sterile water) or
  glucose
• Videotape of the children during the painful
  procedure
• Two specially trained observers independently
  assessed the recordings using a specific scale.
• Observers were blinded to the purpose of the
  study and had been told that they were assessing
  agreement of their scores in different situations
• Patients reported outcomes
• Blinding is impossible if patients cannot be
  blinded

Carbajal R, Bmj. 2003
16
Are NPT trials specific? Complex intervention

• Several components
• Rehabilitation exercises, drugs, education etc
• Description of the intervention
• Quantitative data
• Number of sessions, timing of each session,
  duration of each session
• Qualitative data
• Content of each session, how it is delivered,
  supervision, content of information exchanged etc
• Standardization procedure
• Specific training
• Quality control procedures
• Potential gap between the intended intervention
  (as described in the protocol ) and the actual
  administered intervention
• Barrier for systematic reviews

17
Are NPT trials specific? Influence of care
providers

• Systematic review of the surgical and medical
  volume-outcome literature (Jan 1, 1980 - Dec 31,
  2000)
• 71 of hospital volume-outcomes studies positive
  (88/124)
• 70 of physician volume-outcomes studies positive
  (31/44)
• No studies showed the opposite relationship
• Relationship strongest for high risk/rare
  surgeries
• Pancreas/esophagus cancer, pediatric cardiac
  surgery
• NNT at high v. low volume provider 7 to 11
• Much more modest volume-outcome effect for common
  procedures (CABG, CEA, PTCA, breast/colon cancer)
• NNT 62 to 500

Halm et al, Ann Intern Med 2002
18
Are NPT trials specific?

• Specific issues in assessing NPT
• Blinding
• Placebo
• Complexity of treatments
• Healthcare providers influence
• Need of specific standards

19
Overview

• Specific methodological issues when assessing
  Nonpharmacological treatment
• Quality tools
• Reporting guidelines

20
Quality toolsWhy?

• Assessing the quality of reports of trials is
  particularly important
• clinicians critical appraisal of healthcare
  literature
• systematic reviews
• Accurate estimates of the treatment effect

21
Quality toolsWhy?

• Several quality tools
• Few quality tools were developped according to
  scientific standards
• Jadad scale
• Delphi list
• Validated quality tools were mainly developped in
  the context of PT
• Importance of blinding
• No item related to care providers and the
  complexity of the intervention

Jadad, Control Clin Trials, 1996 Verhagen,
JCE, 1998
22
Development of a specific checklistThe example
of CLEAR NPT

• Quality Internal validity
• Delphi consensus method
• Selection of items
• From existing checklist or scales
• From specific items
• From the Collaborative Review Groups of the
  Cochrane Collaboration recommandations
• Interviews of clinicians
• Experts
• Members of Collaborative Review Groups of the
  Cochrane Collaboration (n41)
• Clinicians involved in RCTs assessing NPTs (n58)
• Methodologists, epidemiologists (n55)
• 55 experts participated(36)

Boutron Ravaud, JCE, 2005
23
Initial checklist
38 items
Decision of the steering comittee
1st Round 54 experts/55
Selection of items
with a score of
8/9
for more than 40 of experts
Checklist
21 items
2nd Round 46 experts/55
Decision of the steering comittee
Selection of items
With a score of
9/9
for more than 40 of experts
Checklist
11 items
Decision of the steering comittee
3rd Round 49 experts/55
Sélection of all items
Addition of 2 items for randomisation
Addition of 1 item for the intention-to-treat
analyses
Final Checklist
10 items
5 sub items
24
CLEAR NPT

• Was the generation of allocation sequences
  adequate?
• Was the treatment allocation concealed?
• Were the main outcomes analyzed according to the
  intention-to-treat principle?
• Were details of the intervention administered to
  each group made available?
• Were care providers experience or skill in each
  arm appropriate?
• Was participant (ie, patients) adherence assessed
  quantitatively?
• Was the follow-up schedule the same in each group?

25
CLEAR NPT

• Were participants adequately blinded?
• Were care providers or persons caring for the
  participants adequately blinded?
• If care providers and or participants were not
  adequately blinded
• Were all other treatments and care (ie,
  co-interventions) the same in each randomized
  group?
• Were withdrawals and lost to follow-up the same
  in each randomized group?
• Were outcome assessors adequately blinded to
  assess the primary outcomes?
• If outcome assessors were not adequately blinded,
  
• Were specific methods used to avoid ascertainment
  bias (systematic differences in outcome
  assessment)

26
CLEAR NPT

• Adequate assessment of trials quality
• Improvement of critical appraisal of medical
  litterature
• Improvement of the quality of systematic reviews

27
Overview

• Specific methodological issues when assessing
  Nonpharmacological treatment
• Quality tools
• Reporting guidelines

28
Reporting guidelinesWhy?

• Critical appraisal of the quality of clinical
  trials is possible only if the design, conduct,
  and analysis of RCTs are thoroughly and
  accurately described in published articles
• Evidence of incomplete and inadequate reporting

29
Reporting guidelines
30
(No Transcript)
31
Reporting guidelines
32
Reporting guidelinesDissemination

• Publication
• 11 journals
• Over than 1000 citations
• Editors endorsements including the ICMJE
  (International Committee for Medical Journal
  Editors)
• instructions to authors
• submission check-list
• Website
• www.consort-statement.org

33
Reporting guidelinesDissemination

• Various extensions of the CONSORT Statement
  since the original version (parallel arm,
  superiority, efficacy)
• Cluster RCT
• Harm
• Herbal therapy
• Non inferiority trials
• Several other extensions are in progress
• 2x2 factorial design

34
Reporting guidelines for NPT trials Why?

• 119 RCTs published in 1998 and 1999 in three
  major cardiothoracic journals (Annals of Thoracic
  Surgery, European Journal of Cardio-thoracic
  Surgery, The Journal of Thoracic and
  Cardiovascular Surgery)
• 70 of RCTs fulfilled less than half of the
  CONSORT criteria

Anyannu, Eur J of Cardio-thoracic Surgery, 2004
35
Reporting guidelines for NPT trials Why?

• Study of trials assessing PT and NPT of hip and
  knee osteoarthritis (1992-2002)
• When blinding was judge possible, blinding was
  less often
• reported in NPT trials

Boutron Ravaud, Jama, 2003
36
Reporting guidelines for NPT trials Why?

• Study of surgical papers (n 158) published in
  2004
• Reporting of details of the intended intervention
• Surgical procedure 87
• Pre-operative care 15
• Anesthesia 35
• Post-operative care 49
• Surgeons
• Selection criteria for surgeon 40
• Number of interventions performed by
  surgeon 11
• Number of surgeons involved 33

Jacquier I, Boutron I, Ravaud P, Ann. Surgery
37
Reporting guidelines for NPT trials Why?

• Study of rehabilitation papers (n 171)
  published between 1997-1998
• Timing of the intervention 68
• Description of the intervention 50
• Interventions adherence 34

Dijkers at al. Am J Phys Med Rehabil, 2002.
38
Reporting guidelinesDevelopment of the extention
of the CONSORT to NPT

• Steering committee
• D. Moher
• P. Ravaud
• I. Boutron
• Selection of experts
• Editors
• Clinicians
• Methodologists
• Two stages method
• Preliminary survey
• Consensus meeting

39
Preliminary survey

• Objective
• To identify items to be discussed based the
  experts opinion
• Methods
• All items of the CONSORT checklist
• 7 specific questions on key issues were added
• Items were selected for discussion if more than
  1/3 of the experts answered that the item should
  be modified or another item should be added

40
Consensus meeting

• 3 days
• February 8-10th 2006, Paris
• 30 experts
• Agenda
• Presentations
• Discussions
• Consensus
• Manuscript in process

41
Conclusions

• Specific issues in assessing NPT
• Placebo
• Blinding
• Complexity of the intervention
• Care providers influence
• Specific guidelines
• Specific quality tools
• CLEAR NPT
• Specific reporting guidelines
• Development of an extension of the CONSORT for
  NonPharmacological treatment (CONSORT meeting,
  Paris, 2006)