Pubmed vs. Full-Text Query Performance in Systematic Reviews : Application to Non-inferiority Clinical Trials

1
Pubmed vs. Full-Text Query Performance in
Systematic Reviews Application to
Non-inferiority Clinical Trials

• André Nguyen Van Nhieu1, 2, Katet Moez1, Michel
  Nougairede 2, Xavier Duval4, Michaël
  Schwarzinger1
• 1 ATIP-AVENIR Inserm Modélisation, Aide à la
  Décision, et Coût-Efficacité en Maladie
  Infectieuses, U738, Université Denis Diderot,
  Paris, France 2 Département de Médecine
  Générale, Université Denis Diderot, Paris, France
• 3 Inserm U738, Université Denis Diderot, Paris,
  France 4 Inserm CIC 007, AP-HP, Hôpital
  Universitaire Bichat, Paris, France

2

• I declare no conflicts of interest

3
Introduction (1)

• Systematic review extensive research of
  appropriate publications in the literature
• Usually performed through Pubmed using key-words
• Methodology appropriate when keywords in
  title/abstract and MeSH

4
Introduction (2)

• Comparison of Pubmed VS FULL-TEXT
• Application Non-inferiority trials in infectious
  diseases1
• According to Piaggio CONSORT statement JAMA 2006
  improving quality of reporting Non-inferiority
  trials
• Hypothesis Pubmed as sensitive as FULL-TEXT

5
Objective

• To compare the performance of 2 query strategies
  to identify non-inferiority trials with mortality
  as a primary outcome in infectious diseases
• Pubmed
• Full-text using the search engine of each journal

6
Methods (1)

• Original articles published in 2001-2012
• In
• Generalist journals
• N Engl J Med, Lancet, JAMA, Ann Intern Med, BMJ,
  Arch Intern Med
• Specialist journals
• Lancet Infectious Disease, Clinical Infectious
  Disease, Journal of Infectious Disease, AIDS,
  Vaccine, Pediatrics, PlosMed

7
Methods (2)
8
(No Transcript)
9
(No Transcript)
10
Methods (3)

• Using
• PUBMED keywords randomi AND (non?inferior
  OR not inferior OR is inferior OR was inferior)
  AND (surviv OR alive OR mortality OR death OR
  fatal) indexation (randomized controlled
  trial, Kaplan-Meier method, survival, mortality,
  death)
• FULL-TEXT method with the same keywords adapted
  according to each search engine

11
Methods (4)

• Definitions
• True positive (TP) Non-inferiority trial (NIT)
  with mortality as primary outcome found by one or
  the other query strategy
• False positive (FP) Original article but it is
  not a NIT or a NIT with primary outcome ?
  mortality
• False negative (FN) NIT fount by one query
  search but not the other
• Positive predictive value (PPV) probability to
  find a TP among all Positive of a query strategy

12
Results (1)
Flowchart
True positive
False positive
13
Results (2)
FULL-TEXT PUBMED Sensitivity Positive Predictive Value
55/55 (100) 19/55 (34,5) 55/256 (21,5) 19/45 (42,2)
False negative 0/55 (0) 36/55 (65,5)
14
Results (3)

• Subgroup analysis
• Sensitivity not different between generalist /
  specialist journals (p0,14)
• Sensitivity FULL-TEXT Pubmed not different
  lt2008 / gt2008 (p0,07)

15
Limits of FULL-TEXT method

• Repetition in each journal search engine
• Specificity of each journal search engine
  learning curve
• Access to journals for GP is not free
• Time consuming

16
Conclusion (1)

• FULL-TEXT is 100 sensitive
• Pubmed detects only 34,5 of non-inferiority
  trials in infectious diseases
• High rate of false positive in FULL-TEXT
• However, False Positive easily identified and
  excluded through reading

17
Conclusion (2)

• Testing the Full-text method in other medical
  fields or repeating in a few years (improving
  Pubmed?)
• Improvement to be made with CONSORT statement
• Adding Non-inferiority in MeSh terms

18

• THANK YOU

19
Bibliography

• Le Henanff et al. / RAVAUD Quality of Reporting
  of Noninferiority and Equivalence Randomized
  Trial JAMA 2010
• Piaggio et al. Reporting of Noninferiority and
  Equivalence Randomized Trials JAMA 2006 and
  Extension of the CONSORT 2010 Statement JAMA 2012