Publication bias

1
Publication bias
Reza Yousefi Nooraie
2
(No Transcript)
3
Publication bias

• The likelihood of finding studies is related to
  the results of those studies

4
Funnel plots

• A funnel plot is a scatter plot of treatment
  effect against a measure of study size.

5
(No Transcript)
6
Why Funnel?

• precision in the estimation of the true treatment
  effect increases as the sample size increases.
• Small studies scatter more widely at the bottom
  of the graph
• In the absence of bias the plot should resemble a
  symmetrical inverted funnel

7
(No Transcript)
8
Publication Bias
Asymmetrical appearance of the funnel plot with a
gap in a bottom corner of the graph
9
Publication Bias

• In this situation the effect calculated in a
  meta-analysis will overestimate the treatment
  effect
• The more pronounced the asymmetry, the more
  likely it is that the amount of bias will be
  substantial.

10
Alendronate for Postmenopausal Osteoporosis
11

• the funnel plot should be seen as a means of
  examining whether the smaller studies tend to
  show larger treatment effects
• This may be due to reasons other than publication
  bias

12
Possible sources of asymmetry in funnel plots

• 1.Selection biases
• Publication bias
• Location biases
• Language bias
• Citation bias
• Multiple publication bias
• 2. Poor methodological quality of smaller studies
• Poor methodological design
• Inadequate analysis
• Fraud
• 3. True heterogeneity
• Size of effect differs according to study size
  (for example, due to differences in the intensity
  of interventions or differences in underlying
  risk between studies of different sizes)
• 4. Chance

13
Publication bias

• Language bias
• Citation bias
• Multiple publication bias

14
Publication bias Approaches

• Attempt to Retrieve all Studies
• Worst Case Adjustment
• Number of unpublished negative studies to negate
  a positive meta-analysis
• X N x (ES) / 1.6452 - N
• where N number of studies in meta-analysis,
• ES effect size
• Example
• If N 25, and ES 0.6 then X 58.2
• Almost 60 unpublished negative studies would be
  required to negate the meta-analysis of 25
  studies.

15
Poor methodological quality

• Smaller studies are, on average, conducted and
  analysed with less methodological rigour than
  larger studies.
• Trials of lower quality also tend to show larger
  treatment effects
• Trials which, if conducted and analysed properly,
  would have been negative may thus become
  positive

16
Poor methodological quality
17
Publication Bias
18
Poor methodological quality

• Furthermore, some interventions may have been
  implemented less thoroughly in larger trials
• therefore, have resulted in smaller estimates of
  the treatment effect

19
True heterogeneity

• For example, substantial benefit may be seen only
  in patients at high risk for the outcome
• These high risk patients are usually more likely
  to be included in early, small studies

20
Chance

• Finally, it is possible that an asymmetrical
  funnel plot arises merely by the play of chance.

21
DrawBacks

• Symmetry or asymmetry is generally defined
  informally, through visual examination
• The visual interpretation of funnel plots may
  vary between observers

22
DrawBacks

• The power of this method is limited when it
  comprises only a few small studies.
• at least 25 being required for modest power

23
DrawBacks

• Finally, it should be remembered that although
  funnel plots may alert to a problem
• they do not provide a solution to it