Critical review of articles

1
Critical review of articles

• Torben Jørgensen
• Diabetology Research Course in Epidemiology and
  Biostatistics
• Mumbai, India
• 2003

2
Contents

• Critical evaluation of an article
• Publication bias
• How do we collect available knowledge (evidence
  based medicine)
• Books
• Review articles
• Systematic reviews

3
Why a critical review?

• To gather the existing knowledge
• Do we need further research into this area?
• the dream of a researcher

4
What is a good article?

• An article published in journals with high impact
  factor?
• An article written by a known famous researcher?
• No - not necessarily!

5
Article standard format

• Introduction
• why the study was undertaken
• Material and method
• how it was done
• Results
• what was found
• Discussion
• what it meant

6
Preliminary questions

• Why was the study done?
• What is the problem? - in a nutshell
• Short summary of published literature
• Why another study? - hypothesis?
• What type of study was done?
• Case study, case-control study, cohort study,
  controlled trial, randomised trial
• Was this design appropriate to the research
  question addressed?

7
How to decide whether an article is worth reading?

• Read the introduction and the method section
• If the method is wrong - the interest value of
  the hypothesis, the results and the speculations
  in the discussion are less important
• If you are going to trash an article you should
  do so before reading results and discussion

8
Material and methods

• Is the study original?
• Which group of people/patients?
• Design of the study?
• Is systematic bias avoided or minimised?
• Is the study large enough and continued for
  sufficient long time?
• Are the statistical tests used correctly?

9
Material and methods

• Are you dealing with an original study?
• (Compare aim of the study with review of
  literature)
• Are former studies
• of insufficient methodology
• too small, too short follow-up .
• Is the present study (compared to former studies)
• focusing on other groups of persons/patients

10
Material and methods

• Which group of people/patients?
• (can you recognise the population?)
• Recruitment
• Population-based, primary health services,
  hospitals
• In- and exclusion criteria
• Age, sex, ethnicity, alcoholism etc.

11
Material and methods

• Design of the study
• (Did it make any sense?)
• Which manoeuvre was considered?
• Examples of insufficient description and wrong
  design
• Measurement of outcome - which and how?
• Surrogate end-point gtlt end-points relevant for
  patients and health care system
• Blind assessment of outcome?

12
Material and methods

• Is bias avoided or minimised?
• Randomised clinical trial
• Table comparing the randomised groups?
• Non-randomised clinical trials
• Selection of patients, comparing the groups
• Cohort studies
• non-responders, confounder control
• Case-control studies
• as cohort studies plus selection of cases and
  controls

13
Material and methods

• Is the study large enough and continued for
  sufficient long time?
• Sample size
• Is power calculation mentioned?
• Duration of follow-up
• Does it make sense? (cricket match between India
  and England)
• Completeness of follow-up
• How were drop-outs treated in the analyses?

14
Statistics

• Are the statistical tests used correctly?
• (common tests, explain why they use these tests?)
• Simple table for the most common problems
• Parametric gtlt non-parametric tests
• Paired gtlt unpaired test
• Multivariate tests
• mixed soup? or
• working according to one or more models

15
The substitution game

• Hierarchy of effect
• ex antihypertensive treatment
• Stroke, IHD, mortality
• Blood pressure, minute volume of heart
• Contraction of vessels
• ion effusion over a membrane

16
Publication bias

• 487 trials registered at the ethical committee in
  Oxford - 1984-87
• In May 1990, 285 (59) studies were analysed and
  only 148 (30) were published
• Significant results were published more often
  (OR2,32 95 c.l. 1,25-4,28)
• Significant results gave more presentations and
  articles (high impact factor)

17
Funnel plot
x
x
x
x
x
18
Collection of available knowledge

• Books?
• Review articles
• How were they performed?
• All articles?
• All language?
• etc
• Systematic reviews a.m. Cochrane
• Scientific protocol

19
Review a.m. Cochrane

• A scientific protocol
• Aim, material, in- and exclusion, plan of
  analysis .
• All articles are gathered
• translation to a language understood by reviewer
• additional data asked for
• Analysis performed
• Meta analysis, test for heterogeneity
• Discussion of results