Evaluating Medical and Public Health Publications

1
Evaluating Medical and Public Health Publications

• PH 530
• Biostatistics
• University of Tennessee
• Public Health Program
• Paula Zemel, PhD

2
Tips for using this presentation

• Read the goal and objectives.
• Review the presentation at your own pace.
• Read Chapter 15 in the text.
• Complete the Practically Speaking activities.
• Review several journals in public health and/or
  your content area.

3
Tips for using this presentation, cont.

• Look for research on the Internet. Apply
  criteria to distinguish credible from
  non-credible sources.
• Select a primary research article from a
  peer-reviewed health journal in an area of
  interest.
• Make sure the statistics used in the article
  include at least one inferential statistic
  included in class.
• Use guidelines in Class Notes to write your
  critique.

4
Goal and Objectives

• Goal Critically read and evaluate research
  reports in public health
• Objectives
• Identify major components of research reports.
• Evaluate information presented in sections of
  research reports.
• Critique published reports of research.
• Apply findings of published research to public
  health practice.

5
Types of Research Reported

• Primary
• original report of planned research
• Secondary
• reanalysis of data (sometimes called post-hoc
  assessment
• evaluation of large data sets often in the public
  domain
• e.g. National Health and Nutrition Examination
  Study (NHANES-3) survey

6
Types of Research Reported, cont.

• Secondary Research, cont.
• Meta-analysis
• technique used to combine analyze results of
  several studies
• Literature Reviews
• author(s) review and interpret published studies.

7
Get to the Research Source

• Primary sources of research
• includes original research
• communicated by principal investigators of
  project
• addresses methods and results
• methodology is complete and clear enough to be
  reproduced

8
Get to the Research Source

• Primary sources of research
• manuscript submission process is described in
  journal, usually in one issue/year
• review process provides more assurance of quality
• research generally has not been published
  elsewhere
• standards and guidelines for electronic
  publications are in development

9
Get to the Research Source

• Where to find primary research?
• Peer-reviewed journals
• Peer-reviewed abstracts from professional
  meetings
• Scientific reports (print and electronic) from
  symposia
• Government reports (print and electronic)

10
Get to the Research Source

• Where to find primary research?
• Peer-reviewed journals
• also called refereed journals
• manuscripts are evaluated by panel of reviewers
  before accepted for publication
• editor(s), editorial board and ad hoc reviewers
  are usually listed in journal

11
Practically Speaking

• Review a current journal in your field.
• What are the guidelines for authors?
• Does it use a peer-review process?
• How does it identify...
• Research articles?
• Review articles?
• Feature articles?
• Editorial or opinion articles?
• Review a research resource on the Internet. Were
  similar methods used to assure quality?

12
Evaluating Research in Public Health

• In order to critique a research article, first
  review the anatomy of a journal article
• Title, Author(s), Key words
• Abstract
• Introduction
• Methods
• Results
• Discussion/Conclusion
• References

13
Title, Author(s), Key words

• Now that you have identified an article that
  presents results of primary research, review the
  title, authors and key words.
• Do the key words match your areas of interest?
• Does the title suggest the article might contain
  information useful to your area of practice?
• Who are the authors? Do the authors or their
  affiliations suggest they are writing in their
  area of expertise?

14
Abstract

• Read the abstract to decide if you should read
  the entire article.
• Abstract should identify the focus of the
  article.
• Two types of abstracts
• Structured A structured abstract follows a
  standard format organized by subheadings.
  Structured abstracts make it easier to evaluate
  an article.
• Unstructured A maximum number of words may be
  specified for author, but the format is not
  specified. Abstract provides a summary of the
  article, but may be difficult to compare to other
  articles.

15
Abstract

• The abstract should help you further decide if
  you should read the article
• If the study is properly designed and analyzed,
  are the results important and worth knowing
  about?
• If the results are statistically significant, are
  they clinically worth knowing about?

16
Practically Speaking

• Searching by computer
• Know the databases you can search
• Use the thesaurus to make sure you are using the
  right terms and that they are not too broad or
  too narrow
• Review the title, authors, keywords and abstract
  to determine if the article meets your needs
• On the internet, a search engine currently
  accesses about 1/3 of all sourcesuse a search
  engine that searches many others, such as
  www.search.com if you want a broader search
• Evaluate sources carefully!

17
Introduction

• Addresses what research has been performed in the
  area and how this study is different.
• The last paragraph or sentence usually includes
  the purpose of the study.
• The lack of a clearly stated research question is
  the most common reason manuscripts are rejected
  in health research.

18
Introduction

• A brief literature review should provide enough
  background information to place the paper in its
  proper context
• What is the general background?
• Why does this study need to be done?
• The literature review is written at an
  appropriate level for the intended audience-not
  at too high (assuming too much prior knowledge)
  or too low (giving too much basic information).

19
Practically Speaking

• Review the abstract and introduction of a
  research article in public health.
• What type of abstract was included?
• Who is the intended audience? Is the literature
  review appropriately targeted for the intended
  audience?
• Is the difference between this study and others
  in the literature clear? We planned this study
  because no one had done this.. is not sufficient
  justification...
• Do you have a clear understanding of the purpose
  and research question? If not, what could the
  authors add to clarify the purpose/research
  question?

20
Methods

• Details how the study was done
• research design and hypotheses
• reliability and validity of new methods
• outcome measures and assessment criteria
• subjects
• procedures
• statistics

21
Methods

• Subjects
• describe inclusion and exclusion criteria for
  subjects
• number? age? Gender? Preexisting conditions?
  Medication use? Pregnancy? How was informed
  consent obtained? What was the sampled
  population? Sampling plan? Refusals?
  Withdrawals? Retention or response rate?
  Unanticipated occurrences or outcomes?

22
Methods

• Procedures
• Should be clear enough to replicate study
• How was study design implemented?
• Are methods referenced or validated?
• How were potential forms of bias minimized?
• (eg observer blias, sampling bias, selection
  bias, response bias, dropout bias, memory bias,
  participant bias?)

23
Methods

• Procedures
• How are data quality ensured?
• Lab assays?
• Data entry?
• Duplicate collection on random sample for
  measured with questionable reliability?
• Triangulation (3 measures that are hypothesized
  to vary in the a systematic way)

24
Methods

• Procedures
• Statistical methods
• should be described in enough detail so the
  reader knows that methods were appropriate and
  how data were analyzed
• what descriptive and/or inferential statistics
  were used?

25
Practically Speaking

• Use the flowcharts in your text to check the
  authors use of statistics
• What are the outcome measures?
• What type of data does each represent? (Nominal?
  Ordinal? Numerical?
• Are research questions addressing
• differences between groups?
• Associations? Predictions?
• Continue to follow the flowchart to check
  assumptions and determine if the test used was
  appropriate!

26
Resultsjust the facts!

• Should relate to research questions and purpose
  of study
• Results should include meaningful descriptive
  statistics, results of inferential statistics
  reported with p-values or confidence levels
• tables and graphs to present data clearly
• interpretation of results and/or discussion of
  what they mean do not belong in the results
  section

27
Discussion/Conclusion

• Should address
• research questionsaccept or refute?
• Are findings consistent with other published
  results in the area? Why or why not?
• Conclusionwhats it all about
• Conclusions should not reach beyond the data.

28
Discussion/Conclusion

• What are shortcomings, what went wrong with the
  study, what would authors have done differently?
  (the best laid plans
• Suggestions for future research, based on
  results? What new questions or issues do results
  raise?
• How can results be applied to public health
  practice?
• What should practitioners continue to do or do
  differently as a result of this research?

29
References

• Are references appropriate? Complete? Timely?
• Do references address primary or secondary
  research?
• What style does journal use? If the guidelines
  for authors do not address a particular type of
  reference, what stylebook do they use (e.g. APA,
  Index Medicus, MLA)

30
Overall Evaluation

• What is your overall critique of the article?
• What do the tables and figures add to the
  manuscript?
• Is this research project creative? Can results
  of this research path eventually make an
  important contribution to health?
• Is the principle author someone you could see as
  a mentor?
• Other comments? questions raised
• as a result of reading this article?

31
Practically Speaking

• How can this article be applied to your graduate
  work now? Next semester or next year?
• Did this article have an impact on you, either
  negatively or positively?
• Will you do anything differently as a result of
  reading this article?