Title: SEARCHING FOR PREPARING SYSTEMATIC REVIEWS
1SEARCHING FOR PREPARINGSYSTEMATIC REVIEWS
- Tehran university students scientific research
center - (TUSSRC)
Kazem Heidari
2How to find studies?
- Very quick search on one electronic database and
find a couple of relevant articles. - Try to find every study thats ever been done
addressing your reviews question.
3- Problems
- Few studies
- Publication bias studies with dramatic results
are much easier to find - Less information
- Limit percision
- Restrict conclusion
4- Problems
- Finding every study
- Not easy and might not be possible
- Many studies are never published
- Published , but not indexed
- Too much effort is needed
5One of the major ongoing efforts of the Cochrane
Collaboration is to make the reports of relevant
studies easier to find.
6Systematic reviews generally focus on reports
from randomized controlled trials (RCTs), when
such data are available because of the most
reliable estimates of effects.
7- Search should
- Be sensitive
- look in a number of different places - not single
- Minimize bias
- Think about finding studies that arent in the
major sources like MEDLINE. - Be efficient
- start looking in the place you expect to have the
highest yield.
8- A comprehensive search for relevant RCTs, which
seeks to minimize bias - One of the essential steps in doing a systematic
review. - One of the factors that distinguishes a
systematic review from a traditional review.
9Where to search?
- Studies have shown that only 30 - 80 of all
known published RCTs were identifiable using
MEDLINE (depending on the area or specific
question)Dickersin 1994 - So MEDLINE, is generally not considered adequate.
10- comprehensive search is necessary because of
- Ensuring that as many studies as possible are
identified - Minimize selection bias for those that are found.
11Bias
- The majority of the journals indexed in MEDLINE
are published in English. - The results of many studies are never published,
and most of these probably remain unknown. - Any summary of only the published reports may
result in an overestimate of effectiveness due to
a publication bias
12SOURCES AND APPROACHES
13Sources for searching
- Electronic databases
- Handsearching
- Checking reference lists
- Identifying unpublished studies
- Personal communication
141.Electronic databases
- CRG specialised register
- Cochrane library
- Other databases
15CRG specialised registers
- Each Cochrane review group is building up a
register of studies relevant to its scope. - It would be more efficient to search these
sources centrally for all trials relevant
16The Cochrane library
- Primary source for clinical effectiveness
information. - Contains 4 databases several other useful
sources - Cochrane Database of Systematic Reviews (CDSR)
- Database of Abstracts of Reviews of Effects
(DARE) - Cochrane Controlled Trials Registry (CCTR)
- Cochrane Methodology Reviews (CRMD)
17(No Transcript)
18(No Transcript)
19CENTRAL
- Contained just over 350,000 citations (Jan 2003)
Includes citations to reports of controlled
trials that might not indexed in MEDLINE, EMBASE
or other bibliographic databases - published in many languages
- citations that are available only in conference
proceedings or other sources that are difficult
to access
20Cochrane controlled tials registry (CCTR)
- Ideally ,it should be a central place to put all
the reports of controlled trials identified
through the work of the Cochrane Collaboration. - It contains the results of searching MEDLINE,
EMBASE, some other databases and a long list of
journals, books and conference proceedings. - Its contents is updated each year
21- The Cochrane Collaboration has been developing an
electronic database of reports of controlled
trials ("CENTRAL") that is now the best single
source of information about records that relate
to studies, which might be eligible for inclusion
in Cochrane Reviews.
22CENTRAL
- The US Cochrane Center and the UK Cochrane Centre
have searched MEDLINE for publication years
1966-2000 using Cochrane highly sensitive search
strategy - Updated each year
- Include citations (RCT CCT) if they meet the
criteria.
23CENTRAL
- Also the UK Cochrane Centre is retrieving records
from EMBASE - checking their titles and abstracts
- submitting these for inclusion in CENTRAL when
appropriate. - First run in 1999(include citations between
1974-99)
24CENTRAL
- Other general healthcare databases published in
Australia, China, and Brazil are undergoing
similar systematic searches to identify reports
of trials for CENTRAL - Each Collaborative Review Group (CRG) is
responsible for the development of a subject
specific specialized register of trials, which
serves to ensure that individual reviewers within
the CRG have easy and reliable access to the
maximum possible number of studies relevant to
their review topic - Using similar methods
- The registers should, in turn, be submitted for
inclusion in CENTRAL. Thus, records included in
the specialized register of one CRG become
accessible to all other CRGs through CENTRAL
25(No Transcript)
26(No Transcript)
27Cochrane Database of Systematic Reviews (CDSR)
- Full-text of systematic reviews undertaken by
Cochrane Collaboration - Protocols
- Updated reviews
28(No Transcript)
29Database of abstracts of reviews of effects (DARE)
- Provided by CRD (Centre for Reviews and
Dissemination) - Database of quality assessed reviews
- Has structured abstracts of systematic reviews
30http//nhscrd.york.ac.uk/darehp.htm
31Other databases
- e.g Medline and EMBASE
- Some other bibliographic and fulltext databases
that are available - Needs an information specialist who is familiar
with the searches needed for systematic reviews
32Medline
- By US National Library of Medicine
- Type of Database Bibliographic
- Over 11,000,000 citations of both clinical and
preclinical studies. - complementary database known as PreMEDLINE
includes citations and abstracts for studies that
have been published recently but not yet indexed. - Subject Coverage All Specialties of Medicine
- 60 of References contain Abstracts
33Medline
- Relatively comprehensive coverage of medical
journals and because it is readily accessible. - Free of charge using
- Pubmed
- Internet Grateful Med .(not available yet)
- Commercial vendor
- OVID
- Knowledge Finder
- Silver Platter
34(No Transcript)
35(No Transcript)
36EMBASE
- Elsevier Science Publishers, Netherlands
- Type of Database Bibliographic
- Numbers of Journals 3500
- Bias European in focus
- Thesaurus Uses own thesaurus
- Subject Coverage Strong on Pharmaceutical and
Clinical Medicine - Materials Indexed Letters, Editorials, Research
Articles from 1974 or 1981 onwards (depending on
method of access) - Overlap with MEDLINE estimated at between 25 and
40 - 75 of References contain Abstracts
- Update Updated Weekly and is about 4 weeks behind
37(No Transcript)
38Scisearch
- Indexes the references cited in articles as well
as usual author ,title ,abstract and citation of
articles - By web of science (from ISI) that is a citation
database - From 4500 major scientific and technical journals
and the articles that cite them - Useful for finding follow up work done on a key
article and for tracking down the address of
authors.
39Scisearch
- Types of Searches
- General Search Search on authors, titles,
keywords, abstracts, sources, institutions, other
topics. - Cited Search Search on specific paper or author
to see who has cited that paper or author in
journal articles. Cited items can include
dissertations, books, book chapters, and any
journal. Weekly Alerting
40Scisearch
- Can be used to
- identify studies for a review by identifying in
the database a known relevant source article - checking each of the articles citing the source
article, to see if it is also relevant to the
review - searching forward in time from the publication
of an important article - includes reference lists for records it indexes.
41(No Transcript)
42(No Transcript)
43Print versions of electronic databases
- Index Medicus is the print version of MEDLINE
- Excerpta Medica is the print version of EMBASE
- Science Citation Index is the print version of
SciSearch
44Electronic databases
- Generally begins with health-related electronic
bibliographic databases.
45Bibliographic databases
- Hundreds of electronic bibliographic databases
exist. - Easiest and least time-consuming
- Example MEDLINE and EMBASE (include abstracts
for the majority of recent records) - Often a searcher can determine an articles
relevance to a review based on the abstract - If the reported study is clearly not eligible for
inclusion, retrieving the full journal article is
necessary.
46Bibliographic databases
- They can be searched electronically
- Title
- Abstract
- Standardized subject related indexing terms
- Example MEDLINE indexing term RANDOMIZED-CONTROLL
ED-TRIAL (Publication Type) was introduced in
1991 and allows a user to search for articles
describing individual randomized trials
47TYPES OF BIBLIRAPHIC DATABASES
- Cover all areas of health care and index journals
published from around the world. - MEDLINE/PubMed
- EMBASE
48- Index journals published in specific regions of
the world. - Australasian Medical Index
- Chinese Biomedical Literature Database
- Latin American Caribbean Health Sciences
Literature (LILACS) - Japan Information Centre of Science and
Technology File on Science, Technology and
Medicine (JICST-E)
49- Focus on specific areas of health
- Cumulative Index of Nursing and Allied Health
(CINAHL) - AIDSLINE
502.HANDSEARCHING
- Definition Handsearching involves a manual
page-by-page examination of the entire contents
of a journal issue to identify all eligible
reports of trials.
51- whether they appear in
- Articles
- Abstracts
- News
columns - Editorials
- Letters
- Conference
proceeding - other
text..
52Need for handsearching
- Not all trial reports are included on electronic
bibliographic databases - For example Conference proceedings are
important to handsearch because individual
conference. These abstracts are not included on
MEDLINE and are not usually included in other
databases. -
- 2. Even when they are included, they may not be
indexed with terms that allow them to be easily
identified as trials
53OTHER REASONS
- Some reports that were published in
MEDLINE-indexed journals do not have
corresponding MEDLINE records - 1- published during a year when the journal was
not indexed - 2- report is of a type not generally indexed
separately by NLM
(e.g., abstracts from conference proceedings) - 3- a report or issue may have been omitted by
mistake (VERY RARELY) -
54- Research has shown that electronic searching for
controlled trials on some databases retrieves
only about half the relevant available studies.
55 - Benefits
- Full Identification of all reports (as much as
possible) - Reducing bias
- most important Published reports tend to
show positive effects, but grey literature in
general has been found to be more likely than
health care journals to contain negative
reports (McAuley 2000) PUBLICATION BIAS - Harms
- Time consuming
56HANDSEARCHINGinCochrane Collaboration
- For complete identification of published reports,
there appears to be no alternative to a
page-by-page search of the literature - Handsearching within the Cochrane Collaboration
refers to the planned searching of a journal page
by page, including editorials, letters, etc., to
identify all reports of randomised controlled
trials and controlled clinical trials.
57HANDSEARCHINGinCochrane Collaboration
- Cochrane Collaboration has encouraged people to
register their handsearching and look for all
reports of controlled trials. - Then collected and put onto the CCTR so that no
one else has to handsearch that source. - List of journals being searched can be downloaded
from the internet.. MASTERLIST
58- Overall coordination of the Collaborations
handsearch of the worlds medical literature is
managed by the US Cochrane Center, which oversees
on the Master List of Journals being Searched - Each CRG is responsible for coordinating
searching activities of specialist health care
journals and to ensure that all RCTs and CCTs
within these journals are identified as
completely as possible and registered
59MASTERLIST
- Almost 3700 journals have been, or are being,
searched within the Collaboration, and are
included in the Master List. - Conference proceedings being searched are also
included. - Managed by US Cochrane Center
- Updated weekly
- Downloaded as an excel file
60- IN SUMMARY MASTER LIST IS THE MOST IMPORTANT
RESOURCE FOR HANSEARCHED JOURNALS - AND
- THE BEST WAY OF AVOIDING THE DUPLICATION OF
EFFORTS.
61- Identify a search coordinator.
- Determine which journal(s) to search.
- Register the intent to search a journal with the
New England Cochrane Center at Providence, using
the Journal Hand Search Registration Form - Train and test searchers
62- Search journal(s)
- Examine each issue, page by page, to identify ALL
reports of RCTs and CCTs, whether they appear in
articles, abstracts, news columns, editorials,
letters or any other text, and regardless of the
relevance of the subject matter to the searcher. - Photocopy the first page of each report
identified, as well as any subsequent pages
requested by a group's search coordinator. - Annotate each photocopy, on either the front or
back, with complete citation information,
wherever necessary. - Classify each report as RCT or CCT (or other as
relevant to the needs of the search group) and
annotate each photocopy with the classification
codes. Highlighting (e.g., with a yellow marker)
or underlining the key terms which justify the
classification may be required by a group's
search coordinator.
63- Forward citations to the New England Cochrane
Center at Providence - Whenever such citation records exist, download
the citation records corresponding with the
search results directly from MEDLINE - Transfer (or enter, when there is none to
transfer) citations into Pro-Cite and add
classification codes to any specified, unused
field.
64- Forward citations to the New England Cochrane
Center at Providence. Include annotated
photocopies of any reports without a
corresponding MEDLINE citation record.
653.Checking reference lists
- Following up references from one article to
another - Looking for previous reviews of the topic and
checking their reference lists - Efficient means of identifying studies for
possible inclusion in a review - Reference lists should never be used as a sole
approach to identifying reports for a review
(because of publication bias)
66Checking other reviews
- Existing reviews are some of the most convenient
and obvious sources of references to potentially
relevant studies. - Obtaining copies of previously published reviews
- Checking for references to the original studies.
67Checking other reviews
- Sources
- CDSR
- DARE (provides information on previously
published reviews of the effects of healthcare) - MEDLINE (the most appropriate review articles
would be indexed under the Publication Type terms
META-ANALYSIS and REVIEW, ACADEMIC) - EMBASE
- Enhanced search strategies
68Print versions of electronic databases
- Searching the earlier printed subject indexes may
be worthwhile, especially if there is reason to
believe that there were early studies of the
intervention being reviewed.
694.Identifying unpublished studies
- What Is Grey Literature?
- Literature that is not widely published
- Dissertations
- Theses
- Government reports
- Ongoing studies
- etc
Grey literature
70Identifying unpublished studies
- Finding out about unpublished studies, and
including them in a systematic review, may be
important to minimizing bias. - There is no easy way to obtain information about
studies that have been completed but never
published
71Identifying unpublished studies
- Sources
- Colleagues
- Web sites
- Informal channels of communication
- Formal letters
- sending a list of relevant articles asking if
they know of any additional studies (published or
unpublished) that might be relevant - send the same letter to other experts or others
with an interest in the area
72Identifying unpublished studies
- Asking researchers for information about
completed but never published studies has not
typically been fruitful
73Identifying unpublished studies
- Identifying ongoing studies may also be important
so that when a review is later updated - No single, central register of ongoing randomized
trials currently exists. - There are hundreds of distinct, predominantly
online registers that vary widely in content,
quality, and accessibility - Various efforts have been made by independent
groups to begin to provide central access to
ongoing trials
74- TrialsCentral
- www.trialscentral.org
- Current Controlled Trials
- www.controlled-trials.com
- clinicaltrials.gov (by NLM)
- www.clinicaltrials.gov
75- SIGLE
- Gateway.nlm.nih.gov
- www.theses.com
- www.dialog.com
- www.dissertationsandtheses.com
76TrialsCentral
- provides free and confidential access to listings
of clinical trials. Information about current
clinical research helps support informed
evidence-based decision making in healthcare - www.trialscentral.org
77(No Transcript)
78Current Controlled Trials
- published by BioMed Central
- providing immediate free access to peer reviewed
biomedical research. - www.controlled-trials.com
79(No Transcript)
80clinicaltrials.gov
- By NLM
- Provides regularly updated information about
federally and privately supported clinical
research in human volunteers. - ClinicalTrials.gov gives you information about a
trial's purpose, who may participate, locations,
and phone numbers for more details - www.clinicaltrials.gov
81(No Transcript)
82System for Information on Grey Literature in
Europe(SIGLE)
- Bibliographic database
- Covering European grey literature
- Fields of natural sciences and technology,
economics, social sciences, and humanities - Produced by EAGLE ,a consortium of major European
libraries, of reports - 1976 to the present
- More than 800,000 citations
- Updated monthly with about 45,000 citations/yr
83Gateway
- Provided by NLM
- Searches
- MEDLINE/PubMed
- LOCATORplus
- MEDLINEplus
- ClinicalTrials.gov
- DIRLINE,
- Meeting Abstracts
- HSRProj
- Gateway.nlm.nih.gov
84(No Transcript)
85(No Transcript)
86Theses.com
- Comprehensive listing of theses with abstracts
accepted for higher degrees by universities in
great Britain and Ireland - www.theses.com
87(No Transcript)
88dialog.com
- providing online-based information services to
organizations seeking competitive advantages in
such fields as business, science, engineering,
finance and law. - Retrieve data from more than 1.4 billion unique
records of key information - www.dialog.com
89(No Transcript)
90(No Transcript)
91dissertationsandtheses.com
- Searchable database of research papers
- Lists more than 25,000 quality research papers,
experimental studies, and even examples of entire
theses - www.dissertationsandtheses.com
92(No Transcript)
935.Personal communication
- To know of studies you havent yet found
- Reviewers send a list of the studies they have
found to the authors of those studies, asking if
they are aware of any other relevant studies. - Or write to the manufacturers of relevant drugs
or devices and ask if they are aware of any other
studies.
94Evidence on adverse effects
- sources
- reports from trials or other studies included in
the systematic review (including excluded
reports) - Electronic sources
- Current Problems (by the UK Medicines Control
Agency) - http//www.open.gov.uk/mca
- MedWatch (by FDA)
- Australian Adverse Drug Reactions Bulletin
- http//www.health.gov.au/
- Other regulatory authorities and the drug
manufacturer - Other types of studies (e.g. cohort and
case-control studies,uncontrolled trials, case
series and case reports) that must be interpreted
with caution.
95Search Strategy
96Developing a search strategy
- For many CRGs, the specialized register is still
the best available source of studies for a given
review - Many Trials Search /Review Group Coordinators
search their CRGs specialized register for
reviewers on request - Specialized registers can also be searched
through CENTRAL, which contains a recent version
of the registers for most CRGs
97Developing a search strategy
- The ultimate goal in developing a specialized
register for a CRG is that it can serve as an
all-inclusive source of reports relevant to the
CRGs scope and topic area such that a relatively
simple search using some key words related to the
intervention could be run against the specialized
register to identify all relevant studies.
98Developing a search strategy
- Most CRG specialized registers have not yet
reached this point of comprehensiveness - For many CRGs, the specialized register is still
the best available source of studies for a given
review - Many Trials Search /Review Group Coordinators
search their CRGs specialized register for
reviewers on request - Specialized registers can also be searched
through CENTRAL, which contains a recent version
of the registers for most CRGs
99Developing a search strategy
- Balance between comprehensiveness and precision
when developing a search strategy is necessary. - Increasing the comprehensiveness of a search
entails reducing its precision and retrieving
more non-relevant articles
100Developing a search strategy
- Developing a search strategy is an iterative
process in which terms that are used are
modified, based on what has already been
retrieved. - There comes a point where the rewards of further
searching may not be worth the effort required to
identify the additional references
101Developing a search strategy
- The decision as to how much to invest in the
search process depends on the question a review
addresses, the extent to which the CRG's
specialized register is developed, and the
resources that are available
102Developing a search strategy
- Search other electronic bibliographic databases
- Information specialists
- To avoid many errors
- Ensure that database-specific search term syntax
will be appropriate. - Advanced searching techniques can be employed
where available. - Aware of greater importance of high recall (i.e.
sensitivity) as compared to precision in
searching for studies for systematic reviews - Ideally,reviewers should be present when the
search is done - Judgments about what to download often need to
be made while the search is being done. (because
of cost) - Helpful in suggesting terms for the health
condition and intervention
103- Electronic search strategy
- terms to search for the health condition of
interest - terms to search for the intervention(s) evaluated
- terms to search for the types of study design to
be included
104- Approach
- Begin with multiple terms that describe the
health condition of interest and join these
together with the Boolean 'OR' operator - Do likewise for a second set of terms related to
the intervention(s) and for a third set of terms
related to the appropriate study design - Three sets of terms can then be joined together
with the AND operator. - CAUTION
- If an article does not contain at least one term
from each of the three sets, it will not be
identified. - No language restrictions should be included in
the search strategy
105One problem
- In Medline
- The term randomized controlled trial from 1991
in publication type - The term controlled clinical trial from 1995 in
publication type
106- The Cochrane highly sensitive search strategy for
MEDLINE was developed specifically with the needs
of Cochrane reviews in mind. (first version in
1994) - It may still be worthwhile for reviewers to
search MEDLINE using the Cochrane highly
sensitive search strategy and to obtain and check
the full reports of possibly relevant citations.
CRGs typically use this plus subject matter terms
107Cochrane high sensitive search strategy for RCTs
108Documenting a search strategy
- How,what,when
- Help to avoid repeat search
- Help people using your review appraise how well
they think youre minimized bias - All details should be documented in the search
strategy section of review
109Documenting a search strategy
- Electronic databases
- described in sufficient detail
- included for each electronic bibliographic
database each time it is searched.
110- Title of database
- Name of the host
- Date search was run
- Years covered by the search
- Complete search strategy used
- Summary of the search strategy(health condition
,intervention and type of study) - The absence of any language restrictions
111- Journal Handsearching
-
- journal years searched should be listed in the
Search strategy - Ideally, the full titles should be used for the
journals - Example British Journal of Surgery January 1948
December 1998
112- Conference Proceedings
- Details of the conference proceedings searched
for the review should be provided - Proceedings with a title in addition to the
conference name - Proceedings without a separate title
- Proceedings in a language other than English
- Proceedings also published as part of a journal
113- Unpublished studies
-
- Brief summary including databases searched
- database details
- Efforts to contact investigators for information
about unpublished studies.
114Selecting studies
- The process by which studies will be selected for
inclusion in a review should be described in the
review protocol. - The first stage of checking the results of an
electronic search involves assessing titles and
abstracts - It is important to err on the side of
over-inclusion because once a study has been
excluded from the selection process it is
unlikely to be reconsidered
115- given the information available, it can be
determined that an article definitely does not
meet inclusion criteria, it can be rejected. If
the title or abstract leave room for doubt that
the article cannot definitely be rejected, the
full text of the article should be obtained.
Reading the full text may lead the reviewers to
exclude the study because it does not meet
inclusion criteria.
116- Assessing for inclusion criteria
- More than one reviewer assess the relevance of
each report - Decisions concerning relevance will be made by
content area experts, non-experts, or both - people assessing the relevance of studies know
the names of the authors, institutions, journal
of publication and results when they apply the
inclusion criteria - How disagreements will be handled between 2
reviewrs - Consensus
- Third person
- categorize as one that is awaiting assessment
117- 2 reviewers is recommended
- One reviewer is knowledgeable in the area under
review - Second reviewer who is not an expert in the area
- Some reviewers may decide that assessments of
relevance should be made by people who are blind
or masked to the journal from which the article
comes, the authors, the institution, and the
magnitude and direction of the results by editing
copies of the articles (takes much time)
118- Excluded studies should be listed
- the reviewer can show that consideration has been
given to these studies.
119Keeping track of identified studies
- You may find several reports of the same study
- You may find the same report of a study in
several databases.
120Keeping track of identified studies
- So we need
- Some way of keeping track of the references weve
looked at. - Some way of grouping together all the reports of
a single study. - Also we might like to keep a record of where you
found each study ,so that you can report how
useful different sources were.
121Keeping track of identified studies
- Some people use reference management software to
do all this - ProCite
- Reference Manager
- EndNote
- IdeaList
- Other people prefer printing out citations and
writing on them.
122Keeping track of identified studies
- We need some system for keeping track of which
references you think are relevant - Which ones you have ordered from the library
- which ones youve received the paper for
- etc
123- ProCite
- the most widely used package and the one for
which support to editorial bases is most widely
available. - the preferred database for submitting controlled
trials and specialized registers to CENTRAL - eases the work of identifying duplicate
references - facilitates storage of information about the
methods and process of a search - Reference Manager
- EndNote
124- General database packages
- Access
- FoxPro
- MeerKat
- developed by the UK Cochrane Centre
- allows for a specialized register to be organized
around studies, instead of the publications or
reports generated from these studies - each report can be associated with the
corresponding study - Facilitating the work of the Review Group
Coordinator/Trials Search Coordinator - allows complex database searches
- Ease the task of managing references within a
CRG.
125Time saving for searching
- Suggestions
- Look at the terms used to index and describe a
few studies you already know are relevant to your
review, and use these terms in your search
strategy - Add new terms to your search strategy and then
pilot them on part of the database - Use date limits for your search if appropriate.
126Thank you!