QUALITY APPRAISAL OF PRACTICE GUIDELINES SYSTEMS AGREE GRADE

1
QUALITY APPRAISAL OF PRACTICE GUIDELINES
SYSTEMS AGREE - GRADE
Paola Andrea Mosquera. Psi. MSc Guideline
Methodology Group Clinical Research Institute
National University of Colombia
2
CLINICAL PRACTICE GUIDELINES

• Clinical practice guidelines are systematically
  developed statements to assist practitioner and
  patient decisions about appropriate health care
  for specific clinical circumstances.
• Their purpose is to make explicit
  recommendations with a definite intent to
  influence what clinicians do
• Lohr KN, Field MJ. A provisional instrument for
  assessing clinical practice guidelines. In Field
  MJ, Lohr KN (eds). Guidelines for clinical
  practice. From development to use. Washington
  D.C. National Academy Press, 1992.
• Hayward RSA, Wilson MC, Tunis SR, Bass EB,
  Guyatt G, for the Evidence-Based Medicine Working
  Group. Users guides to the Medical Literature.
  VIII. How to Use Clinical Practice Guidelines. A.
  Are the Recommendations Valid? JAMA, 1995 274,
  570-574.

Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.
3
DEVELOPMENT THE CPG
Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.
4
DEFINITION OF QUALITY IN A CLINICAL PRACTICE
GUIDELINES

• The certainty that the potential biases of
  guideline development have been addressed
  adequately and that the recommendations are valid
  and feasible for practice.
• This process involves taking into account the
  benefits, harms and costs of the recommendations
• The assessment includes judgements about the
  methods used for developing the guidelines, the
  content of the final recommendations, and the
  factors linked to their uptake.

Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.
5
WHY ASSESS THE QUALITY OF CPG?

• CPG's users need to have confidence in their
  recommendations and how these have been
  developed.
• These recommendations should be
• Developed systematically
• Based on the best available evidence
• With a full understanding of the context of the
  clinical problem and the considerations of users

Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.
6
MANY GRADING SYSTEMS

• Multiple systems are available to assessment the
  quality of recommendations
• Widely known and most used are
• Canadian Task Force on Preventive Health Care
• US Preventive Services Task Force (Agency for
  Healthcare Research and Quality-AHRQ).
• US Agency for Health Care Policy and Research
• Oxford Center for Evidence-based Medicine
• Scottish Intercollegiate Guidelines Network
• National Institute for Clinical Excellence

Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.
7
APPRAISAL OF GUIDELINESFOR RESEARCH AND CLINICAL
PRACTICE
Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.
8
AGREE INSTRUMENT

• Is a generic tool designed primarily to help
  producers and users of clinical practice
  guidelines, in assessing the methodological
  quality of CPGs.
• Assesses both the quality of the reporting and
  the quality of some aspects of recommendations.
• It provides an assessment of the predicted
  validity of a guideline (likelihood that a CPG
  achieve the expected outcome).

Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.
9
STRUCTURE AND CONTENT
Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.
10

• Each item is rated on a 4-point scale ranging
  from 4 Strongly Agree to 1 Strongly Disagree,
  with two mid points 3 Agree and 2 Disagree
• Domain scores can be calculated by summing up all
  the scores of the individual items in a domain
  and by standardizing the total as a percentage of
  the maximum possible score for that domain.

Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.
11

•   VALUE OBTAINED IS IMPORTANT FOR THE OVERALL
  ASSESSMENT

Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.
12
RECOMMENDATIONS TO ASSESS THE QUALITY OF CPG

• Appraisers should attempt to identify all
  information about the guideline development
  process prior to appraisal.
• Is recommended to read the guide and all
  accompanying documentation before beginning the
  assessment.
• Each guideline must be assessed by at least two
  independent reviewers, preferably four because it
  will increase the reliability of the assessment.

Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.
13
GRADING QUALITY OF EVIDENCE AND STRENGTH OF
RECOMMENDATIONS
Clinical guidelines are only as good as the
evidence and judgments they are based on
Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.
14
SYSTEM FOR ASSESSMENT QUALITY OF EVIDENCE AND
STRENGTH OF RECOMMENDATIONS
Grading of Recommendations Assessment,
Development, and Evaluation Working Group

• www.gradeworkingruop.org

Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.
15
GRADE FROM EVIDENCE TO RECOMMENDATION

• Objectives of the GRADE system
• Overcome the limitations of previous systems
• Improving transparency (systematic methods)
• Create a system that could be widely applicable
• Improve communication between developers and users

Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.
16
DEFINITIONS

• The QUALITY OF EVIDENCE indicates the extent to
  which one can be confident that an estimate of
  effect is correct.
• The STRENGTH OF A RECOMMENDATION indicates the
  extent to which one can be confident that
  adherence to the recommendation will do more good
  than harm.

Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.
17
GRADE METHOD

• Can be summarized in three points
• Categorize the outcome variables and their
  relative importance.
• Assess the quality of the evidence for each of
  the outcome variables, then the overall quality.
• Grading strength of the recommendations in only
  two categories (strong or weak recommendations).

Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.
18
1. CLASSIFYING THE RELATIVE IMPORTANCE OF THE
OUTCOMES

• Defining the outcome variables for each question
  to be addressed by the guide and describes the
  relative importance of each one.
• These variables can have different importance in
  decision making process, often patients and
  caregivers can assess their importance in a
  different way to clinicians

Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.
19
2. GRADING QUALITY OF EVIDENCE

• Is performed for each outcome variable in
  decision making, so that quality of each variable
  may be different depending on the number of
  quality studies and their characteristics.
• It takes into account 4 elements to assess the
  quality

Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.
20
CRITERIA FOR ASSIGNING GRADE OF EVIDENCE
Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.
21
OVERALL QUALITY OF EVIDENCE

• The critical variables have a greater weight in
  overall quality assessment and will therefore
  these will be determinant for the recommendation
  strength.
• The lowest quality of evidence in the guideline
  must be based in the lowest quality for every
  critical outcomes.

Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.
22
3. STRENGTH OF RECOMMENDATIONS

• For the development of recommendations you
  should take into account the following factors
• Quality of the evidence
• Balance between desirable and undesirable effects
• Values and preferences
• Costs (resource allocation)

Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.
23
GRADING STRENGTH OF RECOMMENDATIONS

• STRONG RECOMMENDATION
• the panel is confident that the desirable
  effects of adherence to a recommendation outweigh
  the undesirable effects.
• WEAK RECOMMENDATION
• the panel concludes that the desirable effects
  of adherence to a recommendation probably
  outweigh the undesirable effects, but is not
  confident.

Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.
24
SUMMARY POINTS
25
EVIDENCE AND CONTEXT

• Assess the evidence and develop recommendations
  is a complex process, it is necessary to take
  into account many factors, before proposing to do
  or do not with patients.

Guideline Methodology Group. Clinical Research
Institute. National University of Colombia.