Royal College of Surgeons in Ireland Col

1
Royal College of Surgeons in IrelandColáiste
Ríoga na Máinleá in Éirinn
Register of Clinical Prediction Rules,
Methodological Quality Assessment and
Implementation Strategies Emma Wallace, Claire
Keogh, Susan Smith, Borislav D. Dimitrov, Tom
Fahey
2
Outline

• Section 1
• Development of register of Clinical Prediction
  Rules (CPRs)
• Search strategies to identify CPRs relevant to
  primary care from the MEDLINE database
• Section 2
• - Methodological quality assessment of CPRs
• Section 3
• - Implementation strategies

3
SECTION 1

• Development of Register of Clinical Prediction
  Rules (CPRs)

4
Definitions

• Clinical Prediction Rule
• Clinical tool that quantifies the contribution of
• Patient History
• Physical Examination
• Diagnostic Tests
• Stratify patients diagnosis
• Probability of having target disorder.
• Outcome can be in terms of diagnosis, prognosis,
  referral or treatment

5
CPR Examples1) Ottawa ankle rule
6
2) The Centor Score
7
Stages of development of CPR
8
Aim

• Develop a web-based register of CPRs for use in
  primary care as part of Cochrane Primary
  Healthcare Field

9
CPR register development

• Development of an electronic search
  filter-identify CPRs relevant to primary care
• Haynes filter in Clinical Queries
• Broad and narrow search filters

10
30 journals included on the register
Academic Emergency Medicine Family Medicine
American Family Physician Family Practice
American Journal of Medicine Journal of American Medical Association
Annals of Emergency Medicine Journal of the American Board of Family Medicine
Annals of Family Medicine Journal of Clinical Epidemiology
Annals of Internal Medicine Journal of Family Practice
Annals of Medicine Journal of Internal Medicine
Annual Review of Medicine Lancet
Archives of Internal Medicine Medical Care
BMC Family Practice Medical Decision Making
British Medical Journal Medicine
British Journal of General Practice New England Journal of Medicine
Canadian Family Physician Public Library of Science Medicine
Canadian Medical Association Journal Primary Care
Cochrane Database Systematic Reviews Scandinavian Journal of Primary Health Care
11
Methods

• Manually searched 30 journals relevant to primary
  care for the year 2008 (reference standard)
• 7 individual electronic searches of the 30
  journals (each filter treated as diagnostic
  tests)
• Test accuracy analysis Sensitivity and
  specificity
• Aim to maximise sensitivity

12
Database Filter name Filter search string
PubMed Haynes Broad Filter (HBF) (predicttiab OR predictive value of testsmh OR scortiab OR observtiab OR observer variationmh)
PubMed Haynes Narrow Filter (HNF) (validationtiab OR validatetiab)
EBSCO host McGrath/Murphy Broad Filter (MMBF) ((predict N3 rule OR predict N3 model OR predict N3 models) OR (decision N3 rule) OR (TX validat))
EBSCO host McGrath/Murphy Narrow Filter (MMNF) ((predict N3 rule OR predict N3 model OR predict N3 models) OR (decision N3 rule))
PubMed Teljeur/Murphy Inclusion Filter 26 item (TMIF-26) "clinical prediction" OR "clinical model" OR "clinical score" OR "decision rule" OR "diagnostic accuracy" OR "diagnostic rule" OR "diagnostic score" OR "diagnostic value" OR "predictive outcome" OR "predictive rule" OR "predictive score" OR "predictive value" OR "predictive risk" OR "prediction outcome" OR "prediction rule" OR "prediction score" OR "prediction value" OR "prediction risk" OR "risk assessment" OR "risk score" OR "validation decision" OR "validation rule" OR "validation score" OR (derivation AND validation) OR (sensitivity AND specificity) OR (symptoms AND signs)
PubMed Teljeur/Murphy Inclusion Filter 22 item (TMIF-22) (clinicaltiab AND predicttiab) OR (clinicaltiab AND modeltiab ) OR (clinicaltiab AND scoretiab) OR (decision tiab AND ruletiab) OR (derivetiab AND validattiab) OR (diagnostiab AND accuratiab) OR (diagnostiab AND ruletiab) OR (diagnostiab AND scoretiab) OR (diagnostiab AND valuetiab) OR (predicttiab AND outcometiab) OR (predicttiab AND ruletiab OR (predicttiab AND scoretiab ) OR (predicttiab AND validattiab) OR (predicttiab AND valuetiab) OR (risktiab AND assessmenttiab) OR (risktiab AND scoretiab) OR (sensitivitytiab AND specificitytiab) OR (symptomstiab AND signstiab) OR (validattiab AND decisiontiab) OR (validattiab AND ruletiab) OR (validattiab AND scoretiab) OR (predicttiab AND risktiab)
PubMed Teljeur/Murphy Exclusion Filter (TMEF) (allele OR amino OR animal OR apoptosis OR chromosome OR congenital OR dental OR dna OR endogenous OR endothelial OR epithelial OR mammalian OR mice OR molecule OR molecular OR mouse OR mutate OR mutation OR necrosis OR pathogenesis OR phosphorylation OR polymorphism OR receptor OR signal OR species OR tissue OR tumor OR tumour OR tyrosine OR vitro)
13
Results
Manual reference standard search retrieved 6344
articles, 41 of which were CPRs
Filter name N articles retrieved N CPRs retrieved Sensitivity () Specificity ()
Haynes Broad Filter 1251 31 76 81
Haynes Narrow Filter 89 12 29 99
McGrath/Murphy Broad Filter 264 23 56 96
McGrath/Murphy Narrow Filter 63 16 39 99
Teljeur Murphy Inclusion Filter-26 item 2432 39 95 62
Teljeur/Murphy Inclusion Filter-22 item 693 34 83 90
Teljeur/Murphy Exclusion Filter 3589 24 59 43
Haynes Narrow Filter OR Teljeur Murphy Inclusion Filter-26 item 2594 40 98 59
14
MEDLINE versus the final search filter applied
to 30 primary care journals (1966 2008)
Number of articles in PubMEd
Year of publication
15
Creating the International Register Preliminary
findings
2008-1980 resulted in 383 CPRs and 141
Questionnaires relevant to primary care
Years Average articles retrieved per year Average CPRs for primary care identified Average questionnaires for primary care identified
2008 2000 2622 27 8
1999-1990 2328 9 6
1989-1980 2243 5 1
Preliminary findings
16
Overview of classified CPRs
Type of articles Number of articles
CPRs 114 (out of 383)
Original articles 86
Review articles 19
Systematic reviews 9

Age groups Older adults (64), adults (82), adolescents (11), children (7), infants (4), not specified (23)
Setting 32 (28) from primary care with the remaining set in Emergency department, in-patients, nursing home, clinical trials data
17
Preliminary results Original articles (n86)
ICPC2 Clinical area Level 4 Level 3 Level 2 Level 1
K Cardiovascular 17 8 4 7
R Respiratory 11 8 2 3
A General and unspecified 9 10 3 -
U Urological 4 4 - -
T Endocrine/Metabolic and Nutritional 4 3 2 -
L Musculoskeletal 4 2 1 3
D Digestive 4 1 1 1
N Neurological 3 2 - -
- Process codes 3 3 - -
S Skin 1 - - 1
B Blood and Immune 1 - - -
18
SECTION 2

• Methodological Quality assessment of CPR register

19
Health Research Reporting
20
Health Research Reporting

• International initiative Equator set up in 2008
• Aim to provide resources education enabling
  improved reporting and monitor progress
• Develop network of reporting guidelines e.g.
• CONSORT- RCTs
• STARD- Diagnostic Accuracy studies
• STROBE- Observational Studies
• PRISMA- Systematic reviews

21
Methodological Quality assessment of CPRs

• To date Equator have not developed a reporting
  guideline for CPR Quality assessment
• Wasson et al, 1985, Laupacis et al, 1997, Stiell
  et al 1999,McGinn et al, 2000, Cook et al, 2010
• Different quality markers for derivation and
  validation CPR studies
• Consensus to use McGinn quality checklist due to
  content and utility

22
Results-preliminary

• Quality assessment of a small representative
  sample of CPR register assessed to date
• - 2008 as an example

23
Results

• 2008
• Derivation CPRs n18
• Validation CPRs n20

24
ResultsCPRs (derivation), 2008, n18
25
ResultsCPRs (validation), 2008, n20
26
SECTION 3

• Implementation strategies

27
Implementation

• 1.Register of CPRs Cochrane Primary Health Care
  Field
• 2. Clinical Decision Support Systems (CDSSs)

28
Register of CPRs Cochrane Primary Health Care
Field

• Implement International Register of CPRs that is
• Publicly available, web based, user friendly
• Searchable across clinical domains
• Maintained and updated

29
Registry
1
2
3
4
5
6
30
Implementation

• 1.Register of CPRs Cochrane Primary Health Care
  Field
• 2. Clinical Decision Support Systems (CDSSs)

31
Clinical Decision Support Systems
32
Clinical Decision Support Systems

• Clinical Decision Support Systems (CDSSs)
• Systems that are designed to improve clinical
  decision making
• Key points
• Integrated with the electronic patient record
• Available at the point of care
• Computerised knowledge base
• Provide patient-specific content

33
Completing evidence base/CPR register
Register
Systematic reviews