Involving Consumers in Developing and Implementing Systematic Reviews

1
Involving Consumers in Developing and
Implementing Systematic Reviews

• Carol Sakala, PhD, MSPH
• Director of Programs
• Childbirth Connection
• Standards for Systematic Reviews of Clinical
  Effectiveness Research
• Institute of Medicine Workshop
• January 14, 2010

2
Background Pathbreaking 1989 Pregnancy
Childbirth Resources

• Effective Care in Pregnancy and Childbirth
  (Chalmers, Enkin, Keirse)
• probably the most important book in obstetrics
  to appear this century and its value to the
  profession will be profound and long lasting
• British Journal of Obstetrics and
  Gynecology 199097967-73.
• A Guide to Effective Care in Pregnancy and
  Childbirth (Enkin, Keirse, Chalmers
• Oxford Database of Perinatal Trials (Chalmers)
• Inspired establishment of Cochrane Collaboration
  to carry out similar work in all fields of health
  and medicine

3
Background Response to Lack of Uptake in U.S.
Decade Later

• Childbirth Connection (then known as Maternity
  Center Association) begins long-term national
  program to promote evidence-based maternity care,
  1999-
• at core translate best evidence, especially
  systematic reviews, and selectively commission
  and carry them out
• audiences health professionals, childbearing
  women, policy makers, journalists
• continues work beginning in 1918 to improve
  maternity care quality focusing on needs and
  interests of childbearing women and families
• Mission improve the quality of maternity care
  through research, education, advocacy and policy

4
Role of Consumer in SR Process

• Complement professionals and add value as team
  members in all phases
• prioritizing topics and questions
• planning reviews/developing protocols
• participating on author teams that develop
  reviews
• commenting on draft protocols and reviews
  alongside peer referees
• accessing completed reviews
• participating in evidence-informed decision
  making
• contributing to other translational activities
• periodically updating reviews

5
Who Should Be Considered a Consumer?

• Limited role for randomly selected member of the
  public
• Potential for substantive contributions from
  those who
• are experienced with the condition, and perhaps
  intervention, in question
• have been trained and supported to participate in
  clinical effectiveness activities
• are experienced in participating in clinical
  effectiveness activities
• are active in consumer or patient support and
  advocacy organizations
• are on the professional staff of organizations
  devoted to research, education, advocacy and
  policy on behalf of consumers

6
Who Should Be Considered a Consumer?

• Diverse complementary perspectives and
  contributions suggest avoid single consumer
  representative when possible (just as we benefit
  from diverse professional perspectives)
• Consumers drawn to participate in clinical
  effectiveness activities often sophisticated,
  self-selecting for skills and experience
• resourceful users of Pub Med and other online
  resources
• in active communication with others who share
  interests
• basic or greater critical appraisal skills
• continuous environmental scanning

7
Lessons from Existing Models

• Cochrane Pregnancy and Childbirth Group Consumer
  Panel to referee draft protocols, systematic
  reviews and review updates (12 years experience)
• Consumers often identify issues that
  professionals may not raise, e.g.,
• outcomes harms, longer-term effects,
  psychosocial impacts, experience of undergoing
  intervention
• background sections more precise and
  informative sound evidence
• research priorities prevention, less invasive
  interventions, validity of accepted but poorly
  assessed indications, whether or when to use
  interventions
• language clear, sensitive
• outsider insights role of co-interventions,
  limitations of RCTs

8
Lessons from Existing Models

• Consumer Panel evaluation review authors
  generally valued consumer contributions and
  review group editors were very enthusiastic
• Consumers often value opportunity to be at table,
  participate diligently
• Review group editor to independent evaluator
• I think I can speak for all the editors on
  this because its come up at every single one of
  our editorial meetings, that the consumer input
  in general is fantastic. Its the highest quality
  we get. Its certainly the most detailed and
  thoughtful.
• Review author to independent evaluator
• It was very long when it came. It took me
  awhile to get over the heart-sink. I mean, it was
  enormously valuable.

9
Lessons from Existing Models

• Factors contributing to success of Consumer Panel
  model
• leadership strong support from administrative
  office and review group editors
• established structure for consumer involvement,
  e.g., procedures, forms, informational documents,
  regional coordinators
• paid coordinator
• coordinator with strong advocacy and
  interpersonal skills
• coordinator consumer-professional liaison role
• accessible collated feedback with summary of key
  points helps busy professionals

10
Lessons from Existing Models

• Consumer member of Cochrane Review author team
• with 3 experienced, sensitive, respected
  reviewers and researchers
• division of labor study assessment, data
  abstraction/analysis relevance to low-income
  settings methodology consumer concerns
• resulted in strong, gratifying, highly accessed
  review
• with change in funding opportunities, Consumer
  Panel coordinator received support to participate
  in series of reviews, published this month and
  incorporating many of the common consumer concerns

11
Training and Support for Consumer Participation

• Just as professional participants draw on
  education, experience and institutional support,
  so are training and support beneficial for
  consumers
• Consumers likely to be more effective when they
• are conversant in research language and concepts
• have fundamental critical appraisal skills
• understand systematic review procedures
• have confidence to engage respectfully with
  professionals

12
Training and Support for Consumer Participation

• Support Project LEAD model from National Breast
  Cancer Coalition
• Based on current involvement in systematic
  reviews, guidelines (Guidelines International
  Network) and performance measures (National
  Quality Forum)
• Train and support consumers to work in a specific
  clinical domain (e.g., breast cancer) across
  continuum of clinical effectiveness activities
  (e.g., primary research, SRs, guidelines,
  performance measures, decision tools)
• content and quality depend on upstream sources
• basic biology applies across continuum
• key personnel/relationships, organizations and
  agencies involved across continuum

13
Move Toward Standardized Consumer Role

• Standardizing consumer role in systematic reviews
  would foster consistent involvement and assist
  those with limited experience
• Begin with general principles, e.g.,
• involving consumers in all phases of systematic
  reviews
• achieving a balance of perspectives and
  experiences
• avoiding and minimizing conflicts of interest
• endorsing but not requiring sources of training
  and support
• compensating consumers for costs of participation
• acknowledging participation of those involved
• providing those involved with final products
• Encourage evaluation (evidence!) and refinement

14
Welcome Standardized Grading Scheme

• Would reduce confusion for consumers and
  professionals, especially if consistent with
  evolving international standard, such as GRADE
• Would help align all stakeholders to foster
  national quality improvement aims
• Would provide basis for national educational
  programs
• important educational role for advocacy and
  professional organizations
• aids consumer interest in knowing all or most
  benefits/harms
• provides support for developers of consumer
  educational materials
• Consider also a standardized format Plain
  Language Summary

15
Other Consumer Viewpoint Recommendations

• Address strong consumer interest in understanding
  harms, a shortcoming of many SRs, through focused
  attention to harms in report, including use of
  most appropriate study designs
• Give special attention to transparency and
  conflict of interest issues to foster integrity
  of results
• Develop publicly available bibliographic
  databases of SRs, by clinical area make good use
  of available SRs (e.g., estimated 2,000-3,000 for
  maternity care alone, but dispersed, so missed
  opportunities)
• Support access to good sources of information
  about consumer views, preferences, experiences
  (ex. national Listening to Mothers surveys)
• Recognize interrelationships across continuum of
  clinical effectiveness activities and foster
  coherent integration across the full continuum