Using Evidence To Improve Public Health Infrastructure:

1
(No Transcript)
2

• Using Evidence To Improve Public Health
  Infrastructure
• Let the evidence guide our actions

January 7, 2004 Jonathan E. Fielding, M.D.,
M.P.H., M.B.A Director of Public Health and
Health Officer L.A. County Department of
Health Services Chair, CDC Task Force of
Community Preventive Services Professor of Public
Health and Pediatrics University of California,
Los Angeles
3
If we did not respect the evidence, we would have
very little leverage in our quest for truth

• Carl Sagan

4
Quality of the Evidence

• We hear about it often
• TV Networks
• Usually related to high profile trials
• Public Health evidence is different than legal
  evidence
• It is the available information on a particular
  question
• We want the best available evidence in making
  decisions

5
Decisions and Evidence

• Evidence takes many forms
• Opinion of leaders
• Opinion of experts
• Studies without controls
• Studies with controls
• Studies of variable quality in design and
  execution

6
Decisions and Evidence

• Our commitment
• Improve public health
• Health problems well defined
• Our job
• Make a difference through policies and programs
• Inaction is not an option
• Hard to identify best evidence to inform decision
  making

7
Decisions and Evidence

• Decisions on policies and programs are often made
  based on
• Personal experience
• What we learned in formal training
• What we heard at a conference
• What a funding agency required/ suggested
• What others are doing

8
Evidence and Public Health Decision Making

• Good news
• Strong evidence on the effect of many policies/
  programs aimed to improve public health
• Major efforts underway to assess the body of
  evidence for wide range of public health
  interventions

9
What works to improve the publics health?

• Bad news
• Many public health professionals are unaware of
  this evidence
• Some who are aware dont use it
• Many existing disease control programs have
  interventions with insufficient evidence while
  others use interventions with strong evidence of
  effectiveness
• Lack of use of effective interventions can
  adversely affect fulfilling mission and getting
  public support

10
How do we know what works in improving the health
of populations?

• Background
• Many community health improvement efforts have
  not achieved desired results
• Interventions often chosen based on opinions and
  personal preferences
• Evidence based medicine---Clinical Preventive
  Services Task Force mid 80s
• Evidence based population health --- Community
  Preventive Services Task Force mid 90s

11
Systematic Search for the Best Evidence

• U.S. Community Preventive Services Task Force
  Appointed by CDC Director in 1996
• Non-Federal independent task force of experts in
  multiple relevant disciplines
• Epidemiology
• Public Health Practice
• Behavioral Sciences
• Evidence based medicine/ public health
• Other relevant areas of expertise

12
Goals

• Conduct careful analytic reviews of acceptable
  evidence for population health interventions and
  make related recommendations
• Use peer reviewed literature
• Standard rules of evidence
• Standard rules for translating evidence into
  recommendations for interventions

13
Systematic Reviews of Public Health Interventions
are Useful

• Methods first developed by social scientists
  (e.g., Glass, 76)
• Distill and summarize large and diverse bodies of
  evidence
• Reduce errors and biases in interpretation
• Make assumptions explicit

14
Systematic Reviews Are Not

• Limited to randomized controlled trials
• Limited to healthcare interventions
• Restricted to a biomedical model of health
• - Petticrew, 2001

15
Task Force on Community Preventive Services
Members Members
Jonathan E. Fielding, Chair Patricia Dolan Mullen, Vice-chair Noreen M. Clark John M. Clymer Mindy T. Fullilove Alan Hinman George J. Isham Robert L. Johnson Garland Land Patricia A. Nolan Dennis E. Richling Barbara K. Rimer Steven Teutsch
Consultants Robert S. Lawrence J. Michael McGinnis Lloyd F. Novick Consultants Robert S. Lawrence J. Michael McGinnis Lloyd F. Novick
16
Who Is the Audience?

• People who plan, fund, or implement public health
  services and policies for communities and
  healthcare systems
• Public health departments
• Healthcare systems and providers
• Purchasers
• Government agencies
• Community organizations

17
Community Guide Topics
Environmental Influences Environmental Influences
Sociocultural Environment Sociocultural Environment
Physical Environment Physical Environment
Risk Behaviors Specific Conditions
Tobacco Use Alcohol Abuse/Misuse Other Substance Abuse Poor Nutrition Inadequate Physical Activity Unhealthy Sexual Behaviors Vaccine Preventable Disease Pregnancy Outcomes Violence Motor Vehicle Injuries Depression Cancer Diabetes Oral Health
18
Methods for Systematic Reviews of Effectiveness
Evaluations

• Develop conceptual framework
• Search for and retrieve evidence
• Rate quality of evidence
• Summarize evidence
• Translate strength of evidence into finding
• Strongly recommended
• Recommended
• Insufficient evidence

19
Logic Framework Vaccine Preventable Disease
20
Standardized Analysis Process

• Systematic review of literature
• Abstracting of relevant studies
• Grading of evidence
• Study design
• Execution
• Translating from quality of evidence to
  recommendations
• Economic analysis
• Other benefits and harms

21
How Does the Task Force Define Suitability of
Study Design?

• Greatest
• Prospective with concurrent comparison
• Moderate
• Multiple before-and-after measurements but no
  concurrent comparison OR
• Retrospective
• Least
• Single group before-and-after
• Cross-sectional

22
What Factors Determine Quality of Execution?

• Description of intervention and study population
• Sampling procedures
• Exposure and outcome measurements
• Approach to data analysis
• Interpretation of results
• Follow-up
• Confounding
• Other bias
• Other issues

23
How Does the Task Force Draw an Overall
Conclusion About the Strength of a Body of
Evidence?

• Number of studies
• Design suitability
• Quality of execution
• Consistency
• Effect size

24
Evidence of Effectiveness Quality of Execution Design Suitability Number of Studies Consistent Effect Size
Strong Strong Strong Strong Strong Strong
Good Greatest gt 2 Yes Sufficient
Good Greatest or Moderate gt 5 Yes Sufficient
Good or Fair Greatest gt 5 Yes Sufficient
Meet criteria for sufficient evidence Meet criteria for sufficient evidence Meet criteria for sufficient evidence Meet criteria for sufficient evidence Large
2. Sufficient 2. Sufficient 2. Sufficient 2. Sufficient 2. Sufficient 2. Sufficient
Good Greatest 1 -- Sufficient
Good or Fair Greatest or Moderate gt 3 Yes Sufficient
Good or Fair Greatest, Moderate or Least gt 5 Yes Sufficient
3. Insufficient Insufficient design or execution Insufficient design or execution Too few No Small
25
Physical ActivityReview of One Intervention

• Goal increase the amount of time students spend
  doing moderate or vigorous activity in PE class
  through curricular change
• Interventions reviewed included changing the
  activities taught (e.g., substituting soccer for
  softball) or modifying the rules of the game so
  that students are more active (e.g., in softball,
  have the entire team run the bases together when
  the batter makes a base hit). Many interventions
  also included health education.

26
School Curricular Interventions to Improve
Physical Fitness

• 14 acceptable studies in all students physical
  fitness improved.
• 5 studies measured activity levels during PE
  class all found increases in
• amount or percentage of time moderately/
  vigorously active and/or
• intensity level of physical activity during
  class.
• Median estimates--modifying school PE curricula
  as recommended will result in an 8 increase in
  aerobic fitness

27
School Curricula to Improve Physical Activity

• Modifying school P.E. curricula was effective
  across diverse racial, ethnic, and socioeconomic
  groups, among boys and girls, elementary- and
  high-school students, and in urban and rural
  settings.
• In a separate literature review, having students
  attend school PE classes was not found to harm
  academic performance.
• Economic analysis pending.

28
Different Ways to Improve Public Health

• Focus on diseases/ injuries e.g. sexually
  transmitted diseases, intentional injury,
  diabetes
• Focus on risk factors e.g. tobacco use,
  nutrition, physical activity
• Focus on underlying factors that impact multiple
  dimensions of health e.g. poverty, social
  isolation

29
Underlying Health Determinants
30
Impact of Early Childhood Home Visitation Programs

• Program can prevent child maltreatment in
  high-risk families.
• In studies reviewed, home visiting resulted in a
  40 reduction in child maltreatment episodes.
• Longer duration programs produce larger effects
  programs of less than 2 years duration did not
  appear to be effective.
• Professional home visitors may be more effective
  than trained paraprofessionals
• but longer-duration programs with trained
  paraprofessionals can also be effective.

31
Impact of Early Childhood Home Visitation Programs

• All programs reviewed were directed at families
  considered to be at high risk of child
  maltreatment, (e.g., single or young mothers,
  low-income households, families with low birth
  weight infants).
• Other benefits
• Health benefits for premature, low birth weight
  infants and for disabled and chronically ill
  children
• Improved maternal educational attainment, reduced
  public support, improved child educational
  performance, reduction in drug use and contact
  with juvenile justice etc.

32
Tenant-based Rental Voucher Programs

• Background
• Tenant-based vouchers allow very low income
  families to rent safe, decent, and affordable
  privately owned housing in neighborhoods of their
  choice.
• Rental voucher programs, known as housing
  mobility programs, work with landlords and
  tenants to find rental property outside of
  neighborhoods of concentrated poverty and
  relocate families to neighborhoods of greater
  prosperity.

33
Tenant-based Rental Voucher Programs

• Findings from the Systematic Review
• 6 studies rental voucher programs resulted in
  decreases in victimization of tenants or their
  property
• Families enrolled in rental voucher programs who
  moved to better areas were
• 6 less likely to have a household member
  victimized
• 15 less likely to experience neighborhood social
  disorder.
• Changes in victimization in both urban and
  suburban settings.
• Other benefits substantially reduced symptoms of
  maternal depression, boys behavioral problems in
  school, and childhood illnesses and accidents
  requiring medical attention.

34
General Comments on Evidence Based Reviews

• More evidence than sometimes expected, however
• Insufficient evidence common outcome
• Very resource intensive process
• Quality of studies vary widely
• Economic data still uncommon
• Important to consider harms even though uncommon

35
TASK FORCE REVIEWS AND RECOMMENDATIONS

• www.thecommunityguide.org
• Results of all reviews to date
• Frequent updating
• Downloadable Slide Sets

36
How Evidence Can Improve Public Health
Infrastructure

• Explore evidence underlying options to reach each
  public health goal
• e.g. smoking control, reducing disparities in
  infant mortality, increasing physical activity,
  increasing immunization rates
• Comprehensively review the best sources of
  evidence reviews
• Community Guide (best source when topic of
  interest has been covered)
• Recent review articles in peer reviewed journals
• Other meta-analyses funded by responsible federal
  agencies
• Compare results and recommendations of different
  sources

37
How Evidence Can Improve Public Health
Infrastructure

• Use evidence to decide among possible
  interventions
• How does each possible intervention suit the
  problem and the population?
• E.g. was it tried on particular racial/ ethnic/
  age/ gender groups?
• Is there reason to belief it would not be as
  effective for some of these on whom it was not
  tried?
• Is the problem now similar to what it was when
  the major studies took place?

38
Deciding on Interventions

• Single versus multiple component interventions
• Single component interventions easier to develop,
  implement, control and assess, but
• Multi-component interventions usually more
  effective
• E.g. Tobacco control in California

39
Deciding on Interventions

• Consider both policies and programs
• Programs
• Greater control over all aspects
• Organizational unit has primary responsibility
  for design, implementation and outcomes
• Policies
• Control varies broad policies often made by
  elected officials
• Policies have potential for greater public health
  impact
• Credit needs to be shared e.g. LAUSD Nutrition
  Policies, increase in tobacco tax

40
Deciding on Interventions

• Consider effect size
• Median
• Consistency
• Consider breadth of target population
• Together effect size and target population define
  the overall population effect

41
How Evidence Can Improve Public Health
Infrastructure

• Use evidence to determine realistic goals by
  estimate effect size (i.e.. how much you move
  the needle!)

Relative percentage changes in exposure to
environmental tobacco smoke attributable to
workplace smoking bans and restrictions from
studies that qualified for inclusion in this
review (a and b in Study names refer to first
or second study by the same author in that year,
included in this review)
42
Deciding on Interventions

• What is the slope of the effect curve?
• Larger initial effects with significant
  recidivism
• Smaller initial effects with Increasing impact
  over time
• What is the time frame for observed health
  benefits?
• How long were the follow-up periods for the best
  studies?
• For equal benefit, shorter is better, but
• Long term benefit is primary interest

43
Deciding on Interventions

• What is the cost of the intervention?
• Personnel
• Dollars i.e. contracts
• Time to implement
• Likelihood of funding for sufficient period to
  get effect
• Potential for dedicated or incremental funding
• Opportunity cost (i.e. cost of not doing other
  things)

44
Deciding on Interventions

• Relative cost-effectiveness
• Cost effectiveness is dollar cost per health
  outcome (including clear intermediate outcomes)
• e.g. smoker prevented
• lead poisoning prevented
• STD cured
• drug treatment completed
• Note some interventions have multiple health
  benefits e.g. smoking affects CVD, some cancers,
  respiratory disease etc.

45
Deciding on Interventions

• Who else needs to be involved to be successful?
• Within public health
• Within personal health services
• Voluntary agencies
• Health care organizations
• Health plans
• Employers
• How difficult is it to get agreement on
• Roles and responsibilities
• Interventions?
• Time cost versus partnership benefit

46
How Evidence Can Improve Public Health
Infrastructure

• Use evidence to help decide on construction of
  intervention
• Interventions with same name can be very
  different
• Follow the design used in most successful
  interventions
• Talking to those who did the studies is very
  helpful in refining intervention

47
Using the Evidence

• Use evidence to frame objectives
• Use evidence to develop evaluation plan and
  related evaluation
• Approach
• Measures
• Data collection plan
• Develop internal evidence through performance
  measurement system
• Frequent monitoring essential

48
What more is needed?

• More research on public health practice for many
  interventions---insufficient evidence
• Increased funding for evidence based reviews
  using consistent methodologies
• More training on appropriate sources and uses of
  evidence
• in schools of public health and others training
  public health professionals
• in public health practice settings e.g. state and
  local health departments
• Political leaders and others who influence the
  decision making process to improve health

49
DHS Public Health Opportunities

• Become sophisticated user of evidence based
  information and recommendations
• Make use of best evidence key aspect of
  performance of program directors and key managers
• Contribute to the literature on what works in
  public health practice

50

• Presentation available on DHS Intranet
• Also see Evidence-Based Public Health, Ed. Ross
  C. Brownson, Elizabeth Baker, Terry L. Leet etc.
• Oxford University Press, 2003

51
(No Transcript)