Program Evaluation in Public Health

1

• Program Evaluation in Public Health
• Californias Efforts to
• Reduce Tobacco Use 1989-2005

David Hopkins Terry Pechacek
2
California in 1988
Population 28,400,000
Adult smoking prevalence 22.8
4.8 million adult smokers
3
A Funded Mandate

• Voters approved ballot initiative in November,
  1988
• Excise tax increase of 25 cents/pack
• Earmarked funding (20) for a statewide program
• What to do, how to do it, and how to evaluate it?

4
The Status of Tobacco Control, 1988

• Limited experience with effective
  population-based interventions
• Clinical interventions (low success rates,
  relapses were common)
• Price (published econometric studies)
• Mass media (Fairness Doctrine Campaign 1967-1970)

5
California Had Program Options

• A top-down program
• Interventions selected and implemented by the
  Tobacco Program statewide
• A program built on the results of smaller-scale
  demonstration projects
• Trials would help to determine the independent
  impact of intervention options

6
A Comprehensive Approach was Advocated by NCI
(and others)

• Funding of community coalitions
• Local emphasis and control
• Multiple channels of intervention
• Multiple targets of intervention
• A field test was needed

Standards for Comprehensive Smoking Prevention
and Control National Cancer Institute
7
Multiple Channels, Multiple Targets
Interventions
Target Goals Media Campaigns Smoke-free Policies Community Activities School-based Programs
Increase Cessation X (X) X (X)
Reduce Initiation X (X) X X
Reduce Exposure to ETS X X X (X)
8
The California Tobacco Prevention and Control
Efforts, 1990

• An excise tax (price increase of 23)
• Paid mass media campaign
• Funding for community organization and
  interventions (67 Local Lead Agencies)
• Funding for school-based programs
• Funding for intervention and treatment research

9
Challenges in Evaluation

• A Public Health Example

10
Evaluation was Built into the Mandate

• Some surveillance systems were in place
• BRFSS State cigarette tax receipts
• California added some more
• California Tobacco Surveys
• Programs (components) were evaluated through
  contracts (independent evaluators)
• A research program was funded within the
  University of California

11
Local Evaluation was Included

• Funding for local intervention and research
  projects came with strings
• 10 of budget to be spent on evaluation
• and with support
• Directory of experts for consultation or to
  conduct evaluations
• Database of instruments and information
• Annual conferences

12
Oversight was Established

• Appointed committee
• Annual review of surveillance and research
  results
• Advice and recommendations
• Periodic publications summarizing program
  progress and direction
• Guardians

13
Evaluations of the California Program 1990-2004

• Outcomes, Study Designs, and Program Results

14
Smoking Prevalence among Adults in California
Decreased 32.5 between 1988-2004
Percent
22.8
15.4
Definition Changed
Year
Source California DHS 2005
15
Consumption Decreased 55.6 in California
(compared to 32 in the rest of the US) 1988-2003
Packs / Person
Rest of US
141
California
45.8
Year
Source California DHS 2003
16
Comparison of Age-Adjusted Rates of Death From
Heart Disease California 1979-1998
Age-Adjusted Heart Disease Mortality per 100,000
Predicted US rates
California
Year
Source Fichtenberg and Glantz NEJM 2000
17
Some Interventions Have Been Evaluated
Californias Telephone Quitline
Prolonged Abstinence by Study Arm

• Design RCT
• Analysis
• Intention to treat
• Receipt of counseling
• Inter arm 72.1
• Comp arm 32.6

Study Arm N 12m Quit
Comp 1309 6.9
Inter 1973 9.1
Differences Plt0.001 by log-rank test
Source Zhu et al. NEJM 2002
18
Evaluation What Has Gone Well

• Provided dozens of publications influencing
  tobacco prevention and control efforts
• Documented the overall impact of a comprehensive
  tobacco control effort
• Documented the independent contributions of some
  components
• Helpline
• Smoke-free policies
• Contributed to Program survival

19
Evaluation What Has Not Gone Well

• Local program impact is still unclear
• Comparisons have been difficult
• Most evaluations have not been published
• The effectiveness of some interventions remains
  unclear
• School-based programs

20
Evaluation Adjustments

• Adopted more uniform surveillance tools
• Combined BRFSS and CTS survey results since 1993
• New questions provide new directions
• Smoke-free policies (work, home)

21
Californias Program Became the Model

• Other States adapted the comprehensive
  State-level approach to tobacco control
• Massachusetts
• Florida
• Arizona
• Oregon
• Californias experience contributed to the
  contents of CDCs Best Practices Guideline in 1998

22
Discussion
23
Additional Slides
24
Targeted Outcomes and Measurements
Targeted Goals
Outcome Measurement Options
Increasing Cessation
Population Consumption
Prevalence
Cessation (Smokers)
Reducing Initiation
Prevalence
Youth (students)
Reducing Exposure
Policies
Exposures
Home
Work
Home
Work
25
California Tobacco Control Section Funding,
2001-2002 ( 106.5 m )
Admin 1.7m (1.6)
Evaluation 6.3m (6)
Media 45.2 m (42)
Local Lead Agencies 17.4 m (16 )
Competitive Grants 35.7m (33.5)
26
Tobacco Control is Always Outspent by the Industry
Per Capita Expenditures ()
Source California DHS 2004
27
Creating an Environment to Reduce Tobacco Use
Contemplation
Smoker
Quit
Success
Relapse
Modified from Population-based Smoking
Cessation. NCI Monograph 12 2000
28
Creating an Environment to Promote Cessation
Smoker
Quit
Success
Advice
Treatments
Counseling
Modified from Population-based Smoking
Cessation. NCI Monograph 12 2000
29
Creating an Environment to Promote Cessation
Smoke-free Policies
Media Messages
Community Events
Worksite Events
Costs
Smoker
Quit
Success
Advice
Treatments
Counseling
Quitline
Modified from Population-based Smoking
Cessation. NCI Monograph 12 2000