Evaluation of Public Awareness Programs

1
Evaluation of Public Awareness Programs

• Purpose and Methods

2
Why Evaluate?

• Evaluating the effectiveness of a public
  education program is important
• To determine if the program is worth repeating
  in the community or elsewhere
• To elicit feedback on how to improve the
  program and
• To provide rationale for financial support

3
Why Measure Outcomes?

• Reaffirm that you are on the right track
• Benefit your organization in its long range
  planning
• Justify existence
• Be accountable and cost-effective
• Take pride in accomplishment and
  quality

4
The Goal of Evaluation

• Strongest evidence of success demonstrating the
  program achieved desired changes
• If change cannot be shown
• Show that a public education campaign is
  successful in reaching its target audience
• Information clear enough for audience to
  understand what actions to take or behaviors to
  change
• Lesson persuasive enough to make people act

5
Evaluation Measures (1)

• Outreach
• Show that the percentage of the target group was
  reached
• Knowledge / Behavior
• Measure changes in knowledge by conducting pre-
  and post- education tests
• Conduct a random survey of the community before
  a campaign starts and repeat after it is completed

6
Evaluation Measures (2)

• Making Comparisons
• Compare changes in the community before and
  after the introduction of a program
• Compare community with other similar communities
  that did not have the same program
• It is possible to compare similar neighborhoods
  within a community

7
Evaluation Measures (3)

• Anecdotes
• Should be well documented, with testimonials
  specifically stating benefit of the program
• Several stories have more power than one
• Anecdotes in combination with statistics present
  make the strongest case of all

8
Evaluation Measures (4)

• Tailoring measures to your program
• Look at changes in knowledge, behavior or
  statistics that reflect the content of the
  messages being delivered
• Look for changes in specific population group
  that programs were aimed at

9
Designing an Evaluation (1)

• ...Even a thousand mile journey must begin with
  the first step
• The likelihood of reaching destination is
  enhanced if the first and subsequent steps take
  the traveler in the correct direction
• Wandering about here and there without a clear
  sense of purpose or direction consumes time,
  energy, and resources

10
Designing an Evaluation (2)

• Front end planning is necessary to designing and
  implementing an evaluation successfully
• At the same time that goals are being developed
  for the campaign, strategies for measuring and
  evaluating success should be identified

11
Elements of Outcome Evaluation
12
Elements of Outcome Evaluation
13
Levels of Outcome Measures
14
Indicators of Success (1)

• Community Outreach Program Postcards are mailed
  inviting eligible community members to receive
  free screening kit lab reading FU

• respondents to mailing each year
• Increase in respondents over time
• at higher risk for cancer identified
• Necessary follow-up achieved

15
Indicators of Success (2)

• Public awareness media campaign includes
• PSAs in newspapers radio TV
• Presentations to community groups
• Health fairs
• Printed materials disseminated in local clinics

• Patients who present for screenings report
  related sources of information
• Pre- and Post-tests of knowledge
• Level of community coverage by media messages

16
Outcome Evaluation of the Michigan Cancer
Consortium Initiative (MCCI)
17
MCC Priority Objective

• By 2004, increase to 50 percent the proportion of
  average-risk people in Michigan who have received
  appropriate colorectal cancer screening and
  received appropriate follow-up of abnormal
  screening results

18
MCC Call to Action 2002-03

• MCC organizations will participate in a
  statewide colorectal awareness network (CRAN)
• MCC organizations will disseminate resource
  materials to encourage regular colorectal cancer
  screening and follow-up

19
Any Appropriately Timed Colorectal Cancer
Screening by Population Group, MI 2001-02
MCC
20
Proportion of Cases Diagnosed atLocalized Stage,
1999
21
Total Mortality Rates by Cancer Site, Michigan
1985-2000
22
Concluding Remarks

• Make a commitment to evaluate your projects, use
  your findings, and share results with others
• Keep it simple and straightforward
• Get agreement on concrete measures of success
  that are specific and focused on your programs
• Identify appropriate data sources and methods to
  answer your questions and to measure progress of
  your programs

23
(No Transcript)