Title: Evaluating Community Interventions
1Evaluating Community Interventions
- Sarah Levin Martin, PhD
- CDCs Physical Activity and Health Branch
- SLMartin_at_cdc.gov
2Program Evaluation is
- The systematic collection of information about
the activities, characteristics, and outcomes of
programs to make judgments about the program,
improve program effectiveness, and/or inform
decisions about future program development - (Michael Quinn Patton)
3What is
- Formative assessment
- Process evaluation
- Impact / Outcome evaluation
4Before, During, and After
5Intervention Stage of Development (and
corresponding evaluation)
Planning stage (formative assessment) - needs
and resource assessment Implementation stage
(process / impact) - process measures and early
outcomes Maintenance stage (process /
outcome) - process measures and intermediate and
long-term outcomes
6Things to Consider
- Guide to Community Preventive Services
- Socio Ecological Model
- National Objectives
7Evaluation Basics (1)
- Evaluation questions
- Indicators
- Data sources
- Performance measures
8Evaluation Grid- Process Eval
9Evaluation Grid Outcome Eval
10Key Evaluation Questions
- Implementation (Process)
- Documents extent to which the program is
operating as expected - Effectiveness (Outcome)
- Examines the extent to which a program meets its
defined short-, mid-, or long-term
outcomes/effects - Efficiency (Input-Output relationship)
- Introduces the element of cost
- Causal Attribution
- Examines if progress on any of above is due to
your program
11Indicators (Physical Activity)
12Data Sources
- Existing Data
- Agency records
- Census
- Prospective Data
- Surveys
- Key Informants
- Focus Groups
- Community Forums
- Ethnographic approaches (observation)
13More on Data
- Quantity
- Quality
- Triangulation
14Re-Cap of Basics (1) applied to SEM
15Evaluation Basics (2)
- Evaluation standards
- Evaluation designs
16Evaluation Standards
- Utility Who needs the info from this evaluation
and what info do they need? - Feasibility How much money, time, and effort
can we put into this? - Accuracy What design and methods will lead to
accurate information? - Propriety Who needs to be involved in the
evaluation to be ethical?
17Evaluation Design Continuum
Non Experimental
Quasi Experimental
Experimental
X O O
X O (R) O (R)
X O
O X O (R) O O (R)
O X O O O
O X O
O O O X O O O O O O O
O O
O O O X O O O(R) O O O
O O O(R)
18Tools
- Warning There are many more caveats than tools!
19Resources Physical Activity
- Physical Activity Evaluation Handbook
http//www.cdc.gov/nccdphp/dnpa/physical/handbook/
pdf/handbook.pdf - Previous Day Physical Activity Recall
http//cparg.sph.sc.edu/ tools.asp -
- International Physical Activity Questionnaire
- http//www.ipaq.ki.se/
- Jim Salliss website
- http//www-rohan.sdsu.edu/faculty/
sallis/measures.html - A Collection of Physical Activity Questionnaires
- MSSE, 29(6), June 1997
NOTE These are just a sampling of readily
available tools CDC does not endorse
these tools other than HHS docs.
20Resources Planning and Evaluation
- CDC Evaluation Working Group http//www.cdc.gov/ev
al/index.htm - Harvard Family Research Project
http//www.gse.harvard.edu/hfrp/ - Kellogg Foundation Logic Model Development Guide
www.wkkf.org - University of Wisconsin-Extension
http//www1.uwex.edu/ces/lmcourse
21Resources Other
- SMART BRFSS
- http//apps.nccd.cdc.gov/brfss-smart/index.asp
-
- Tools for Measuring Environmental Change
- http//socialmarketing-nutrition.ucdavis.edu/Tools
/somarktoolsEnv.php
NOTE These are just a sampling of readily
available tools CDC does not endorse
these tools other than HHS docs.