If You Can Measure It, You Can Manage It The Role of Key Indicators Shawn OConnor Ontario Tobacco Re

1
If You Can Measure It,You Can Manage ItThe
Role of Key IndicatorsShawn OConnorOntario
Tobacco Research UnitCTCRIs National Advisory
Group on Monitoring Evaluation

• CDPAC 2nd National Conference
• Tuesday, November 7

2
(No Transcript)
3
Purpose of NAGME Report

• Identify core outcomes/indicators for tobacco
  control
• Improve robustness of provincial/ territorial
  evaluations (pan-Canadian standards)
• Provide a tool for F/P/T stakeholders
• decision makers, mangers, evaluators,
  researchers, etc.

4
IndicatorsCritical for Monitoring

• Tracking progress
• Evaluation of public health efforts
• To find at need populations
• To discover what is changing (propose new
  initiatives required)
• For basic research determinants of smoking and
  effects of smoking

5
Indicators as a Tool

• Managing up and down
• Managing resources (evidenced-based decision
  making)
• Managing expectations
• Issue management

6
Developing Key Indicators
7
Expertise
8
National Advisory Group on Monitoring and
Evaluation Current Members Allison McKinnon Ann
Royer Bob Walsh Candace Nykiforuk Cheryl Moyer
Chris Lovato Louise Guyon Murray
Kaiserman Robert Sparks Roy Cameron Shawn
OConnor Steve Manske Tom Stephens Past
Members Dexter Harvey, Marilyn Boake, Marino
Francispillai, Sharon Campbell Staff Oonagh
Maley, Robin Reece, Tobin Copley
9
Expertise

• Ontario Tobacco Research Unit
• 12 years experience monitoring the OTS
• OTS Logic Models (Shawn OConnor, Tom Stephens,
  Harriet Robbins, Steven Savvaidis, Melody
  Roberts)
• Other Jurisdictions
• CDC
• California Tobacco Control Program

10
Data Elements
11
Outcome Indicators
An outcome is the results of an
activity Centers for Disease Control and
Prevention, 2005 An outcome indicator is a
specific, observable, and measurable
characteristic or change that will represent
achievement of the outcome United Way of
America, 1996
12
Logic Models
Prevention
Cessation
Protection
13
Steps

• Identify/assess existing indictors data
• Identify possible indicators
• Resources needed
• Strength of evaluation evidence
• Utility
• Face validity
• Accepted practice

14
Steps (contd)

• Technical Documentation
• Patterns by person, place, and time
• Definitional (x/y)
• Methodological (data source, eval design)
• Data quality
• Data limitations
• Identify Data Gaps
• Not accessible secure access
• No data source evaluate options

15
Lessons Learned

• View revision as inevitable and desirable
• Include indicator even if no current measure
• Keep working groups small (4 to 5 experts)
  (tobacco) industry not invited
• Keep evaluation component of program at arms
  length (independent)
• Authoritative sources

16
How many indicators?

• 120 (CDC)
• 100 (OTRU)
• 50 (NAGME)
• 25
• 1 (MHP)

17
Indicators as a Tool

• Managing up and down
• Managing resources (evidenced-based decision
  making)
• Managing expectations
• Issue management

18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
To order free copies (max. 5) contact Canadian
Tobacco Control Research Initiative (CTCRI) 10
Alcorn Avenue, Suite 200 Toronto, Ontario M4V
3B1 T 416.934.5968 F 416.961.4189 E
info_at_ctcri.ca This publication is available
online at www.ctcri.ca