Measuring Organizational Performance in Regional Substance Abuse Prevention Programs

1
Measuring Organizational Performance in Regional
Substance Abuse Prevention Programs
2
Why Measure Performance?

• Link overall prevention strategy to key
  activities
• Promote activities that are logically related to
  goals and intended outcomes
• Resource allocation and focus
• Management tool
• Accountability to various publics
• Federal requirements (GPRA, PPGs)
• Usefulness in selling the viability and
  relevance of prevention in the overall system of
  care

3
What areas should be measured by an
organizational performance system?

• Capacity ability of regional prevention
  activities to address priority needs
• Effectiveness extent to which goals are
  attained (intended outcomes are reached)
• Accountability appropriate, ethical, and
  cost-efficient use of resources fidelity of
  program implementation

4
Capacity Goal Areas

• Overall capacity to deliver prevention services
  (scope)
• Capacity for evidence-based practice
• Capacity for collaboration and coalitions
• Capacity to facilitate environmental change

5
Effectiveness Goal Areas

• Use, risk perception, risk resilience factors,
  perceived attitudes, availability, perceived
  norms, etc.
• Evidence of change or goal attainment associated
  with prevention program participation
• Evidence of change or goal
  attainment in regional social
  indicators

6
Accountability Goal Areas

• Staffing and work patterns
• Service delivery/program implementation
• Responsiveness to community needs

7
The Logic of Measurement

• Norm-referenced comparisons contrasting current
  performance score with a normative standard,
  such as a group average, rank within a group, or
  median of a distribution
• Criterion-referenced comparisons contrasting
  current performance scores with a pre-established
  expectation or standard (e.g., goal statement,
  agreed upon level of performance, regulatory
  requirement, change in relation to past
  performance)

8
Performance measurement is useful to the extent
that it

• Accurately reflects performance
• Discriminates levels of performance that have
  meaning
• Suggests methods and strategies for
  improvement

9
Why Do Performance Measurement Systems Fail?

• Not clearly linked on a conceptual level with
  overall strategy
• Links to overall strategy not validated over time
• Poorly chosen performance targets
• Measuring incorrectly

10
Goal Attainment Scaling A Criterion-Based
Approach

• Begins with a goal statement of an expected
  outcome
• Based on historical, logical, program-centered
  data
• Anchoring benchmark
• Specifies a range of possible outcomes
• Creates an equal interval scale and metric
• Allows for data aggregation and
  organizational profiling
• Useful planning tool