Measurable Objectives

1
Measurable Objectives

• A Review Plan of Action
• Presentation to the JWB Childrens Services
  Council Board of Directors July 19, 2007
• Starr Silver, Ph.D.
• Director of Research and Evaluation

2
Performance Measures

• Definition
• Purpose Importance
• Their role in JWB performance management
• Limitations
• Implications of new approach

3
Performance Measures - Defined

• Objective, quantitative indicators of various
  aspects of performance of public programs
• Effectiveness
• Cost-effective
• Service quality

4
Performance Measurement

• Is a process
• Identify what must be measured
• Measure it and enter into data system (SAMIS)
• Use this information to assess how program(s) are
  performing

5
Why important?

• What gets measured, gets done
• Essential to knowing how well programs
  are performing
• Assists contract management function
• Helps communicate with Board public

6
Performance Measurement

• Performance measures are part of JWB performance
  management system
• ASSET
• Site visits
• Fiscal strength
• Adherence to program methodology
• Contracted service levels

7
Challenges of Performance Measurement

• Specifying
• Goals of program
• Intended population
• Intended outcomes
• Theory of change (logic model)
• Measuring outcomes using reliable and valid
  methods

8
Where We Have Been

• Measures of outcome varied in quality
• - Some validated, many not
• Self-report
• Satisfaction
• Very large number of measuring tools
• Resource demanding
• What does this measure?
• How do I interpret the numbers?
• Administratively unfeasible

9
Where We are Headed

• Standard measures across many programs
• Smaller of instruments/tools
• Goal Attainment Scaling
• Healthy Kids Resiliency Survey
• Leveraging systems data (school, Safe Ch)
• Independent verification of referrals
• Programs retain option to use supplementary
  measures for own use

10
Standard Measures

• Children Not Ready for School Ages Stages
  Qrre
• School Data
• Children Not Successful in School Goal Attainmt
  Scale School DJJ Data
• Child Maltreatment Safe
  Children Coaln
• Children Unsupervised Goal Attainmt Scale
  School DJJ Data
• Families at Risk Goal Attainmt Scale
• Families with Special Needs Goal Attainmt Scale
• Children Committing Crimes/ DJJ Data
• Status Offenders

11
These replace
12
Evaluation of selected programswithin portfolio

• Longitudinal studies of impact
• Independent assessment of satisfaction

13
Implications

• Agency staff trained on goal setting and goal
  measurement
• JWB staff provide hands-on follow-up
• Implement Healthy Kids Resiliency
• Baseline
• Transition year

14
Goal Attainment Scales

• Developed by Kiresuk and Sherman in 1968
• Measures achievement of treatment or intervention
  goals
• Can address different kinds of treatment issues
• Can accommodate different numbers of goals
• Produces a Goal Attainment Score to track progress

15
Establishing S.M.A.R.T Goals and Measuring
Outcomes
16
Characteristics of SMART Goals

• Specific
• Measurable
• Attainable
• Realistic
• Time bound

17
References

• Kiresuk, T.J. Sherman, R.E. (1968). Goal
  attainment scaling A general method for
  evaluating comprehensive community mental health
  programs. Community Mental Health Journal, 4(6),
  443-453.
• Poister, T.H. (2003). Measuring performance in
  public and nonprofit organizations. San
  Francisco Wiley Bass.