Outcome Based Evaluation OBE

1
Outcome Based Evaluation (OBE)

• Summer Reading

2
Why evaluate?

• How does this promote your program?
• Data can prove need for or the success of a
  program or service.
• A quality evaluation can be used to prove to
  funders you can make good use of their support.

3
What can we evaluate?

• How Much We Do Statistics
• How Well We Do It Customer Satisfaction,
  Quality Benchmarks Standards
• What We Are Worth ROI, Cost Benefit
• Why We Matter Outcomes Measurement, Assessment
  of Impact

4
Why evaluate with OBE?

• A desire to expand services.
• A desire to improve management and prioritize
  services.
• A need to know the impact of services and
  programs.
• Being responsive to a grantor/funder.
• Prove the need for or success of a program.

5
What is OBE?

• Outcome based evaluation focuses on the impact on
  the recipient of a service or program.
• OBE is favored by the Institute of Museum and
  Library Services so it is important in relation
  to federal grants.

6
What is OBE?

• IMLS definition Your target populations
  changed or improved skills, attitudes, knowledge,
  behaviors, status, or life condition brought
  about as a result of experiencing your programs
  services/activities. Benefits or changes can
  occur either during the service period and/or
  after the service period is completed.

7
What is OBE?

• It is a systematic way to assess if a program or
  service met its goals.
• It does so by answering two questions
• How has this program/service made a difference?
• How are the lives of participants/recipients
  better?

8
OBE Definitions

• Inputs Resources a program uses to achieve
  program objectives.
• Activities What a program does with its inputs
  the services it provides to fulfill its
  mission.
• Outputs Products of a programs activities.

9
OBE Definitions

• Outcomes Benefits for participants during or
  after their involvement with a program (increased
  knowledge or skills changed attitudes, values
  behavior, condition, etc.)
• Outcome Indicators Specific information that
  track a programs impact. Measurable.

10
OBE Definitions

• Outcome Targets Measurable objectives for a
  programs level of achievement of its outcomes.
• Benchmarks Performance data that are used for
  comparative purposes.
• Logic Model A chart of the planning elements.

11
How does OBE impact programs?

• The focus is on
• Need A condition or deficit common to a group
  of individuals
• Solution A program that will change attitudes,
  skills, behavior, knowledge or condition
• Results The change or improvement that is
  expected (outcome)

12
How does OBE impact programs?

• Notice the planning flow
• Need Phase I, Gathering Information
• Solution
• Goal Phase II, Developing Goal Based on Need
• Program Phase III, Planning Activities Based on
  Goal
• Results Phase IV, Conducting Evaluation

13
How does OBE impact programs?

• You are looking for a program
• to meet the need,
• not a need
• to meet the program.

14
Why outcomes?

• It makes assumptions explicit
• It promotes collaboration with users/stakeholders
• Milestones can be identified for long range plans
• It focuses staff/stakeholders on the goal of the
  work, not just the work.

15
Why outcomes?

• It helps staff innovate and improve by consistent
  examination of results, thus strengthening
  existing services.
• It provides insight into why a program is used by
  some and not others.

16
Why outcomes?

• It is a vehicle for demonstrating the
  contributions a library makes to patrons and
  communities.
• It can focus funders attention on community
  needs.

17
Are there issues with outcomes?

• You must survey or test users, sometimes both
  before and after.
• Relevant data may be not available due to privacy
  concerns.
• If you cant locate patrons, long-term results
  are difficult.
• It may be difficult to isolate the librarys
  impact from the contributions of others.

18
How do we do OBE?

• So how do we prove the following question?
• We do what, for whom, for what benefit
  (outcome)?
• Hint Its all about the user.

19
How do we do OBE?

• Identify a community need
• Develop a goal to meet the need
• Design an objective to accomplish goal
• Identify inputs
• Plan implement activities
• Measure outputs
• Assess outcomes
• Did you meet the goal?

20
Community Need

• Children in the community need to maintain
  reading levels over the summer.
• Number of kids in that age
• Research on reading skills and summer break
• Area school reading scores

21
Goal

• The library will help maintain or improve
  childrens reading levels over the summer through
  Summer Reading programming.

22
Objectives

• Develop a Summer Reading program with marketing,
  evaluation, activities, etc.

23
Inputs

• Staff time
• Materials
• Performers
• Initial evaluation of children
• Funding
• Facilities

24
Activities

• Buy materials
• Hire performers
• Contact schools
• Put up marketing materials
• Host activities

25
Outputs

• Number of books read
• Number of children participating
• Number of events

26
Outcomes

• How many children maintained their reading level?
• Interview parents of participating children?
• Work with day care providers who participate to
  do a brief assessment of participating children?
• Note the reading level of books by participants
  at the beginning end.

27
Data Collection

• For each outcome you wish to achieve, identify
• Indicators
• Data Source
• Who it is applied to
• Data Intervals
• Target

28
Data Collection

• Indicators
• of participating children maintain reading
  level.
• of participating children improve reading
  level.
• of children who choose reading as a desirable
  activity

29
Data Collection

• Data Source
• Evaluations given to parents/day care providers.
• School reading test scores
• Survey of participants

30
Data Collection

• Who it is applied to
• Parents/Day Care Providers
• Collaborating teachers
• Participants

31
Data Collection

• Data Intervals
• At end of Summer Reading program
• Testing during previous and next school year
• At beginning and end of Summer Reading program

32
Data Collection

• Target
• 60 of participating children maintain reading
  level.
• 25 of participating children improve reading
  level.
• 80 of participating children choose reading as a
  desirable activity.

33
OBE Resources IMLS sites

• IMLS information on OBE
• www.imls.gov/applicants/obe.shtm
• OBE Purpose
• www.imls.gov/applicants/overview.shtm
• Webography
• www.imls.gov/applicants/learning.shtm
• Resources
• www.imls.gov/applicants/resources.shtm

34
OBE Resources - Books

• Demonstrating Results Using Outcome Measurement
  in Your Library / Rhea Joyce Rubin, American
  Library Association, 2006.
• Dynamic Youth Services through Outcome-Based
  Planning and Evaluation / Eliza Dresang, American
  Library Association, 2006.

35
OBE Resources - Classes

• Shaping Outcomes Making a Difference in Museums
  Libraries by Indiana State University
• shapingoutcomes.org/

36
Presentation Resources

• Demonstrating Results Using Outcome Measurement
  in Your Library / Rhea Joyce Rubin, American
  Library Association, 2006.
• Outcome-Based Evaluation / Robert L. Schalock,
  Plenum Press, 1995.
• The Evaluation and Measurement of Library
  Services / Joseph R. Matthews, Libraries
  Unlimited, 2007.
• Measuring for Results / Joseph R. Matthews,
  Libraries Unlimited, 2004.

37
Contact Information

• Karren Reish
• 517-241-0021
• kreish_at_michigan.gov