Using the Logic Model to Develop Evaluation Research Design and Measures

1
Using the Logic Model to Develop Evaluation
Research Design and Measures
2
Program Planning and Evaluation

• Solid program planning is vital to a well
  designed evaluation.
• Community partners (stakeholders) need to have a
  meaningful way to provide input to program
  planning.
• Extension staff must be able to communicate
  outcomes to funders and stakeholders.

3
Logic Models

• Well developed, easy to use tool for planning
  educational programs, using stake holder input.
• Provide a common language
• Simple, graphic way to communicate program theory
• Provides planners with a road map asking them
  to determine where they want to end up (goals)
  and then chart their course of action

4
Outcomes based

• Outcomes must be clearly written and the
  relationship between learning, action and
  long-term outcomes must be logical.
• Forces planners to think about and articulate
  their goals and objectives.
• Is an easy model for stakeholders to understand
  and provide feedback on.

5
Generic logic model
Strategy
Impact
C O N T E X T

A diagram of the theory of how a program is
supposed to work A graphic depiction of
relationships between activities and results
Source University of Wisconsin
Extension http//www.uwex.edu/ces/pdande/evaluatio
n/evallogicmodel.html
6
University of New Hampshire Cooperative Extension

• Began staff development efforts in 2001
• Provided in-services, mouse pads, web support and
  even one-on-one support to staff
• Began presenting the model to advisory committees
• Some resistance from staff, but many liked the
  model
• More staff are using it now to obtain grant
  funding
• Suggested model for communities proposing CYFAR
  New Communities projects

7
Benefits of logic modeling to partnerships

• Provides framework to discuss and articulate
  joint work
• Keeps end outcome upfront and center
• Helps facilitate conversation with focus on
  agreed upon goal that might not happened
  otherwise
• Utilizes local needs assessments and works with
  community partners to inform Logic Model
  development
• Separates indicators of achievement - papers
  published, and type of experiments completed
  from theory of change. Indicators of achievement
  are part of evaluation plan for the partnership.

8
Case Examples

• Seacoast Youth Leadership Program

• Hillsboro After School Program

9
Seacoast Youth Leadership Program

• This project will evaluate the efficacy of the
  youth diversion programs ability to achieve
  improvements in the following objectives
• Leadership
• Future Aspirations/Goal Setting
• Social Responsibility
• Decision Making/Social Skills
• Social Connectedness
• Media Literacy

10
SYLP Logic Model
11
SYLP Logic Model, cont
12
SYLP Logic Model, cont
13
SYLP Logic Model, cont
14
SYLP Logic Model, cont
15
SYLP Logic Model, cont
16
Hillsboro After school Program

• This project will evaluate the efficacy of the
  after school programs ability to achieve
  improvements in the following objectives
• Academic Performance
• Future Aspirations
• Social Connectedness
• Social Responsibility
• Sense of Self
• Parenting Efficacy
• Social Skills/Decision Making
• Community Involvement with Youth/School

17
Hillsboro After School Future Aspirations
18
Future Aspirations, cont
19
Future Aspirations, cont
20
Future Aspirations, cont
21
Future Aspirations, cont
22
Future Aspirations, cont
23
Hillsboro After School Community Involvement
24
Hillsboro After School Community Involvement,
cont
25
Hillsboro After School Community Involvement,
cont
26
Hillsboro After School Community Involvement,
cont
27
Hillsboro After School Community Involvement,
cont
28
Hillsboro After School Community Involvement,
cont
29
Seacoast Youth Leadership Program

• This project will evaluate the efficacy of the
  youth diversion programs ability to achieve
  improvements in the following objectives
• Leadership
• Future Aspirations/Goal Setting
• Social Responsibility
• Decision Making/Social Skills
• Social Connectedness
• Media Literacy

30
Hillsboro After school Program

• This project will evaluate the efficacy of the
  after school programs ability to achieve
  improvements in the following objectives
• Academic Performance
• Future Aspirations
• Social Connectedness
• Social Responsibility
• Sense of Self
• Parenting Efficacy
• Social Skills/Decision Making
• Community Involvement with Youth/School

31
Collapsing extensive outcome goals and objectives
into manageable and meaningful evaluation
research conceptual categories for design and
measurement development
32
Identifying existing measures for conceptual
categories

• LEADERSHIP
• INDEPENDENCE (WRIGHTSMAN, 1964,1974)
• MASTERY (PEARLIN, 1981)
• SELF-ESTEEM (ROSENBERG, 1965)
• FUTURE ASPIRATIONS (PEER LEADER SURVEY DYS)
• SOCIAL CONNECTEDNESS
• SOCIAL SUPPORT (VAUX)
• ATTACHMENT TO PARENTS SCHOOL
• (RESNICK, 1997 ADD HEALTH SURVEY)

33
Collaboration with Extension staff to determine
appropriate evaluation design and measurement

• EVALUATION AS A PROCESS
• COMMUNICATION IS KEY

34
Utilization of triangulation of methods to
provide depth and breadth to the evaluation
research process
35
Contact InformationScott R. Meyer, MSW, Ph.D
- [phone] scottm_at_mail.plymouth.eduStepha
nie Halter,M.A., Research Associate, (603)
536-5725 stephanie.halter_at_unh.eduRick Alleva,
Extension Educator, [phone]
rick.alleva_at_unh.eduDan Riedy, Extension
Educator, [phone] dan.reidy_at_unh.eduPenny
Turner, Extension Educator, [phone],
penny.turner_at_unh.eduLisa Townson, Extension
Specialist, [phone] lisa.townson_at_unh.edu