Creating Logic Models

1
Creating Logic Models

• Stephanie Baird Wilkerson, Ph.D.
• Carol Haden, Ed.D.

2
Overview of Logic Model Session

• What is a logic model?
• What needs does MRSEC EO address?
• Who do we reach through EO?
• What is our common work?
• How does our work fit together to achieve common
  outcomes and impacts?
• How can logic models support evaluation?

3
A logic model is

• A depiction of a program showing what the program
  will do and what it is to accomplish.
• A series of if-then relationships that, if
  implemented as intended, lead to the desired
  outcomes
• The core of program planning and evaluation

University of Wisconsin-Extension, Program
Development and Evaluation
4
Many people say a logic model is a road map
University of Wisconsin-Extension, Program
Development and Evaluation
5
Logic model may also be called

• Theory of change
• Program action
• Model of change
• Conceptual map
• Outcome map
• Program logic

University of Wisconsin-Extension, Program
Development and Evaluation
6
Theory of change

• A theory of change is a description of how and
  why a set of activities be they part of a
  highly focused program or a comprehensive
  initiative are expected to lead to early,
  intermediate, and long-term outcomes over a
  specified period.
• (Anderson, 2000)

University of Wisconsin-Extension, Program
Development and Evaluation
7
What logic model is not

• A theory
• Reality
• An evaluation model or method
• It is a framework for describing the
  relationships between investments, activities,
  and results.
• It provides a common approach for integrating
  planning, implementation, evaluation and
  reporting.

University of Wisconsin-Extension, Program
Development and Evaluation
8

• Logic models can be applied to a small
  program, a process (i.e. a team working
  together), a large, multi-component program, or
  even to an organization or business.

University of Wisconsin-Extension, Program
Development and Evaluation
9
Simple Logic Model
10
How will activities lead to desired outcomes? A
series of if-then relationships

Tutoring Program Example
IF then
IF then
IF then
IF then
IF then
We invest time and money
They will move to next grade level on time
We can provide tutoring 3 hrs/week for 1 school
year to 50 children
They will learn and improve their skills
They will get better grades
Students struggling academical-ly can be tutored

University of Wisconsin-Extension, Program
Development and Evaluation
11
Why Do We Need This Program?

• Identification of needs is a prerequisite to the
  design and operations of an effective program.
• Program activities should directly address needs
  of target population.

12
Everyday example
H U N G R Y
Feel better
Get food
Eat food
University of Wisconsin-Extension, Program
Development and Evaluation
13
Needs Identification

• Define the need
• Assess its extent
• Define and identify target populations
• Determine how the program can best serve the
  target population

14
INPUTS

• The resources you invest in a program
• Funding
• In-kind contributions
• Staff
• Time
• Materials equipment
• Partnerships

15
Target Population

• The audience you will impact through your
  centers activities

16
Activities

• The activities your center conducts to reach and
  impact target audiences
• Address a clearly defined need in your target
  population or community

17
Outputs

• Quantitative documentation of the extent of reach
  immediately resulting from activities
• of people reached
• of programs provided
• of products disseminated

18
Outputs vs.outcomes

• Example Number of patients discharged from
  state mental hospital is an output. Percentage
  of discharged who are capable of living
  independently is an outcome
• Not how many worms the bird feeds its young,
  but how well the fledgling flies
• (United Way of America, 1999)

University of Wisconsin-Extension, Program
Development and Evaluation
19
Outcomes

• Short-term
• Intermediate
• Long-term

20

• If you dont know where you are going, how are
  you gonna know when you get there?
• Yogi Berra

Where are you going? How will you get
there? What will show that youve arrived?
University of Wisconsin-Extension, Program
Development and Evaluation
21
Short-term Outcomes

• Reflect changes in learning and affect that are
  directly influenced most immediately by an
  activitys processes and services.
• Awareness
• Interest attitudes
• Knowledge skills
• Motivation aspirations

22
Intermediate Outcomes

• Reflect how target participants will move into
  action
• Behaviors
• Practices, applications
• Policies
• Decision making
• Institutionalization of activities

23
Long-term Outcomes

• Reflect the ultimate purpose of your centers
  activities in impacting your target audiences
• Distal outcome that is achieved if the short-term
  and intermediate outcomes are realized
• The activity or program typically has less direct
  influence on distal outcomes
• Reaches secondary audiences
• Refers to impacts down the road

24
Hierarchy of effects
Social Improvements
Source Bennett and Rockwell, 1995, Targeting
Outcomes of Programs
University of Wisconsin-Extension, Program
Development and Evaluation
25
Goal outcome definition
Goal represents a general, big-picture statement
of desired results. We find that it is useful
to think of goals as the answer to the question
What are issues that you would like the program
to address? (e.g., the goal of the program is to
address existing community laws and norms about
ATOD use) and outcomes as the answer to What
changes do you want to occur because of your
program? (e.g., the outcome of the program will
be to increase the number of community residents
who believe teenaged smoking is
dangerous). (Western CAPT)
University of Wisconsin-Extension, Program
Development and Evaluation
26
Outcomes vs. Impacts

• OUTCOMES are the major effects or results of a
  program, usually longer-term, and usually but not
  always intended.
• IMPACT is the term used to refer to the totality
  of outcomes or effects.
• From Scrivens posting to EVALTALK listserve
  (suggested new definitions for Evaluation
  Thesaurus)

27
SMART Outcomes

• Specific Who, what, when, where, which, why
• Measurable Concrete criteria for measuring
  progress
• Attainable Possess abilities, skills, resources
  to reach outcomes
• Realistic Represents work you are willing and
  able to do
• Timely Grounded in a timeframe

28
Assumptions

• Assumptions underlie much of what we do. It is
  often these underlying assumptions that hinder
  success or produce less-than-expected results.
  One benefit of logic modeling is that it helps us
  make our assumptions explicit.

University of Wisconsin-Extension, Program
Development and Evaluation
29
Assumptions

• The beliefs we have about the program, the
  participants, and how the program will work.
  Includes ideas about
• the problem or existing situation
• program operations
• expected outcomes and benefits
• the participants and how they learn, behave,
  their motivations
• resources
• staff
• external environment influences
• the knowledge base
• etc.

University of Wisconsin-Extension, Program
Development and Evaluation
30
Tend not be included in a logic model graphic

• Situational statement
• Priorities
• List of assumptions
• List of external factors
• Evaluation methods

University of Wisconsin-Extension, Program
Development and Evaluation
31
RET Example
32
RET Example (cont)
33
Dont forget the arrows

• Arrows and feedback loops show the links between
  inputs, outputs and outcomes
• Arrows depict the underlying causal connections

University of Wisconsin-Extension, Program
Development and Evaluation
34
Feedback loops and multi-dimensions
INPUTS
OUTPUTS
OUTCOMES
Program investments
Activities
Participation
Short
Medium
Long-term
What we invest
What we do
Who we reach
What results
University of Wisconsin-Extension, Program
Development and Evaluation
35
Logic model of a training workshop
Situation Funder requires grantees to include a
logic model in their funding request grantees
have limited understanding of logic models and
are unable to fulfill the funding requirement
OUTCOMES
-Participants will increase knowledge of logic
models -Participants will increase ability to
create a useful logic model of program
-Participants will Increase confidence in using
logic models
INPUTS
ACTIVITIES
Fulfill requirement of funder
Create meaningful logic models Use logic models
in own work

• 3 hour training
• Interactive activities
• Group work
• Practice
• Q and A

Trainer Funds Equipment Research base Training
curriculum
Grantees
Improved planning Improved evaluation
Accountable here
University of Wisconsin-Extension, Program
Development and Evaluation
36
Parent Education Program Logic model
SITUATION During a county needs assessment,
majority of parents reported that they were
having difficulty parenting and felt stressed as
a result
OUTCOMES
INPUTS
ACTIVITIES
Parents increase knowledge of child dev
Assess parent ed programs
Parents identify appropriate actions to take
Reduced stress
Staff
Parents of 3-10 year olds attend
Parents better understanding their own parenting
style
Improved child-parent relations
Design- deliver evidence-based program of 8
sessions
Money
Parents use effective parenting practices
Partners
Parents gain skills in new ways to parent
Research
Facilitate support groups
Parents gain confidence in their abilities
University of Wisconsin-Extension, Program
Development and Evaluation
37
A common problem is that activities and
strategies often do not lead to the desired
outcomes. Check your if-then statements and
ensure that they make sense and lead to the
outcomes you want to achieve. A logic model
makes the connections EXPLICIT.
I think you should be more explicit here in Step
Two.

University of Wisconsin-Extension, Program
Development and Evaluation
38
Check for Validity of Model

• Are all efforts and resources reflected in the
  model? Is anything missing?
• Is it feasible?
• Is the program logic sound? Will the parts add up
  to the whole? Are there gaps?
• Is it meaningful?
• Engage wider group of stakeholders in review

39
What does a logic model look like?

• Graphic display of boxes and arrows vertical or
  horizontal
• Relationships, linkages
• Any shape possible
• Circular, dynamic
• Cultural adaptations storyboards
• Level of detail
• Simple
• Complex
• Multiple models
• Multi-level programs
• Multi-component programs

University of Wisconsin-Extension, Program
Development and Evaluation
40
Common variations
UWEX logic model
Other common logic model used by United Way,
Center for Disease Control and others
Short
University of Wisconsin-Extension, Program
Development and Evaluation
41
Families of models or nested models

• Multiple models may be needed to describe and
  explain complex systems or initiatives.
• Bring coherence across an organization
• Multi-level a way to describe and link
  activities across an organization to depict
  varying levels such as national-state-county
  levels OR, institution-division-unit levels.
• Multi-component programs A series of models to
  depict various components (goals, sites, target
  populations) within a comprehensive initiative.

University of Wisconsin-Extension, Program
Development and Evaluation
42
Nested logic models families of logic models

• View from space big picture overall roadmap
• View from mountaintop more detail by component
  program, player, participant group
• View from ground level you are here

University of Wisconsin-Extension, Program
Development and Evaluation
43
Multi level - Logic models can be linked to
display consistency of purpose and strategy
across levels and show how parts work to achieve
organizational goals
NSF Broader Impacts
Macro level MRSEC EO
Cross-center Program Level (REU, RET, etc.)
Each logic model is built with reference to the
levels above and below, and in relation to the
organizations or programs mission.
Center Level
University of Wisconsin-Extension, Program
Development and Evaluation
44
Programs as systems within the community setting
Single organization
SHORT-TERM OUTCOMES
INPUTS
OUTPUTS
Org 1
Multi-Org partnership
MED-TERM OUTCOMES
Org 2
LONG-TERM OUTCOMES
Org 3
Community
Org 4
Adapted from Chapel, 2006
University of Wisconsin-Extension, Program
Development and Evaluation
45
Limitations

• Logic Model
• Represents intention, is not reality
• Focuses on expected outcomes
• Challenge of causal attribution
• Many factors influence process and outcomes
• Doesnt address Are we doing the right thing?

University of Wisconsin-Extension, Program
Development and Evaluation
46
How Is a Logic Model Useful?

• Helps differentiate between what we do and
  results
• Clarifies the results that are supposed to be
  achieved helps staff stay focused on the
  outcomes both project-level and organizational
• Keeps the big picture increases ability of staff
  to address and respond to audience needs
  strategically
• Establishes/revises/confirms that there is a
  valid theory of change underlying the results
  being pursued and the approaches being taken to
  achieve these results
• Brings staff together on the same page provides
  common language with which to discuss work

47
How Is a Logic Model Useful?

• Identifies gaps or inconsistencies among project
  activities, assumptions and goals strengthens
  work
• Provides a base for accountability for results
  and fostering ownership of accountability
  throughout the organization
• Guides planning and development facilitates
  decision making defines institutional
  development priorities
• Provides a starting point for monitoring and
  evaluation also iterative and should reflect
  program changes evaluation therefore, must
  follow suit.