Components of a Logic Model

1
As part of the first activity, please complete
the Pre-Test Worksheet and the Outcomes
Planning Survey and return it to an NCDDR staff
member.
2
RRTC Outcomes Planning and Reporting Workshop
From Activity to Outcome
Overview of the Shift in Focus from Activity
to Outcome

• April 27, 2005
• 800 a.m. - 1200 p.m. - Discovery II
• Holiday Inn Capitol
• 550 C St., SW, Washington, DC 20024

3
Overview of Shift from Activity to Outcome

• NIDRRs responsiveness to questions of worth
  (accountability)
• How does NIDRR make a difference?
• How was the purpose of the funding area advanced?
• Goal responses come from the basis of
• outcomes or changes produced
• not from conducting an activity
• Challenging development process

4
RRTC Outcomes Planning and Reporting Workshop
From Activity to OutcomeComponents of a Logic
ModelMark Linder

• April 27, 2005
• 800 a.m. - 1200 p.m. - Discovery II
• Holiday Inn Capitol
• 550 C St., SW, Washington, DC 20024

5
Components of a Logic Model

• I. Terminology
• Situation
• The goal of your NIDRR-sponsored project.
• Identifies the problem or priority your project
  is addressing and the benefit to specific
  audiences.

6
Components of a Logic Model

• Inputs
• The human and financial resources and systems
  needed to conduct a high quality,
  outcomes-oriented program or project.
• Types of resources staff expertise, volunteers,
  time, money, host institutional support,
  materials, equipment, technology, research
  results, previous accomplishments, etc.

7
Components of a Logic Model

• Activities
• The action steps, tasks, procedures, and services
  performed in conjunction with implementing a
  planned program of research and development,
  capacity-building, training and/or technical
  assistance and knowledge translation and
  dissemination to achieve objectives and
  demonstrate results.
• Project activities include the research,
  development, training, technical assistance,
  dissemination, utilization, and other activities
  specified in your NIDRR grant proposal.

8
Components of a Logic Model

• Outputs
• The direct results of program activities, such as
  the discoveries or findings, models, tools,
  devices, products, publications, events, and/or
  services produced for external audiences.
• Outputs may be expressed in quantitative or
  qualitative terms.
• Outputs are directly observable.
• Outputs are important indicators of productivity
  and are the essential building blocks of
  outcomes.
• Though outputs contribute to outcomes, not every
  output will have a corresponding outcome, nor
  should it.

9
Components of a Logic Model

• Target Systems
• The group of individuals, organizations, or other
  entities expected to be affected by a project.
  Target systems are specific, realistically
  measurable consumer audiences in which the group,
  the commonalities bonding the group, and the
  context within which the group exists are all
  defined within a project's scope of work.
• The specific members of the overall target
  audience identified as potential beneficiaries of
  your projects work and among whom data
  collection will occur.

10
Components of a Logic Model

• Outcomes
• The effects of RD and related activities and
  outputs and constitute changes or improvements in
  identified target systems.
• Outcomes are separated into 3 groups short-term,
  mid-term, and long-term.
• Outcome goals differ by the type of change or
  improvement intended (knowledge, action, etc.),
  the target system in which change is expected to
  occur, and the level of accountability for which
  grantees are responsible.
• Outcomes are produced in more than one target
  system and occur sequentially over time,
  resulting in an incremental chain of outcomes,
  starting with short-term and progressing to
  mid-term and eventually longer-term outcomes.

11
Components of a Logic Model

• Short-term Outcomes
• The anticipated or actual changes or improvements
  in the knowledge base and/or learning system
  within an identified target system.
• Short-term outcomes are under the direct
  influence of project activities.
• They represent the first level of change that
  must occur in order to bring about mid-term
  outcomes.
• Types of change learning, awareness, knowledge,
  attitudes, skills, opinions, aspirations, and
  motivations.

12
Components of a Logic Model

• Mid-term Outcomes
• The anticipated or actual changes or improvements
  that occur in part as a result of the use or
  adoption of program outputs.
• Mid-term outcomes involve changes in use and,
  thus, take longer to achieve than short-term
  outcomes.
• Unlike short-term outcomes which occur under the
  direct influence of program activities, there may
  be other causal factors contributing to the
  achievement of mid-term outcomes.
• Types of change actions, behaviors, practices,
  decisions, policies, system capacity, and social
  actions.

13
Components of a Logic Model

• Long-term Outcomes
• The desired end-results of an RD related
  program, constituting changes or improvements in
  the overall condition of a population or target
  system.
• Given their scope, long-term outcomes usually
  take many years to occur, and therefore, neither
  NIDRR nor its grantees are held directly
  accountable for producing this type of change.
• Types of change conditions, social contexts, and
  environmental characteristics.

14
Components of a Logic Model

• II. Activity, Output, or Outcome?
• Relevant instruments to assess longer term
  program outcomes in consumer-relevant settings.
• 50 of e-Learning practitioners within Federal
  Region VI participating in authoring tool
  tutorial sessions report increased awareness of
  strategies to design and produce accessible
  e-Learning.
• Develop presentation for technical conferences
  pertaining to e-Learning.
• Concise, targeted materials to help enhance
  consumer choice in rehabilitation program
  selection.
• Develop and validate usability and educational
  effectiveness of model course on CD-ROM.

15
Components of a Logic Model

• II. Activity, Output, or Outcome? (continued)
• 50 of e-Learning practitioners within Federal
  Region VI report increased development of
  accessible courses within 12 months of
  participation in technical assistance regarding
  application of e-Learning authoring tool.
• Conduct 3 separate focus groups of 10 AAC users
  each.
• 25 of the VR counselors in Illinois respond with
  80 accuracy to the RTC assessment of counseling
  skills administered at the end of RTC project
  year three.
• Conference presentations.
• Enhanced instrumentation and methods to assess
  community participation.

16
RRTC Outcomes Planning and Reporting Workshop
From Activity to Outcome
Steps in Developing Your Projects Logic Model
Joann Starks Connecting Dissemination
Activities with Projected Outcomes John
Westbrook

• April 27, 2005
• 800 a.m. - 1200 p.m. - Discovery II
• Holiday Inn Capitol
• 550 C St., SW, Washington, DC 20024

17
Steps in Developing Your Projects Logic Model
Answering questions about the following
project elements will help you develop your logic
model

• Situation
• Inputs
• Activities
• Outputs
• Target System(s)
• Outcomes
• Short-term outcomes
• Mid-term outcomes
• Long-term outcomes

18

SituationClearly describe the purpose
of your NIDRR-sponsored project. What
is the overall goal of your project?
Why is this important?
Steps in Developing Your Projects Logic Model

19

Situation (continued) What
specific problem or problems are you trying
to address with your center or project?
What are you trying to accomplish relative to
this problem?
Steps in Developing Your Projects Logic Model

20

Example of Situation Published Personal
Attendant Care (PAC) Quality Indicators
(QIs) are not designed for low-literate,
multicultural populations. (RRTC on
Measuring Rehabilitation Outcomes and
Effectiveness)
Steps in Developing Your Projects Logic Model

21

InputsWhat resources are available
to your NIDRR-sponsored project to carry out the
scope of work? What human, financial,
or other resources are available to
apply to the project? What supports do
you have from NIDRR, other contributing
agencies, and from your host
organization?
Steps in Developing Your Projects Logic Model

22

Inputs (continued) What resources
and planning and management infrastructures do
you need in order to conduct high-quality
activities, produce outputs, and demonstrate
progress toward completing anticipated
outcomes? Do you have sufficient resources
and infrastructures in place to meet your
goals?
Steps in Developing Your Projects Logic Model

23

Examples of Inputs Previous research
findings on the significant inequity in
allocation of transportation resources between
urban and rural areas.
(RTC on Disability in Rural
Communities) The data resources of the
StatsRRTC. (RRTC
on Employment Policy)
Steps in Developing Your Projects Logic Model

24
Steps in Developing Your Projects Logic Model
Activities What activities are specified in your
NIDRR grant proposal?

• What actions, processes, events, services,
  products, technologies, or other elements will be
  used to implement your project?
• Will your activities be exclusively offered to a
  specific audience or to members of a specific
  target system?
• Will your activities be organized or phased in
  a particular manner?

25

Examples of Activities Hold scientific
conference in the third year. Conduct 12 to
15 Disability Policy Forums.
(RRTC on
Employment Policy) Conduct a national
assessment of potential involvement of rural
Faith-Based Organizations in providing
community transportation for people with
disabilities.
(RRTC on Disability in Rural Communities)
Conduct randomized clinical trial of the
Executive Plus program.
(RRTC on TBI Interventions)
Steps in Developing Your Projects Logic Model

26
Connecting Dissemination Activities with
Projected OutcomesActivities

• Proposed Activities by Area
• Research
• Development
• Demonstration
• Dissemination
• Utilization
• Technical Assistance

27
Connecting Dissemination Activities with
Projected Outcomes Activities (continued)

• Coordinate all activities to address outcomes.
• Dissemination activities should promote outcome
  accomplishments.
• Product orientation (newsletter) - nonstrategic
• Activity orientation (web site) - nonstrategic
• Outcome orientation - strategic
• Outcome planning increases efficiency,
  effectiveness, and relevance.

28


OutputsDescribe the services,
products or events you anticipate will be the
results of your NIDRR-sponsored project
activities. What are the direct
services, products or research-based
materials that will be developed through
your project?
Steps in Developing Your Projects Logic Model

29

Outputs (continued) What findings,
publications, products, devices, systems,
and services) do you plan to produce ?
How do these vary for different target
systems?
Steps in Developing Your Projects Logic Model

30

Examples of Outputs Conference proceedings.
Papers and corresponding policy briefs based
on Disability Policy Forum results.
(RRTC on Employment
Policy) Journal articles to report on the
distribution and use of Section 5310 resources
and on the extent of adoption of coordinated
and voucher models for rural transportation.
(RTC on Disability in
Rural Communities)
Steps in Developing Your Projects Logic Model

31
Steps in Developing Your Projects Logic Model
Steps in Developing Your Projects Logic Model

• Measuring Outputs
• Outputs are directly observable.
• Output measures can be expressed in a
• quantitative or qualitative manner -- e.g.,
  of
• manuscripts published, students trained,
  patents
• filed, or the nature of new findings, tools,
  devices,
• products, etc.

32
Connecting Dissemination Activities with
Projected Outcomes Outputs

• Generic outputs.
• Individualized, tailored outputs.
• Clarity of relationship among
• Output
• Target system element
• Outcome

33

Target Systems In what
segments of the overall target audience will
change be measured through strategic data
collection? Who are the specific
members of your target audience from
whom data will be collected (such as people with
disabilities, service providers, etc.)?
What specific stakeholder groups have been
strategically selected in your project
dissemination plan to receive project
outputs?
Steps in Developing Your Projects Logic Model

34

Steps in Developing Your Projects Logic Model
Examples of Target Systems
Members of the RERC-AAC network who currently
use AAC and their families and advocates.

(RERC-AAC)
Rural Faith-Based Organizations.
(RTC on Disability in
Rural Communities) Members of the
Congressional Disability Caucus.

35
Connecting Dissemination Activities with
Projected OutcomesTarget Systems

• Relationship to outputs and outcomes
• Specificity by
• Type(s) of systems
• Geographic location vs. universe
• Sub-groups within target systems
• Whole or percentage of the whole
• Service (receiving services in last 12 months)

36
Connecting Dissemination Activities with
Projected OutcomesTarget Systems (continued)

• Age
• Gender
• Educational level
• SES
• Primary/secondary conditions
• Race/ethnicity

37

OutcomesHow will members of target
systems awareness, attitudes, knowledge,
behavior, skills, actions, decisions, policies or
level of functioning be measurably changed
(benefited) by your activities and outputs?
Short-term outcomes Mid-term outcomes
Long-term outcomes
Steps in Developing Your Projects Logic Model

38

Steps in Developing Your Projects Logic Model
Short-term Outcomes What
changes or improvements in the
identified target system will have occurred
under the direct influence of project
activities? What changes or improvements
in learning, awareness, knowledge and
attitudes will have occurred?

39
Steps in Developing Your Projects Logic Model

• Short-term Outcomes (continued)
• Changes or Improvements in
• Aspirations
• Attitudes
• Awareness
• Knowledge
• Learning
• Motivations
• Opinions
• Skills

40
Steps in Developing Your Projects Logic Model

• Examples of Short-term Outcomes
• Increased knowledge and awareness among target
• system about the disproportionate allocation
  of
• resources for rural transportation, as
  observed
• in focus groups.
• (RTC
  on Disability in Rural Communities)
• By the end of Year 3, at least 50 percent of
  the target
• system will demonstrate an increased
  understanding
• of arthritis prevention strategies (25 percent
• improvement on pre/post measurement).
• 
  (MARRTC)
• 
  

41
Steps in Developing Your Projects Logic Model

• Measuring Short-term Outcomes
• Do you have a data collection and tracking
  system
• in place to monitor and document progress
  towards
• the completion or satisfaction of short-term
  outcomes?
• What data and data sources will you use?
• Are they specific enough to provide evidence
  of your
• performance and accomplishments?

42
Connecting Dissemination Activities with
Projected OutcomesShort-term Outcomes

• Need for Specificity in Outcome Statements
• What will change?
• Who/what will manifest the change?
• What is the consistency of the change
• All members of the target system?
• In all cases?
• What outputs do the outcomes derive from?

43
Connecting Dissemination Activities with
Projected OutcomesShort-term Outcomes
(continued)

• What is the change in
• Degree of consistency (4 out of 5 times)
• Degree of accuracy
• Degree of frequency (used 80 of time)
• Degree of time
• Relationship to target system and outputs

44

Steps in Developing Your Projects Logic Model
Mid-term Outcomes What
expected or actual changes or
improvements in the identified target system
will have occurred in part as a result of the
use or adoption of project outputs?
What changes in behaviors, actions,
decisions, and policies will have taken place
in the identified target system in response
to short-term outcomes?

45
Steps in Developing Your Projects Logic Model

• Mid-term Outcomes (continued)
• Changes or Improvements in
• Actions
• Behaviors
• Decisions
• Policies
• Practices
• Social Actions

46
Steps in Developing Your Projects Logic Model

• Examples of Mid-term Outcomes
• 25 percent of the target system will
  demonstrate
• increased implementation of strategies to
• eliminate disproportionate allocation of
• resources for rural transportation, as
  identified
• through a survey.
• (RTC on Disability in
  Rural Communities)
• By the end of Year 4, at least 50 percent of
  the
• target system will demonstrate an increased
• use of arthritis self-management techniques as
• observed in focus groups.
• 
  (MARRTC)
• 
  

47
Steps in Developing Your Projects Logic Model

• Measuring Mid-term Outcomes
• Do you have a data collection and tracking
  system in place to monitor and document progress
  towards the completion or satisfaction of
  mid-term outcomes?
• What data and data sources will you use?
• Are they specific enough to provide evidence of
  your performance and accomplishments?

48
Connecting Dissemination Activities with
Projected OutcomesMid-term Outcomes

• Specificity
• Linkage to short-term outcomes
• Research-based Interventions become acted upon
• Within specific environments
• By specific practitioners
• Data and Data Sources
• Feasible
• Not comprehensive
• Suggestive of movement as change

49

Steps in Developing Your Projects Logic Model
Long-term Outcomes
What changes would you expect to occur
in overall conditions experienced generally
by society from your project work or related
efforts? What changes in universal
environments or social policies would be
expected to occur?

50

Steps in Developing Your Projects Logic Model

• Examples of Long-term Outcomes
• Frequency of deaths due to bicycling accidents
  will decrease
• Rate of traumatic brain injuries from bicycling
  accidents will decline
• (Bicycle Helmet Public Information
  Campaign, RUSH Web Site)

51
RRTC Outcomes Planning and Reporting Workshop
From Activity to Outcome Common Weaknesses in
Grantee Formulation of Outcome Goals Mark
Linder

• April 27, 2005
• 800 a.m. - 1200 p.m. - Discovery II
• Holiday Inn Capitol
• 550 C St., SW, Washington, DC 20024

52
Common Weaknesses in Grantee Formulation of
Outcome Goals

• Types of Weaknesses
• Weaknesses in the formulation of outcomes
  statements
• Weaknesses in the application of performance
  measurement components
• Weaknesses related to evidence

53
Common Weaknesses in Grantee Formulation of
Outcome Goals

• I. Weaknesses in the Formulation of Outcome
  Statements
• Lack of Specificity in What and Where (i.e.
  target systems)
• Number and Scope of Outcomes
• Speculation vs. Expectation

54
Common Weaknesses in Grantee Formulation of
Outcome Goals

• Lack of Specificity in What and Where
• Weaknesses
• Generic statements suggesting a Center is
  producing new knowledge in some broad area
  affecting a boundless target system of end-users.
• Suggested Remedy
• Identify the specific nature of the anticipated
  change or improvement and the particular group or
  sample population (i.e., target system) within
  which the change will occur and could be measured.

55
Common Weaknesses in Grantee Formulation of
Outcome Goals

• Number and Scope of Outcomes
• Weaknesses
• Identification of too many outcomes might exceed
  what can effectively be monitored for progress.
• Outcomes stated too broadly might be perceived as
  beyond the reach of Center activities.
• Outcomes stated too narrowly might sound trivial
  or of limited significance.
• Suggested Remedy
• Ensure that anticipated outcomes are commensurate
  with Center objectives and can be linked to
  activities and outputs.

56
Common Weaknesses in Grantee Formulation of
Outcome Goals

• Speculation vs. Expectation
• Weaknesses
• Outcomes expressed as speculations or a wish
  list of what could happen rather than as
  verifiable statements of expected changes or
  improvements that are linked to outputs.
• Suggested Remedy
• Anticipated outcomes should be based on the
  realistic expectation that evidence can be
  provided to demonstrate progress toward achieving
  expected change/improvement.

57
Common Weaknesses in Grantee Formulation of
Outcome Goals

• II. Weaknesses in Application of Performance
  Measurement Components
• Confusion Between Short-Term and Mid-Term
  Outcomes
• Activities vs. Outcomes
• Outputs vs. Outcomes
• Linkage Between Short-Term Outcomes and Outputs

58
Common Weaknesses in Grantee Formulation of
Outcome Goals

• Confusion between Short-Term and Mid-Term
  Outcomes
• Weaknesses
• Changes in awareness and understanding mislabeled
  as intermediate outcomes.
• Changes in behavior and practice mislabeled as
  short-term outcomes.
• Suggested Remedy
• Develop a logic model to diagram
  interrelationships among short-term and mid-term
  outcomes.

59
Common Weaknesses in Grantee Formulation of
Outcome Goals

• Activities vs. Outcomes
• Weaknesses
• Short-term outcomes expressed in terms of
  activities (i.e. what is being done) rather than
  as expected change/improvement.
• Anticipated change/improvement implied but not
  specifically stated.
• Suggested Remedy
• Develop a logic model to diagram
  interrelationships among activities and outcomes.

60
Common Weaknesses in Grantee Formulation of
Outcome Goals

• Outputs vs. Outcomes
• Weaknesses
• Outcome statements describe outputs (i.e.
  products and/or services) rather than
  change/improvement in the knowledge or action
  system.
• Suggested Remedy
• Develop a logic model to diagram
  interrelationships among outputs and outcomes.

61
Common Weaknesses in Grantee Formulation of
Outcome Goals

• Linkage Between Short-Term Outcomes and Outputs
• Weaknesses
• Short-term outcomes that do not reference the
  outputs (i.e. findings products, services) upon
  which they are based.
• Suggested Remedy
• The production of outputs should be explicitly
  linked to planning for and achieving short-term
  outcomes.
• A Centers decisions about which outputs to
  produce and how many should depend on what
  changes/improvements are intended and in which
  target systems they are anticipated.

62
Common Weaknesses in Grantee Formulation of
Outcome Goals

• III. Weaknesses Related to Evidence
• Missing Information on Types of Evidence
• Source of Data vs. Type of Evidence

63
Common Weaknesses in Grantee Formulation of
Outcome Goals

• Missing Information on Types of Evidence
• Weaknesses
• Identified outcomes lack reference to the types
  of evidence grantees need to collect to
  demonstrate progress toward achieving anticipated
  goals.
• Suggested Remedy
• Formulating outcome statements that are
  adequately specified in terms of the nature of
  the change/improvement, the outputs supporting
  this change, and the sub-group of the target
  system in which change will occur will clarify
  the evidence to be collected for assessing
  progress.

64
Common Weaknesses in Grantee Formulation of
Outcome Goals

• Source of Data vs. Type of Evidence
• Weaknesses
• Confusing source of data (i.e. where you would
  look, whom you would sample) with type of
  evidence (i.e. what information you need to
  collect, which strategies you would use to
  collect it).
• Suggested Remedy
• Formulating outcome statements that are
  adequately specified in terms of the nature of
  the change/improvement, the outputs supporting
  this change, and the sub-group of the target
  system in which change will occur will clarify
  the evidence to be collected for assessing
  progress.

65
RRTC Outcomes Planning and Reporting Workshop
From Activity to Outcome Please contact us with
any additional question(s) or technical
assistance needs. Contact NCDDR for
dissemination questions or needs at 800-266-1832,
by fax [phone]), by e-mail
ncddr_at_ncddr.org Contact RUSH for logic models
and/or outcomes planning questions or needs at
800-761-RUSH (7874), by fax [phone]), by
e-mail rush_at_sedl.org