Components of a Logic Model

1
As part of the first activity, please complete
the Pre-Test Worksheet and the Outcomes
Planning Survey and return it to an NCDDR staff
member.
2
Dissemination and Outcomes Planning Workshop
From Activity to Outcome Overview of the Shift
in Focus from Activity to Outcome

• June 1, 2005
• 800 a.m. - 500 p.m.
• National Press Club
• 529 14th St. NW, 13th Floor - Washington, DC
  20045

3
Overview of the Shift in Focus from Activity
to Outcome

• NIDRRs responsiveness to questions of worth
  (accountability)
• How does NIDRR make a difference?
• How was the purpose of the funding area advanced?
• Goal responses come from the basis of
• outcomes or changes produced
• not from conducting an activity
• Challenging development process

4
Dissemination and Outcomes Planning Workshop
From Activity to OutcomeThe Process of
Dissemination Essential ComponentsJohn
Westbrook

• June 1, 2005800 a.m. - 500 p.m.National Press
  Club529 14th St. NW, 13th Floor - Washington, DC
  20045

5
Dissemination ? Distribution

• Dissemination is not the physical movement of
  products from Point A to Point B.

6
Dissemination ? Documentation

• Dissemination is not the production of
  materials.

7
The Goal of Dissemination is Utilization

• Dissemination and Utilization (DU) denotes a
  process that is two-way and provides support for
  actual changes to occur as the intended result of
  dissemination activities.

8
Working Definition

• The purpose of dissemination activity is to
  assure that information/ knowledge useful in
  reaching decisions, making changes, or taking
  specific action is available to those who can
  most benefit from it.

9
What Contributes to the Effectiveness of DU
Efforts?

• Extent to which it is oriented to the needs of
  the targeted system/audience.
• Use of varied methods to accommodate different
  learning styles.
• Options are available for targeted user to
  indicate their needs and priorities - make
  choices.
• Maximize use at existing networks and
  communication channels.

10
What Contributes to the Effectiveness of DU
Efforts? (continued)

• Ensures accuracy, relevance, and
  representativeness of information/message shared.
• Provides sufficient information for targeted
  user to analyze.
• Allows technical assistance as necessary -
  informational and in-person.

11

Essential Elements of Dissemination and
Utilization

User
Medium
DISSEMINATION AND KNOWLEDGE UTILIZATION
Source
Context
Content
12

DISSEMINATION AND KNOWLEDGE UTILIZATION
Source
Where does it come from?
13
Source

• The source of information is important to
  potential users in determining value of
  information content.
• The nature of information disseminated is less
  important than the links between developers,
  intermediaries, and users.
• Target systems/audiences tend to accept
  assistance, information, and ideas from sources
  they know and trust.

14

DISSEMINATION AND KNOWLEDGE UTILIZATION
Content
What is your message?
15
Content

• Content is highly variable and wide-ranging.
  Quality of content is a major concern in DU.
  Content of the message needs to be compatible
  with the context of potential users lives.
  Content needs to be sufficient to address
• How to use/apply the information and
• Implications (benefits) of use/application.

16

DISSEMINATION AND KNOWLEDGE UTILIZATION

Context
How does it relate?
17
Context

• Context includes characterizations of packaging.
• The context of your message is reflective of
• I use information when I think I need it.
• I use information when I understand/believe the
  results can be realized by me.
• Context includes characterizations of packaging.

18

How can I get it?

Medium
DISSEMINATION AND KNOWLEDGE UTILIZATION
19
Medium

• Medium is selected dissemination. The more
  narrow and restrictive it is the less likely your
  message will be accessed and used.
• Cost
• Time
• Technology-based

20

How can I benefit from it? Do I need this?

User
DISSEMINATION AND KNOWLEDGE UTILIZATION
21
User

• It is not possible to know too much about your
  targeted users.
• Primary language
• Economic status
• Reading level
• Primary information channels
• television
• radio
• newspaper
• magazines
• junk mail
• Important to recognize readiness for change
  will prevail.

22
Do you have a dissemination policy?

• A dissemination policy can
• Clarify value placed on DU
• Facilitate utilization measurement
• Focus on target systems/audiences
• Reduce staff confusion
• Link to your organizations mission

23
Dissemination and Outcomes Planning Workshop
From Activity to Outcome Key Elements of a
Dissemination PlanJoann Starks

• June 1, 2005
• 800 a.m. - 500 p.m.
• National Press Club
• 529 14th St. NW, 13th Floor - Washington, DC
  20045

24
Key Elements of a Dissemination Plan

• Goals
• Objectives
• Users
• Content
• Sources(s)

• Medium
• Success
• Access
• Availability
• Barriers

25
Key Elements of a Dissemination Plan

• Goals
• What would utilization look like?

26
Key Elements of a Dissemination Plan

• 2. Objectives
• What actions will help arrive at utilization
  goal(s)?

27
Key Elements of a Dissemination Plan

• 3. Users
• Identify target systems not target audiences.
• Clarify potential user characteristics.

28
Key Elements of a Dissemination Plan

• 4. Content
• What is your key point?
• How can you help integrate your message?

29
Key Elements of a Dissemination Plan

• 5. Source(s)
• Your organization.
• Partner organizations.
• Another organization.

30
Key Elements of a Dissemination Plan

• 6. Medium or Media
• What is your communication strategy?

31
Key Elements of a Dissemination Plan

• 7. Success
• Collect formative data as you go.

32
Key Elements of a Dissemination Plan

• 8. Access
• How will you promote awareness?

33
Key Elements of a Dissemination Plan

• 9. Availability
• Prepare for anticipated demand.
• Make alternate formats available.

34
Key Elements of a Dissemination Plan

• 10. Barriers
• What barriers are out there to thwart your
  dissemination efforts?

35
Key Elements of a Dissemination Plan

• Goals
• Objectives
• Users
• Content
• Sources(s)

• Medium
• Success
• Access
• Availability
• Barriers

36
Dissemination and Outcomes Planning Workshop
From Activity to OutcomeWhat Makes a
Dissemination Plan Effective?John Westbrook and
Mark Linder

• June 1, 2005
• 800 a.m. - 500 p.m.
• National Press Club
• 529 14th St. NW, 13th Floor - Washington, DC
  20045

37
What Makes a Dissemination Plan Effective?

• NCDDR Technical Assistance
• Need help? The NCDDR provides individualized
  and group technical assistance (TA) products to
  NIDRR grantees for the purposes of facilitating
  dissemination and outcomes planning. TA
  activities are free of charge and include
• Tailored onsite and offsite consultations and
  training sessions.
• Assistance in developing effective and realistic
  dissemination and utilization plans.
• Assistance in planning and conducting evaluations
  of their dissemination and utilization outcomes.
• Working with grantees to promote integration of
  dissemination and outcomes plans.

38
What Makes a Dissemination Plan Effective?

• Dissemination Self Inventory
• Want to know if you need help? The NCDDRs
  Dissemination Self Inventory is a planning tool
  designed to help NIDRR grantees assess the
  effectiveness, strengths, and weaknesses of their
  planned dissemination strategies. This tool
  addresses four primary areas
• Organizational structure and policies
• Research design
• Dissemination plan and
• Evaluation plan.

39
What Makes a Dissemination Plan Effective?

• Strategic Qualities of Budgeting
• What financial resources have you set aside to
  implement your planned dissemination activities?
• Are the financial resources earmarked for each
  dissemination activity adequate to support their
  full implementation?
• Does your budget for dissemination activities
  consider the appropriate monetary needs for each
  planned strategy?

40
What Makes a Dissemination Plan Effective?

• Staff Time Available
• All NIDRR grantees share in the responsibility to
  disseminate their project results in accessible
  formats to all appropriate target audiences.
• Do you have staff assigned to implementing
  dissemination plans? If not, why not?
• What areas of expertise are necessary to
  implement effective dissemination strategies?

41
What Makes a Dissemination Plan
Effective?Example Dissemination Plan -
Budget/Staff
42
What Makes a Dissemination Plan
Effective?Example Dissemination Plan -
Budget/Staff(continued)
43
What Makes a Dissemination Plan Effective?

• Targeting Key Audiences with
• Effective Formats/Media
• Do you match each dissemination medium to the
  expressed needs and preferences of specific use
  groups?
• Are products adapted for ease of understanding by
  specific user groups?
• Users will accept assistance, information, and
  ideas from sources they believe to be credible
  and trustworthy.

44
What Makes a Dissemination Plan
Effective?Example Dissemination Plan -
Media/Formats
45
What Makes a Dissemination Plan
Effective?Example Dissemination Plan -
Media/Formats (continued)
46
What Makes a Dissemination Plan Effective?

• Evaluation as Key to Improving Utilization
• The role of evaluation is to help disability
  researchers understand how much and how
  effectively consumers are using the research
  produced and disseminated.
• Evaluation is not research.
• Does your evaluation plan identify indicators of
  change among intended users of project outputs?
• Does your evaluation plan methodology include
  strategies and procedures that are likely to
  elicit useful data from diverse audiences?

47
What Makes a Dissemination Plan
Effective?Example Dissemination Plan -
Evaluation
48
What Makes a Dissemination Plan
Effective?Example Dissemination Plan -
Evaluation (continued)
49
What Makes a Dissemination Plan Effective?

• Understanding the Change Process
•   
• The individual needs of information users will
  vary according to the levels of use and stages of
  personal concern demonstrated.
• Programs and practices must be adapted to meet
  the particular needs of each individual
  organization.
• Information users will more effectively implement
  change if they, themselves, understand the
  process and the flow of activities that will be
  involved.

50
Dissemination and Outcomes Planning Workshop
From Activity to OutcomeThe Logic Model
Approach to Outcomes PlanningMark Linder and
Joann Starks

• June 1, 2005
• 800 a.m. - 500 p.m.
• National Press Club
• 529 14th St. NW, 13th Floor - Washington, DC 20045

51
The Logic Model Approach to Outcomes Planning

• What is a Logic Model?
• A logic model is a highly visual method of
  demonstrating relationships among project
  resources, activities, outputs, and outcomes.
• Logic models are planning tools that indicate the
  resources a NIDRR project will employ to conduct
  activities that are intended to produce specific,
  describable, and measurable changes or results in
  people, organizations, or the broader physical
  and social environment.

52
The Logic Model Approach to Outcomes Planning

• Why a Logic Model?
• Logic models help researchers target identified
  goals and desired outcomes by demonstrating how
  each step in an unfolding program or project is
  critically linked to the preceding step, whether
  it is needs sensing, building a conceptual
  framework, conducting activities and implementing
  interventions, or performing follow-up.
• Logic models to help identify feasible types of
  data and data collection strategies to suggest
  progress toward identified project-relevant
  short-term and/or mid-term outcomes.

53
The Logic Model Approach to Outcomes Planning

• Logic Model Strategies Backward Chaining
• Backward chaining refers to the process of
  beginning logic model development by considering
  project outcomes and working in reverse to link
  the anticipated outcomes to audiences reached,
  outputs produced, and activities conducted.
• Backward chaining enables newer projects to
  envision the realistic end results of their
  project work and then plan the activities from
  their NIDRR-funded scope of work will achieve
  such changes.

54
The Logic Model Approach to Outcomes Planning

• Backward Chaining (continued)
• Backward chaining is of particular value to
  so-called in process projects further down the
  road in their NIDRR grant cycle. Such projects
  might already have outcomes to report or data to
  suggest progress toward anticipated outcomes.
  Backward chaining allows these projects to
  benefit from logic model in spite of having a
  shorter time to plan for outcomes.
• All projects must remember that backward chaining
  is a strategy to plan for outcomes, not an
  opportunity to create of new activities outside
  of their proposed scope of work. Outcomes should
  still be linked to the activities from the
  original project proposal.

55
The Logic Model Approach to Outcomes Planning

• Components of a Logic Model
• Situation
• Inputs
• Activities
• Outputs
• Target Systems
• Outcomes
• Short-term Outcomes
• Mid-term Outcomes
• Long-term Outcomes

56
The Logic Model Approach to Outcomes Planning

• Situation
• The goal of your NIDRR-sponsored project.
• Identifies the problem or priority your project
  is addressing and the benefit to specific
  audiences.

57
The Logic Model Approach to Outcomes Planning

• Situation Key Questions to Answer
• Clearly describe the purpose of your
  NIDRR-sponsored project
• What is the overall goal of your project?
• Why is this important?
• What specific problem or problems are you trying
  to address with your center or project?
• What are you trying to accomplish relative to
  this problem?

58
The Logic Model Approach to Outcomes Planning

• Example of Situation
• Published Personal Attendant Care (PAC) Quality
  Indicators (QIs) are not designed for
  low-literate, multicultural populations.
• (RRTC on Measuring Rehabilitation Outcomes and
  Effectiveness)
• There is a lack of awareness and implementation
  of Section 508 accessibility standards among
  e-Learning practitioners.
• (Accessible e-Learning Authoring System Software
  and Model Course for Vocational Rehabilitation
  Services Personnel)

59
The Logic Model Approach to Outcomes Planning

• Inputs
• The human and financial resources and systems
  needed to conduct a high quality,
  outcomes-oriented program or project.
• Types of resources staff expertise, volunteers,
  time, money, host institutional support,
  materials, equipment, technology, research
  results, previous accomplishments, etc.

60
The Logic Model Approach to Outcomes Planning

• Inputs Key Questions to Answer
• What resources are available to your
  NIDRR-sponsored project to
• carry out the scope of work?
• What human, financial, or other resources are
  available to apply to the project?
• What supports do you have from NIDRR, other
  contributing agencies, and from your host
  organization?

61
The Logic Model Approach to Outcomes Planning

• Inputs Key Questions to Answer (continued)
• What resources and planning and management
  infrastructures do you need in order to conduct
  high-quality activities, produce outputs, and
  demonstrate progress toward completing
  anticipated outcomes?
• Do you have sufficient resources and
  infrastructures in place to meet your goals?

62
The Logic Model Approach to Outcomes Planning

• Examples of Inputs
• Previous research findings on the significant
  inequity in allocation of transportation
  resources between urban and rural areas.
• (RTC on Disability in Rural Communities)
• Previous expertise in design and testing of
  accessible e-Learning.
• Programming expertise from project
  collaborator.(Accessible e-Learning Authoring
  System Software and Model Course for Vocational
  Rehabilitation Services Personnel)
• The data resources of the StatsRRTC.
• (RRTC on Employment Policy)

63
The Logic Model Approach to Outcomes Planning

• Activities
• The action steps, tasks, procedures, and services
  performed in conjunction with implementing a
  planned program of research and development,
  capacity-building, training and/or technical
  assistance and knowledge translation and
  dissemination to achieve objectives and
  demonstrate results.
• Project activities include the research,
  development, training, technical assistance,
  dissemination, utilization, and other activities
  specified in your NIDRR grant proposal.

64
The Logic Model Approach to Outcomes Planning

• Activities Key Questions to Answer
• What activities are specified in your NIDRR grant
  proposal?
• What actions, processes, events, services,
  products, technologies, or other elements will be
  used to implement your project?
• Will your activities be exclusively offered to a
  specific audience or to members of a specific
  target system?
• Will your activities be organized or phased in
  a particular manner?

65
The Logic Model Approach to Outcomes Planning

• Examples of Activities
• Hold scientific conference in the third year.
• Conduct 12 to 15 Disability Policy Forums.
• (RRTC on Employment Policy)
• Develop journal articles focusing on technology
  transfer and research findings.
• (RERC on Communication Enhancement AAC-RERC)
• Conduct a national assessment of potential
  involvement of rural Faith-Based Organizations in
  providing community transportation for people
  with disabilities.
• (RRTC on Disability in Rural Communities)

66
The Logic Model Approach to Outcomes Planning

• Examples of Activities (continued)
• Build and field-test e-Learning authoring system
  software.
• (Accessible e-Learning Authoring System Software
  and Model Course for Vocational Rehabilitation
  Services Personnel)
• Conduct randomized clinical trial of the
  Executive Plus program.
  
• (RRTC on TBI Interventions)

67
The Logic Model Approach to Outcomes Planning

• Outputs
• The direct results of program activities, such as
  the discoveries or findings, models, tools,
  devices, products, publications, events, and/or
  services produced for external audiences.
• Outputs may be expressed in quantitative or
  qualitative terms.
• Outputs are directly observable.
• Outputs are important indicators of productivity
  and are the essential building blocks of
  outcomes.
• Though outputs contribute to outcomes, not every
  output will have a corresponding outcome, nor
  should it.

68
The Logic Model Approach to Outcomes Planning

• Outputs Key Questions to Answer
• Describe the services, products or events you
  anticipate will be the
• results of your NIDRR-sponsored project
  activities.
• What are the direct services, products or
  research-based materials that will be developed
  through your project?
• What findings, publications, products, devices,
  systems, and services) do you plan to produce ?
• How do these vary for different target systems?

69
The Logic Model Approach to Outcomes Planning

• Examples of Outputs
• Conference proceedings.
• Papers and corresponding policy briefs based on
  Disability Policy Forum results.
• (RRTC on Employment Policy)
• Journal articles.
• AAC Website.
• Writers Brigade Publications.
• (AAC-RERC)

70
The Logic Model Approach to Outcomes Planning

• Examples of Outputs (continued)
• Journal articles to report on the distribution
  and use of Section 5310 resources and on the
  extent of adoption of coordinated and voucher
  model for rural transportation.
• (RTC on Disability in Rural Communities)

71
The Logic Model Approach to Outcomes Planning

• Measuring Outputs
• Outputs are directly observable.
• Output measures can be expressed in a
  quantitative or qualitative manner -- e.g., of
  manuscripts published, students trained, patents
  filed, or the nature of new findings, tools,
  devices, products, etc.

72
The Logic Model Approach to Outcomes Planning

• Target Systems
• The group of individuals, organizations, or other
  entities expected to be affected by a project.
  Target systems are specific, realistically
  measurable consumer audiences in which the group,
  the commonalities bonding the group, and the
  context within which the group exists are all
  defined within a project's scope of work.
• The specific members of the overall target
  audience identified as potential beneficiaries of
  your projects work and among whom data
  collection will occur.

73
The Logic Model Approach to Outcomes Planning

• Target Systems Key Questions to Answer
• In what segments of the overall target audience
  will change be
• measured through strategic data collection?
• Who are the specific members of your target
  audience from whom data will be collected (such
  as people with disabilities, service providers,
  etc.)?
• What specific stakeholder groups have been
  strategically selected in your project
  dissemination plan to receive project outputs?

74
The Logic Model Approach to Outcomes Planning

• Examples of Target Systems
• Members of the RERC-AAC network who currently use
  AAC and their families and advocates.
• (RERC-AAC)
• Rural Faith-Based Organizations.
• (RTC on Disability in Rural Communities)
• Members of the Congressional Disability Caucus.

75
The Logic Model Approach to Outcomes Planning

• Examples of Target Systems (continued)
• e-Learning practitioners producing CD-ROM based
  educational courses
• State offices of vocational rehabilitation within
  Federal Region VI
• (Accessible e-Learning Authoring System Software
  and Model Course for Vocational Rehabilitation
  Services Personnel)

76
The Logic Model Approach to Outcomes Planning

• Outcomes
• The effects of RD and related activities and
  outputs and constitute changes or improvements in
  identified target systems.
• Outcomes are separated into 3 groups short-term,
  mid-term, and long-term.
• Outcome goals differ by the type of change or
  improvement intended (knowledge, action, etc.),
  the target system in which change is expected to
  occur, and the level of accountability for which
  grantees are responsible.
• Outcomes are produced in more than one target
  system and occur sequentially over time,
  resulting in an incremental chain of outcomes,
  starting with short-term and progressing to
  mid-term and eventually longer-term outcomes.

77
The Logic Model Approach to Outcomes Planning

• Outcomes Key Questions to Answer
• How will members of target systems awareness,
  attitudes, knowledge, behavior, skills, actions,
  decisions, policies or level of functioning be
  measurably changed (benefited) by your activities
  and outputs?
• Short-term outcomes
• Mid-term outcomes
• Long-term outcomes

78
The Logic Model Approach to Outcomes Planning

• The Outcomes Continuum Maximal to Minimal
• Maximal Compelling data has been collected to
  suggest progress toward anticipated outcomes.
  Researcher confidence in outcomes data is high.
• Minimal Researcher has lower confidence in data
  due to limited scope of the target system, amount
  of data collected, limited demonstrable linkage
  of project activities and outputs to documented
  outcomes, etc.

79
The Logic Model Approach to Outcomes Planning

• Short-term Outcomes
• The anticipated or actual changes or improvements
  in the knowledge base and/or learning system
  within an identified target system.
• Short-term outcomes are under the direct
  influence of project activities.
• They represent the first level of change that
  must occur in order to bring about mid-term
  outcomes.
• Types of change learning, awareness, knowledge,
  attitudes, skills, opinions, aspirations, and
  motivations.

80
The Logic Model Approach to Outcomes Planning

• Short-term Outcomes Key Questions to Answer
• What changes or improvements in the identified
  target system will have occurred under the direct
  influence of project activities?
• What changes or improvements in learning,
  awareness, knowledge and attitudes will have
  occurred?

81
The Logic Model Approach to Outcomes Planning

• Examples of Short-term Outcomes
• Increased knowledge and awareness among target
  system about the disproportionate allocation of
  resources for rural transportation, as observed
  in focus groups.
• (RTC on Disability in Rural Communities)
• Increase literacy, skills, and ability among
  sampled individuals who use AAC.
• (AAC-RERC)

82
The Logic Model Approach to Outcomes Planning

• Examples of Short-term Outcomes (continued)
• 50 of VR counselors in Federal Region VI report
  improved knowledge of utility and accessibility
  of e-Learning courses.
• (Accessible e-Learning Authoring System Software
  and Model Course for Vocational Rehabilitation
  Services Personnel)
• By the end of Year 3, at least 50 percent of the
  target system will demonstrate an increased
  understanding of arthritis prevention strategies
  (25 percent improvement on pre/post measurement).
• (MARRTC)

83
The Logic Model Approach to Outcomes Planning

• Measuring Short-term Outcomes
• Do you have a data collection and tracking system
  in place to monitor and document progress towards
  the completion or satisfaction of short-term
  outcomes?
• What data and data sources will you use?
• Are they specific enough to provide evidence of
  your performance and accomplishments?

84
The Logic Model Approach to Outcomes Planning

• Mid-term Outcomes
• The anticipated or actual changes or improvements
  that occur in part as a result of the use or
  adoption of program outputs.
• Mid-term outcomes involve changes in use and,
  thus, take longer to achieve than short-term
  outcomes.
• Unlike short-term outcomes which occur under the
  direct influence of program activities, there may
  be other causal factors contributing to the
  achievement of mid-term outcomes.
• Types of change actions, behaviors, practices,
  decisions, policies, system capacity, and social
  actions.

85
The Logic Model Approach to Outcomes Planning

• Mid-term Outcomes Key Questions to Answer
• What expected or actual changes or improvements
  in the identified target system will have
  occurred in part as a result of the use or
  adoption of project outputs?
• What changes in behaviors, actions, decisions,
  and policies will have taken place in the
  identified target system in response to
  short-term outcomes?

86
The Logic Model Approach to Outcomes Planning

• Examples of Mid-term Outcomes
• 25 percent of the target system will demonstrate
  increased implementation of strategies to
  eliminate disproportionate allocation of
  resources for rural transportation, as identified
  through a survey.
• (RTC on Disability in Rural Communities)
• New policies enacted by AAC manufacturers and
  policymakers encouraging increased use and
  availability of AAC technologies.
• (AAC-RERC)

87
The Logic Model Approach to Outcomes Planning

• Examples of Mid-term Outcomes (continued)
• 50 of e-Learning practitioners report increased
  development of accessible courses.
• (Accessible e-Learning Authoring System Software
  and Model Course for Vocational Rehabilitation
  Services Personnel)
• By the end of Year 4, at least 50 percent of the
  target system will demonstrate an increased use
  if arthritis self-management techniques as
  observed in focus groups.
• (MARRTC)

88
The Logic Model Approach to Outcomes Planning

• Measuring Mid-term Outcomes
• Do you have a data collection and tracking system
  in place to monitor and document progress towards
  the completion or satisfaction of mid-term
  outcomes?
• What data and data sources will you use?
• Are they specific enough to provide evidence of
  your performance and accomplishments?

89
The Logic Model Approach to Outcomes Planning

• Long-term Outcomes
• The desired end-results of an RD related
  program, constituting changes or improvements in
  the overall condition of a population or target
  system.
• Given their scope, long-term outcomes usually
  take many years to occur, and therefore, neither
  NIDRR nor its grantees are held directly
  accountable for producing this type of change.
• Types of change conditions, social contexts, and
  environmental characteristics.

90
The Logic Model Approach to Outcomes Planning

• Long-term Outcomes Key Questions to Answer
• What changes would you expect to occur in overall
  conditions experienced generally by society from
  your project work or related efforts?
• What changes in universal environments or social
  policies would be expected to occur?

91
The Logic Model Approach to Outcomes Planning

• Examples of Long-term Outcomes
• Frequency of deaths due to bicycling accidents
  will decrease
• Rate of traumatic brain injuries from bicycling
  accidents will decline
• (Bicycle Helmet Public Information Campaign,
  RUSH Web Site)

92
The Logic Model Approach to Outcomes Planning

• Activity, Output, or Outcome?
• Relevant instruments to assess longer term
  program outcomes in consumer-relevant settings.
• 50 of e-Learning practitioners within Federal
  Region VI participating in authoring tool
  tutorial sessions report increased awareness of
  strategies to design and produce accessible
  e-Learning.
• Develop presentation for technical conferences
  pertaining to e-Learning.
• Concise, targeted materials to help enhance
  consumer choice in rehabilitation program
  selection.
• Develop and validate usability and educational
  effectiveness of model course on CD-ROM.

93
The Logic Model Approach to Outcomes Planning

• Activity, Output, or Outcome? (continued)
• 50 of e-Learning practitioners within Federal
  Region VI report increased development of
  accessible courses within 12 months of
  participation in technical assistance regarding
  application of e-Learning authoring tool.
• Conduct 3 separate focus groups of 10 AAC users
  each.
• 25 of the VR counselors in Illinois respond with
  80 accuracy to the RTC assessment of counseling
  skills administered at the end of RTC project
  year three.
• Conference presentations.
• Enhanced instrumentation and methods to assess
  community participation.

94
Dissemination and Outcomes Planning Workshop
From Activity to OutcomeProposed Steps in
Developing Well-Formulated Outcome GoalsJoann
Starks

• June 1, 2005
• 800 a.m. - 500 p.m.
• National Press Club
• 529 14th St. NW, 13th Floor - Washington, DC 20045

95
Proposed Steps in Developing Well-Formulated
Outcome Goals

• Purpose The following steps represent the
  proposed components of well-formulated outcome
  goals that NIDRR is experimenting with in the
  revised web-based annual project performance
  reporting (APPR) system).
• Source Components are discussed in more detail
  in the forthcoming technical assistance document
  titled Steps in Formulating Outcome Goals An
  Instructional Guide for NIDRR Grantees,
  Applicants, Staff, and Reviewers (May 2005).

96
Proposed Steps in Developing Well-Formulated
Outcome Goals

• Anticipated Outcomes and Goal Statements
• Identify the types of changes or improvements in
  the Knowledge-Base and Learning System
  (short-term outcomes) and/or in the Action System
  (intermediate outcomes) that are anticipated to
  occur as a result of grant activities and
  outputs, and provide operational definitions of
  outcomes, including the quantitative performance
  measures or qualitative indicators that will be
  used to gauge progress.

97
Proposed Steps in Developing Well-Formulated
Outcome Goals

• Target Systems
• Provide a concrete delineation of the specific
  sub-populations or subsystems in which
  anticipated outcomes are expected to occur.
• 3) Relevance
• Indicate the importance of the anticipated
  outcomes for identified target system or systems.
  
• 4) Consistency with Agency Priorities and/or
  Performance Measures
• Describe how grant-specific outcome goals are
  aligned with NIDRR published priorities and
  applicable agency long-term or annual performance
  measures.

98
Proposed Steps in Developing Well-Formulated
Outcome Goals

• Specify Thresholds, Milestones, and Timeframes
  for Each Outcome Goal
• Thresholds The amount of change that must occur
  to define goal attainment,
• Milestones The significant events or interim
  objectives that will be used to mark meaningful
  progress towards achievement of outcome goals,
  and
• Timeframe The period of time and/or the dates by
  which identified milestones will be accomplished.
  

99
Proposed Steps in Developing Well-Formulated
Outcome Goals

• Linkage to Activities and Outputs
• Document the program activities (e.g., RD,
  capacity-building and knowledge-translation
  strategies and key partners) and outputs (e.g.,
  significant findings, publications, products,
  and/or services) that are or will be responsible
  for producing outcome goals.

100
Dissemination and Outcomes Planning Workshop
From Activity to Outcome Common Weaknesses in
Grantee Formulation of Outcome Goals Mark
Linder

• June 1, 2005
• 800 a.m. - 500 p.m.
• National Press Club
• 529 14th St. NW, 13th Floor - Washington, DC 20045

101
Common Weaknesses in Grantee Formulation of
Outcome Goals

• Types of Weaknesses
• Weaknesses in the formulation of outcomes
  statements
• Weaknesses in the application of performance
  measurement components
• Weaknesses related to evidence

102
Common Weaknesses in Grantee Formulation of
Outcome Goals

• I. Weaknesses in the Formulation of
• Outcome Statements
• Lack of Specificity in What and Where (i.e.
  target systems)
• Number and Scope of Outcomes
• Speculation vs. Expectation

103
Common Weaknesses in Grantee Formulation of
Outcome Goals

• Lack of Specificity in What and Where
• Weaknesses
• Generic statements suggesting a Center is
  producing new knowledge in some broad area
  affecting a boundless target system of end-users.
• Suggested Remedy
• Identify the specific nature of the anticipated
  change or improvement and the particular group or
  sample population (i.e., target system) within
  which the change will occur and could be measured.

104
Common Weaknesses in Grantee Formulation of
Outcome Goals

• Number and Scope of Outcomes
• Weaknesses
• Identification of too many outcomes might exceed
  what can effectively be monitored for progress.
• Outcomes stated too broadly might be perceived as
  beyond the reach of Center activities.
• Outcomes stated too narrowly might sound trivial
  or of limited significance.
• Suggested Remedy
• Ensure that anticipated outcomes are commensurate
  with Center objectives and can be linked to
  activities and outputs.

105
Common Weaknesses in Grantee Formulation of
Outcome Goals

• Speculation vs. Expectation
• Weaknesses
• Outcomes expressed as speculations or a wish
  list of what could happen rather than as
  verifiable statements of expected changes or
  improvements that are linked to outputs.
• Suggested Remedy
• Anticipated outcomes should be based on the
  realistic expectation that evidence can be
  provided to demonstrate progress toward achieving
  expected change/improvement.

106
Common Weaknesses in Grantee Formulation of
Outcome Goals

• II. Weaknesses in Application of Performance
  Measurement Components
• Confusion Between Short-Term and Mid-Term
  Outcomes
• Activities vs. Outcomes
• Outputs vs. Outcomes
• Linkage Between Short-Term Outcomes and Outputs

107
Common Weaknesses in Grantee Formulation of
Outcome Goals

• Confusion Between Short-Term and
• Mid-Term Outcomes
• Weaknesses
• Changes in awareness and understanding mislabeled
  as intermediate outcomes.
• Changes in behavior and practice mislabeled as
  short-term outcomes.
• Suggested Remedy
• Develop a logic model to diagram
  interrelationships among short-term and mid-term
  outcomes.

108
Common Weaknesses in Grantee Formulation of
Outcome Goals

• Activities vs. Outcomes
• Weaknesses
• Short-term outcomes expressed in terms of
  activities (i.e. what is being done) rather than
  as expected change/improvement.
• Anticipated change/improvement implied but not
  specifically stated.
• Suggested Remedy
• Develop a logic model to diagram
  interrelationships among activities and outcomes.

109
Common Weaknesses in Grantee Formulation of
Outcome Goals

• Outputs vs. Outcomes
• Weaknesses
• Outcome statements describe outputs (i.e.
  products and/or services) rather than
  change/improvement in the knowledge or action
  system.
• Suggested Remedy
• Develop a logic model to diagram
  interrelationships among outputs and outcomes.

110
Common Weaknesses in Grantee Formulation of
Outcome Goals

• Linkage Between Short-Term Outcomes and Outputs
• Weaknesses
• Short-term outcomes that do not reference the
  outputs (i.e. findings products, services) upon
  which they are based.
• Suggested Remedy
• The production of outputs should be explicitly
  linked to planning for and achieving short-term
  outcomes.
• A Centers decisions about which outputs to
  produce and how many should depend on what
  changes/improvements are intended and in which
  target systems they are anticipated.

111
Common Weaknesses in Grantee Formulation of
Outcome Goals

• III. Weaknesses Related to Evidence
• Missing Information on Types of Evidence
• Source of Data vs. Type of Evidence

112
Common Weaknesses in Grantee Formulation of
Outcome Goals

• Missing Information on Types of Evidence
• Weaknesses
• Identified outcomes lack reference to the types
  of evidence grantees need to collect to
  demonstrate progress toward achieving anticipated
  goals.
• Suggested Remedy
• Formulating outcome statements that are
  adequately specified in terms of the nature of
  the change/improvement, the outputs supporting
  this change, and the sub-group of the target
  system in which change will occur will clarify
  the evidence to be collected for assessing
  progress.

113
Common Weaknesses in Grantee Formulation of
Outcome Goals

• Source of Data vs. Type of Evidence
• Weaknesses
• Confusing source of data (i.e. where you would
  look, whom you would sample) with type of
  evidence (i.e. what information you need to
  collect, which strategies you would use to
  collect it).
• Suggested Remedy
• Formulating outcome statements that are
  adequately specified in terms of the nature of
  the change/improvement, the outputs supporting
  this change, and the sub-group of the target
  system in which change will occur will clarify
  the evidence to be collected for assessing
  progress.

114
Dissemination and Outcomes Planning Workshop
From Activity to Outcome
Connecting Dissemination Activities with
Projected Outcomes John Westbrook

• June 1, 2005
• 800 a.m. - 500 p.m.
• National Press Club
• 529 14th St. NW, 13th Floor - Washington, DC
  20045

115
Connecting Dissemination Activities with
Projected Outcomes

• Activities
• Proposed Activities by Area
• Research
• Development
• Demonstration
• Dissemination
• Utilization
• Technical Assistance

116
Connecting Dissemination Activities with
Projected Outcomes

• Activities (continued)
• Coordinate all activities to address outcomes.
• Dissemination activities should promote outcome
  accomplishments.
• Product orientation (newsletter) - nonstrategic
• Activity orientation (web site) - nonstrategic
• Outcome orientation - strategic
• Outcome planning increases efficiency,
  effectiveness, and relevance.

117
Connecting Dissemination Activities with
Projected Outcomes

• Outputs
• Generic outputs.
• Individualized, tailored outputs.
• Clarity of relationship among
• Output
• Target system element
• Outcome

118
Connecting Dissemination Activities with
Projected Outcomes

• Target Systems
• Relationship to outputs and outcomes
• Specificity by
• Type(s) of systems
• Geographic location vs. universe
• Sub-groups within target systems
• Whole or percentage of the whole
• Service (receiving services in last 12 months)

119
Connecting Dissemination Activities with
Projected Outcomes

• Target Systems (continued)
• Age
• Gender
• Educational level
• SES
• Primary/secondary conditions
• Race/ethnicity

120
Connecting Dissemination Activities with
Projected Outcomes

• Short-term Outcomes
• Need for Specificity in Outcome Statements
• What will change?
• Who/what will manifest the change?
• What is the consistency of the change
• All members of the target system?
• In all cases?
• What outputs do the outcomes derive from?

121
Connecting Dissemination Activities with
Projected Outcomes

• Short-term Outcomes (continued)
• What is the change in
• Degree of consistency (4 out of 5 times)
• Degree of accuracy
• Degree of frequency (used 80 of time)
• Degree of time
• Relationship to the target system and outputs

122
Connecting Dissemination Activities with
Projected Outcomes

• Mid-term Outcomes
• Specificity
• Linkage to short-term outcomes
• Research-based Interventions become acted upon
• Within specific environments
• By specific practitioners
• Data and Data Sources
• Feasible
• Not comprehensive
• Suggestive of movement as change

123
Dissemination and Outcomes Planning Workshop
From Activity to Outcome Please contact us with
any additional question(s) or technical
assistance needs. Contact NCDDR for
dissemination questions or needs at 800-266-1832,
by fax [phone]), by e-mail
ncddr_at_ncddr.org Contact RUSH for logic models
and/or outcomes planning questions or needs at
800-761-RUSH (7874), by fax [phone]), by
e-mail rush_at_sedl.org