Evaluation Theory

1

• Evaluation TheoryProgram Theory

By K.S. Calbick, B. Tech., MRM.Prepared for REM
356 Management Institutions,September 27, 2004
2
Goals

• 1. Review Evaluation Theory
• 2. Develop Program Theory
• 3. Discuss Evaluation Research

3
Key Points

• Evaluation Definition
• Evaluation Components
• Evaluation Types
• Program Theory
• Evaluation Research
• Case Studies
• Program Theory Application

4
What is Evaluation ?

• The utilization of scientific research methods
  and techniques (Suchman 1967 7)
• Assists sensemaking about policies and programs
  through the conduct of systematic inquiry that
  describes and explains the policies and
  programs operations, effects, justifications,
  and social implications (Mark et al. 2000 3)

5
What is Evaluation ?

• Program evaluation is the use of social research
  procedures to systematically investigate the
  effectiveness of social intervention programs
  that is (sic) adapted to their political and
  organizational environments and designed to
  inform social action in ways to improve social
  conditions (Rossi et al. 1999 20)
• Evaluation is the systematic assessment of the
  operation and/or the outcomes of program or
  policy, compared to a set of explicit or implicit
  standards, as a means of contributing to the
  improvement of the program or policy (Weiss 1998
  4)

6
Components of Evaluation
Source Calbick 2003
7
Types of Evaluations
Source Calbick 2003
8
Components of Program Theory
Source Rossi et al. 1999
9
Overview of Program Theory
Source Rossi et al. 1999
10
Evaluation Plan
11
Bay Conservation Development Commission
12
Bay Conservation Development Commission

• Created in 1965 with the McAteer-Petris Act
• First coastal management agency in the U.S.
• In 1968, commission adopted the First San
  Francisco Bay Plan
• The executive director participated in this study

13
Fraser Basin Council
14
Fraser Basin Council

• Established in 1997, the council is a
  not-for-profit, charitable organization
• FBCs vision, principles and goals are
  articulated in their Charter for Sustainability
• FBC provides a forum for stakeholders to develop
  consensus-based plans
• The executive director and a project coordinator
  participated in this study

15
Land Conservation Development Commission
16
Land Conservation Development Commission

• In 1973, Senate Bill 100 established a statewide
  program for land use planning
• A citizens board, LCDC, was created with an
  administrative arm, Department of Land
  Conservation and Development
• LCDC ensures all Oregonian cities and counties
  have a comprehensive land use plan
• A senior planner assigned by the executive
  director participated in the study

17
Puget Sound Water Quality Action Team
18
Puget Sound Water Quality Action Team

• In 1985, the Puget Sound Water Quality Authority
  was created
• In 1987, the Authority developed the first Puget
  Sound Water Quality Management Plan
• In 1996, the Authority was transformed into the
  Action Team
• An ambitious program of public participation and
  stakeholder involvement
• The policy director participated in this study

19
U.S.D.A. Forest Service
20
U.S.D.A. Forest Service

• In 1905, Congress established the forest service
• The forest service is responsible for managing
  192-million acres in 42 states, the Virgin
  Islands, and Puerto Rico
• This land is composed of 155 national forests and
  20 national grasslands
• Natural resources are managed from an ideology of
  stewardship and public service
• The director of strategic planning and resource
  assessment participated in this study

21
FBCs Program Characterization
22
Impact Theory for PSWQAT
Source Calbick 2003
23
Impact Theory for FBC
Source Calbick 2003
24
Forest Services Organizational Plan
Source Calbick 2003
25
A Service Utilization Plan
Source Following Rossi et al. 1999
26
Questions
27
BCDCs Impact Theory
Source Calbick 2003
28
BCDCs Organizational Plan
Source Calbick 2003
29
FBCs Organizational Plan
Source Calbick 2003
30
LCDCs Impact Theory
Source Calbick 2003
31
LCDCs Organizational Plan
Source Calbick 2003
32
Forest Services Impact Theory
Source Calbick 2003
33
PSWQATs Organizational Plan
Source Calbick 2003