Chapter 5: Expressing and Assessing Program Theory Roberta Ballard

1
Chapter 5 Expressing and Assessing Program
TheoryRoberta Ballard
2
Evaluability Assessment Perspective

• Evaluability assessment is determining if
  preconditions exist for evaluation prior to
  preceding
• Description of program model
• Assessment of definition and evaluability of
  model
• Identification of stakeholder interest and use of
  findings

3
Describing Program Theory

• Basis for formulating and prioritizing evaluation
  questions, designing evaluation research, and
  interpreting evaluation findings
• Falls under various names
• Logic model, program model, outcome line, cause
  map, and action theory

4
Components of Program Theory

• Impact Theory
• Assumptions about the change process and improved
  conditions that are expected to result
• Service utilization plan
• Programs assumptions and expectations
• Organizational plan
• Program resources, personnel, administration, and
  general organization

5
Eliciting Program Theory

• Articulated program theory
• Well understood by staff and stakeholders
• Define boundaries of program
• Implicit program theory
• Presumed and not fully articulated and recorded
• Must be explicated
• New program
• prior practice and research
• Existing program
• Successive approximation

6
Explicating the Program Theory

• Primary sources of information
• Review program documents
• Interviews with program stakeholders
• Site visits and observation
• Social science literature
• Three types of useful information
• Goals and objectives
• Must be integrated into description of program
  theory

7
Contd.

• Program functions, Components, and Activities
• Identified and linked
• Represent elements of program process theory
• Description of program theory must be
  corroborated with realistic picture
• Examination of discrepancy is task of process or
  implementation evaluation

8
Assessing Program Theory

• Outside of evaluability assessment literature,
  little written on assessment
• Certain critical tests to provide assurance
• Relation to Social Needs
• Requires judgment call
• Specificity
• Logic and Plausibility
• Panel of reviewers
• Unstructured and open-ended process

9
Contd.

• Comparison with Research and Practice
• Examine evaluations of programs based on similar
  concepts
• Basic research on social/psychological processes
• Applied research literature
• Via Preliminary Observation
• Provides reality check
• Focus on interactions with production
  expectations and intended outcomes
• Examine circumstances of target population
• Observations and interviews relating to program
  resources and activities

10
Possible Outcomes of Program Theory Assessment

• Program Reconceptualization
• Ambiguity
• Program process theory ill-defined
• Black box evaluation
• Inadequate specification of program impact theory
• Well-defined and well-justified program theory