PROJECT EVALUATION

1
PROJECT EVALUATION

• JUSTICE G. DJOKOTO
• Ph. D. (cand.), M.Phil, P.G.D.E., B.Sc.
• Pan African Institute for Leadership and
  Governance Studies

2
Overview

• What is it?
• A systematic and objective assessment of an
  ongoing or completed project
• Design
• Implementation
• Results
• Involves gathering, analysing, interpreting and
  reporting information
• Should be based on credible data

3
Purpose

• Learning and improvement
• Accountability
• provide useful feed back to stakeholders
• entrepreneurs,
• sponsors,
• donors,
• client-groups,
• administrators,
• staff,
• and other relevant constituencies

4
Types of Evaluation

• There are several types of evaluation.
• The classification is based on
• purpose of the evaluation,
• methodology,
• timing,
• who is involved in the evaluation
• position of the evaluators.
• Based on purpose (Trochim, 2002)
• formative
• summative

5
Types of Evaluation

• Based on Timing
• Ex-ante evaluation
• Ex-Post evaluation
• Based on position of evaluator
• External evaluation
• Internal evaluation or self-assessment

6
Types of Evaluation

• Exante evaluation
• Conducted before the implementation of a project
  as part of the planning
• Also referred to as appraisal or quality at entry
• Ex-post evaluation
• Conducted after the project is completed
• Used to assess sustainability of project effects,
  impacts
• Identifies factors of success to inform other
  projects

7
Types of Evaluation

• External evaluation
• Initiated and controlled by the donor as part of
  contractual agreement
• Conducted by independent people who are not
  involved in implementation
• Often guided by project staff

8
Types of Evaluation

• Internal or self assessment
• Internally guided reflective processes
• Initiated and controlled by the group for its own
  learning and improvement.
• Sometimes done by consultants who are outsiders
  to the project
• Need to clarify ownership of information before
  the review starts

9
Types of Evaluation

• By methodology employed
• Quantitative
• Qualitative

10
Steps in Managing a Project Evaluation

• Establishing the need for an evaluation
• Initial Planning and Resourcing
• Developing Terms of Reference
• Engaging the Evaluator or Evaluation Team
• Approving the Workplan
• Implementing and Monitoring the Evaluation
• Assessing the Results of the Evaluation
• Developing a Plan for Follow-up

11
Step 1 Establishing the need for an Evaluation

• AGRIS project manager(s) need to clarify the
  purpose of evaluations. E.g.
• - Donor requirement
• Accountability
• Innovation
• Learning and change
• Responding to changed circumstance

12
Step 2 Assessing the ability andreadiness to
evaluate

• Evaluations take up significant time and
  resources
• Need to ensure that the costs are appropriate for
  the
• anticipated benefits. Some considerations
• Importance of the evaluation to AGRIS or the
  donor-does it
• need to take place?
• Ability and readiness of AGRIS and partners to
  engage in
• the evaluation-Is there a reason that the
  evaluation should be postponed? Cancelled?
• Size of the evaluation. Setting the focus and
  scope for the evaluation
• Resourcing the evaluation Money? Technical
  expertise?

13

• Defining scope and size
• Clarify
• if external or internal
• Level of effort and resources required /
  available
• Stakeholder groups to be involved and how.

14

• Full stakeholder desirable, but could be limited
  to the following
• Deciding whether or not to evaluate.
• Defining the type of evaluation, its scope, and
  criteria.
• Defining the evaluation questions, what are the
  key issues to explore in the evaluation?
• Defining evaluation workplan.
• Evaluation activities must be scheduled and fit
  into the stakeholders' agendas.
• Deciding which recommendations to adopt and which
  to reject.
• Disseminating and gathering feedback on the
  results.

15
Providing resources for the Evaluation

• Evaluations require substantial investments of
  financial and human resources.
• Funding source would have been indicated in the
  project document

16
Developing Terms of Reference (TORs)

• TOR are the key guide for an evaluation.
• They should
• clarify reasons for the evaluation
• highlight issues that have become apparent
• indicate the general depth and scope required
• spell out any imperatives for the evaluators
• provide details about methodology, scheduling,
  cost and the qualifications of the members on the
  evaluating teams

17
Developing Terms of Reference (TORs)

• The project manager is responsible for ensuring
  clear and focused TORs
• This is as far as the Manager is responsible for
  development of the TOR

18
Contents of Terms of Reference

• Context for the evaluation
• Rationale or purpose for the evaluation
• Evaluation issues and questions
• Evaluation stakeholders
• Methodology
• Qualifications of evaluators
• Schedule
• Outputs and Deliverables
• Cost
• Action Plan
• Appendices - Evaluation Matrix, Evaluation
  Policy, LFA

19
Engaging the Evaluator or Evaluation Team

• Evaluators can be selected by you, imposed by
  donors or jointly agreed to.
• Which ever it is some guide is useful here
• The appropriate level of technical expertise or
  evaluation expertise
• The previous experience or profile of the
  evaluator
• Suggested profile of a good evaluation team
• Using peers as evaluators
• Roles and responsibilities

20
Reviewing and Approving theWorkplan

• The evaluation work plan is developed by the
  evaluator and the evaluation team
• It should
• provide roadmap for conducting the evaluation
  (interprets TORs)
• include proposed methodology and means of
  analysis
• A poor work plan leads to poor evaluation
• Important that the leadership of the project
  review and approve the evaluation work plan

21
Reviewing and Approving theWorkplan

• Suggested outline of a Workplan
• Introduction - purpose and stakeholders
• Evaluation Questions (framework)
• Methodology (sources, methods)
• Schedule (Gantt chart)
• Resource Allocation and Budget
• Evaluation Team
• Outline of Evaluation Report

22
Implementing and Monitoring theEvaluation Work

• Managers required to facilitate evaluators work
  by
• Supporting field data collection
• Making documents available
• Responding to regular evaluation reports and
  feedback
• Distributing draft reports for comments to
  appropriate partners
• Participating in donor and evaluator meetings
  when requested
• Reviewing drafts of findings and reports and
  providing feedback

23
Different Audiences may haveDifferent Needs

• Internal staff might need a verbal report and a
  memo with key points
• Donors and external stakeholders might need a
  full report
• Ministries might need an abstract
• Public at large might need an abstract of
  findings only
• Know your audience and match your reporting
  approach

24
Effective Communication ofEvaluation Results

• Captures the data in its conclusions
• Speaks in language of users
• Detached, non-possessive stance
• Objective - truth to power, but
• Is pragmatic - goes only as far as the key
  stakeholders will accept

25
Assessing the quality of an evaluationreport and
process

• Meeting needs commissioning managers,
  stakeholders
• Relevant scope
• Suitable methods
• Reliable data
• Sound analysis
• Credible findings
• Impartial conclusions
• Clear reporting

26
Doing It Yourself
27
What to Evaluate

• Outcomes
• Processes

28
Steps In Evaluation

• Planning
• Selecting object (setting objectives)
• Methodology
• Deciding on standards
• Choice of measures
• Data collection
• Data analysis
• Implementing evaluation
• Reporting

29
Thanks for Participation