Title: ..Instead of writing reports or documenting experiences in ways that help us to learn, we spend a gr
1Some thoughts on Monitoring and Evaluation..
2- My brief provocative, challenging..
- Challenges we all face in ME
- Barriers to changing ME systems
- Ideas of potential ways to moving forward
3- ..Instead of writing reports or documenting
experiences in ways that help us to learn.. - we spend a great deal of time trying to please
you and doing what you want - Everjoice Win, Zimbabwean NGO, feminist (If it
doesnt fit in the blue square its out! An open
letter to a donor friend Christine 2004)
4- What comes to mind when we think of
- Monitoring and Evaluation?
- A light-hearted look.
5Data, bureaucracy
Is the monitoring or the evaluation used?
What learning is derived from it?
6Formats/form filling
7Processes that we do to them in order to
provide information upwards
8Preset indicators monitoring against initial
project docs. Monitoring is often about
documenting what we and our partners expect to
happen..
9Other words that spring to mind..
- Measurement
- not everything that can be counted counts
- and not everything that counts can be counted
Albert Einstein - Often NGOs are working on..
- Rights enabling people to access their basic
rights hold decision makers accountable - Control/choice
- Empowerment
- Confidence
- Freedom from abuse..
10Challenges to monitoring and evaluation
- Introducing Learning cycles.
- Local NGO and community learning. Monitoring to
influence change action? - (b) NZ NGO using the learning from
projects/programmes to influence on-going annual
plans and funding decisions. - Consolidated overview of impact?
- Assessments of problems/learning?
- ME influencing funding decisions?
CDRA James Taylor
11- NGO quote..
- ..most of our organizations have effective means
of ignoring findings which challenge us. .. - This suggests that the challenge is as much to
look at how open we are to dealing with difficult
information about our performance, as it is to
develop sophisticated methods. - which produce results which we subsequently
ignore.. - Head of Programmes,
- Oxfam Australia (2005)
- C.Roche Author of Impact Assessment for
Development Agencies (1999)
12- Challenge 2 - Introducing Participation
- Participation is often limited to communities
participating in initial project ideas - V. little participatory monitoring. V. little
participatory assessment of ( and -) changes by
community groups beneficiaries - V. little participatory evaluation (in one
evaluation report participatory processes
limited to consultation with a handful of
beneficiaries) - Often hear the reality at the end evaluation
13Challenge 3 better understanding of impact
- Hear a lot about activities - outputsbutV.
little about impact - Consolidated understanding of how (and if) the
NZ NGO is making a difference - Across the portfolio spending X then are we
making a difference of Y? -
- What is changing in peoples lives
- What are the problems and issues faced?
- What are we achieving against our strategy?
- Do our programmes/projects add up to a good way
of tackling poverty?
14Challenge 4 improving downward accountability
sharing knowledge/power
- Normally accountability systems for upward
reporting? to Trustees, councils, donors
because our system are devised to feed
information upwards WE often fall into the
pitfalls - How often do we invite feedback? Invite
criticism? - How often do we ask what information our partners
or people in communities may want from us? - How often do we (or our partners) ask them what
we could do differently to make their lives
easier? And the work more appropriate? - ..we can only respond to the questions raised if
you can promise that you will not victimize us by
canceling our project.. - (Concerns expressed by CBO member during AA
Kenya PRRP 200)
15What may be the barriers? 4
- The words e.g. Measurement
- ME often tries to simplify complex reality of
social change often preset indicators - Numbers/figures important but dont add up
apples and pears - Life is not linear Whats changed and what is
significant is most interesting if it comes from
beneficiaries themselves..
16Possible barriers
- 2) Our own western conditioning/education our
paradigm - Ask for reports in English (but maybe 2nd or 3rd
language) - Maybe oral traditions (video, most significant
change R.Davies, J.Dart) - Rarely start with their concepts, but ask
them to respond to our questions/expectations
about what should be happening (James Taylor
talks about starting from peoples own
experience) - Reality is often more complex.
17Possible barriers..
- 3) Our own expectations of upward accountability
to our funders to the NZ public.. - Rigour sometimes leads to inflexibility/reducti
onism.. - Their expectations of us (DFID/PPA wanted
fixed indicators rather than indicative
outcomes).
18Possible Barriers
- 4. Our own systemsways of working
- Financial systems Sometimes hard to get
financial staff to work with programme staff
(analysis of achievements v. costs each year).
Sometimes work in silos - Programme staff V busy. Sometimes hard to change
practice. - Often NGOs in NZ often dont have a dedicated
person working on ME and impact assessment - Changing ME systems ways of working takes a
lot of hard work and effort
19Potential ways forwardAssessment of contrasting
paradigms (Chambers Ideas for Devel 2005 Ros
Eyben Andrea Cornwall) Shifts in Development
thinking..
20Shifts in Development thinking..
21Final thoughts Participatory Review and
Reflections can be rigorous AA- Hear the
stories Understanding impact(participatory
Video Upper East Ghana Basket weavers for
fair trade market. - Exposure of the pressure of
the middle men (problems of being paid)-
problems they faced in getting materials for
weaving. - but full descriptions of the freedom
and the confidence that the work given them and
their ability to deal with the middle
men.Vietnam Ms Kieu Thi Tim of An Hai Commune
is a single parent with two girls. She was
formerly houseless and ranked as one of the
poorest in the village. With a small first loan
she built a home, which she repaid in a year.
With a second loan she started a breakfast bar
and began duck rearing. When asked what
difference the loans had made she replied ..I
can not compare the difference look at me
There was a deep sense of satisfaction in her
eyes and lots of confidence in her face.
22- Uganda. Partner organisation working with street
children who tell their stories - A 13 year old girl narrates her story
- When my father died, I resorted to selling
local brew at my sisters house in Jambula zone.
I can not express the magnitude of the physical
and mental torture I have gone through. My sister
did not give me food despite the long hours of
work serving drunkards, who instead of
appreciating my work forced me into sex. I had no
protection from my sister instead, she supported
the men who harassed me because they paid her
whenever they slept with me. - When I got pregnant, my sister threw me out of
her house I then went to my friends who gave me
refuge until I delivered. The night I delivered,
I was all by myself ... I screamed loudly,
awakening the neighbours who found me lying in a
pool of blood. The ordeal left me lame. I do not
know if I will ever walk normally again - Peoples testimonies put in context
- Numbers of street children
- Expenditure of partner NGO
- Long term changes to childrens lives and
prospects.
23- 1. How can we make (more) space for our partners
to tell us about what is really going on? 2.
How can we make (more) space to hear from
beneficiaries about what difference the work is
making to their lives from their own
perspectives? - 3. How can we give peoples analysis and
perspective meaning by adding numbers and
context?