Monitoring and Evaluation of Knowledge Management

1
Monitoring and Evaluation of Knowledge Management

• Simon Hearn, ODI, s.hearn_at_odi.org.uk
• Ewen LeBorgne, IRC International Water and
  Sanitation Centre, leborgne_at_Irc.nl
• Valerie Brown, Australia National University,
  valeriebrown_at_ozemail.com.au

2
Overview
3

• It is, in fact, nothing short of a miracle that
  the modern methods of instruction have not
  entirely strangled the holy curiosity of
  inquiry.
• - Albert Einstein

4
Definitions
When I use a word, it means just what I choose
it to mean neither more nor less
5
Monitoring and evaluation

• OECD definitions
• Evaluation The systematic and objective
  assessment of an on-going or completed project,
  programme or policy, its design, implementation
  and results.
• Monitoring A continuing function that uses
  systematic collection of data on specified
  indicators to provide indications of the extent
  of progress and achievement of objectives
  (abridged)

6
Monitoring and evaluation

• OK, but... Any definition must recognise
• ME as universal functions, not specialised roles
• Presence of different worldviews
• Validity of evidence from different knowledge
  domains
• The ethical basis for the desired social change
• The importance of the unexpected and the
  intangible

7
Knowledge

• Objective and subjective
• Individual and society
• Facts and values
• Tacit and implcit
• E.g. Western scientific conception of knowledge
  as justified true belief vs African concept of
  Ubuntu

8
Development

• Often conceptualised as a service industry
• Delivery of even basic services (roads,
  sanitation..) requires an understanding of the
  social, political and economic contexts
• Thus, development is more like a knowledge
  industry (Powell 2006)
• But development is more than donor aid and we
  must recognise civic-driven change also

9
Challenges
10
Challenges in ME of KM4D

1. KM4D does not as yet have a well grounded theory
2. Knowledge for development practice is still young
3. KM4D goes beyond what is labeled KM
4. Competing ontological and epistemological
   perspectives (and related knowledge systems)
5. Existing reporting frameworks are designed for a
   service industry rather than a knowledge industry
6. There can be no simple cause-effect relationship
7. KM initiatives often lack explicit linkages to
   individual, specialist, organisational or social
   results
8. Knowledge is not static
9. Lack of methods for interpreting intangibles

11
Signposts
12
1. KM ripple model
Hulsebosch et al (2009)
13
2. The KM Framework
Talisayon (2009)
14
Need a better understanding of what intangibles
are
Value creation through intangibles
Based on Talisayon (2009)
15
Need a better understanding of knowledge
transitions
SECI
Nonaka and Takeuchi (1995)
16
Need a better understanding of how knowledge is
put to use
Knowledge to action cycle
Graham et al (2001)
17
Need a better understanding of organisational
factors affecting knowledge use
The RAPID Framework for Knowledge Strategies
Ramalingam (2005)
18
We need to understand the level of complexity
Cynefin framework
Snowden (2002)
19
Summary a range of perspectives

• Ontological What world-views are reflected in
  the initiative and how do we recognise them?
• Epistemological What are the knowledge domains
  contributing to ME and how do they relate?
• Socio-political Who has a stake in the
  monitoring process and who has power? How can we
  monitor these interdependent relationships?
• Methodological How to choose tools and
  approaches relevant to the parties and processes
  involved?
• Operational How do we organise ME activities
  according to each of the knowledge domains?

20
Your reflections?

• Do you identify with these signposts?
• What signposts do you use?
• How do you see these models supporting your work?

21
Multiple knowledges ME as multiple partners
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
Collective knowledge as a nested set
Individual knowledge
Local knowledge
Specialist knowledge
Organisational knowledge
Holistic knowledge
A collaborative system
26
Port Pirie small town with the biggest
lead smelter in the world
KNOWLEDGES STRUCTURE CONDITIONS
Children diagnosed with lead
INDIVIDUAL
People long resigned to risk
COMMUNITY
Health Centre stays aloof
SPECIALIST
ORGANISATION
Mine muzzles council
Fear for future livelihood
HOLISTIC FOCUS
27
New alliances in Port Pirie
Parent, grandparent
INDIVIDUAL
COMMUNITY
Outrage, political action
SPECIALISTS
Technical skills, advocacy
ORGANISATIONAL
Public/private good
HOLISTIC
Childrens well-being
28

• ME as collective learning
• multiple interests
• multiple knowledges
• collaborative action

29
Next steps - The IKM-E approach- Emergent
questions on the horizon
30
Our approach Multi-evidence based?

• Each knowledge community uses different ME
  criteria, evidence bases, databases for
  judgments...
• Individuals (experiences)
• Communities (observations)
• Experts (practitioner stories)
• Organisations (monitoring reports as stated)
• Holistic thinkers (ideas, forecasts)

31
Our approach Purposes of conducting ME

• Financial accountability
• Operational improvement
• Strategic readjustment
• Capacity strengthening
• Contextual understanding
• Deepening understanding (research)
• Self-auditing
• Advocacy
• Sensitisation
• (From I. Guijts PhD thesis seeking surprise)

32
Our approach KM as collective learning

• Key to nested knowledge cultures
• Individual
• (Local) Community
• Specialised
• Organisational (strategic)
• Holistic

Describe
Develop
Initiative
Design
Do
33
Our approach critical questioning

• A series of questions at each step of the way
• Overall, a sound questioning practice
• And specifically, a guideline to tailor ones
  approach
• What questions to address?
• Who to involve, in what function?
• What tools and methods to choose?
• What lessons to draw from the approach?

34
Our approach A nested iterative inquiry
35
Emergent questions on the horizon

• How would our approach work in practice?
• Specific methods and metrics to go light
• Particularly complexity-focused approaches
• Power vs. collective?

36
What now?
37
IKM-E KMIC IKMEKMIC?

• Avoiding overlaps...
• Connecting KMIC and IKM (blogs...)
• Organising another webinar?
• Identifying different models / approaches?
• Having creative leaps...
• Reviewing the IKM papers?
• Expanding parts of this paper?
• Testing the IKM-E framework (later)?

38
Additional resources

• IKM-Emergent website http//wiki.ikmemergent.net
  
• The giraffe, Working group 3 blog
• Working paper 3 Monitoring and Evaluation in
  Knowledge Management for Development
  http//su.pr/5rqp8c
• Background paper Monitoring and evaluating
  knowledge management strategies
  http//su.pr/28Q9Yu

39
Thank you!