Monitoring vs Evaluation

1
Monitoring vs Evaluation
Monitoring Evaluation
2
WHEN Monitoring and Evaluation can Take
Place Evaluation
2001 Start
Beginning
Formative
During
Summative
End
2007 End
3
Monitoring

• Monitoring only takes place during

4
(No Transcript)
5
Monitoring (Car Journey)

• Keep Check on
• Fuel Gauge
• Temperature Gauge
• Engine Noise
• Tyre Pressure
• Speedometer
• Clock

6
(possibleconfusionwith evaluation)

• Monitoring may involve
• some diagnosis and remedial action
• e.g.
• Luggage loose make secure
• Fuel low fill up tank
• Behind schedule increase speed

7
Monitoring is all about . . . KEEPING TO
PLAN (It is not questioning the plan)
8
Evaluation QUESTIONS THE PLAN (in some way)
9
For example Is it the best way? Will it be
achieved? What can we change to increase
success? What have we learned? Is it worth
doing?
10
Motel Stop Formative Evaluation Are we
going to get there on time?
Becomes Evaluation)
Fuel leak will it be disastrous?
Traffic jam should we abandon car and walk?
11
Other Evaluation questions Is there a better
way to do the journey (e.g. air/train)? How long
will the car last? What maintenance will it
require, etc., etc. Is the journey worth doing?
12
Good Bread ! ?
Monitoring ?
13
Inputs
Process
Outputs

• Flour
• Water
• Yeast
• Sugar
• Salt
• Oil

• Time
• Temperature
• Size of Oven
• Shape of Loaf
• Position in Oven

• Taste
• Delicious ! ?
• Too sweet ! ?
• Too salty ! ?
• Nutritious ! ?

14
Good Schools
Monitoring ?
15

• Buildings
• Equipment
• Students
• Teachers
• Etc

• Teaching Methods
• Motivation of Students
• No. of Teaching Hours
• Time on Task
• Etc

• No. of Graduates
• Exam Results
• Jobs/Earnings
• Etc

16
Outputs

• Inputs and Process are only a means to an end
• We should monitor inputs, processes and outputs
• Outputs have greatest importance
• Monitoring should also be used to help us
  understand

relationships between inputs and outputs
e.g. When an input changes, what effect does
this have on outputs? Which is the most important
input? Etc.
17
Inputs
Process
Outputs

• Flour
• Water
• Yeast
• Sugar
• Salt
• Oil

• Time
• Temperature
• Size of Oven
• Shape of Loaf
• Position in Oven

• Taste
• Delicious ! ?
• Too sweet ! ?
• Too salty ! ?
• Nutritious ! ?

• Flour

• Taste

18

• Buildings
• Equipment
• Students
• Teachers
• Etc

• Teaching Methods
• Motivation of Students
• No. of Teaching Hours
• Time on Task
• Etc

• No. of Graduates
• Exam Results
• Jobs/Earnings
• Etc

• Exam Results

• No. of Teaching Hours

19
COLLECTING AND INTERPRETING DATA
20

• There should be good reasons for collecting ME
  data.
• Known to all involved.

21
HOW DO WE INTERPRET DATA?

• Have a reason for collecting data!
• Know what we are looking for!
• Form hypotheses/theories (Use the data to test
  them)
• Look for patterns and trends

22
DESIGNING MONITORING FORMS/QUESTIONNAIRES

• Know the REASON for Every Question

• What are we trying to find out?

• Visualise a PICTURE of
• possible response patterns

• This Picture might help us to
• REFORMULATE the question in a
• better way

23
EXAMPLES OF DATA INTERPRETATION AND METHODS
24
Boys as a Percentage of Enrolment
Spotting TRENDS and Making PROJECTIONS is an
important part of ME!
Girls as a Percentage of Enrolment
25
(No Transcript)
26
(No Transcript)
27
INTERPRETING DATA
What can we learn?
How can A help B?
28
INTERPRETING DATA
Sometimes it might be desirable to move towards
the NORM
For Example Class Size, or Teacher/Pupil Ratio
BEFORE
29
AFTER
30
INTERPRETING DATA
Sometimes it might be desirable to move towards
the Strongest Performers
For Example Examination Results
BEFORE
31
AFTER
32
Often, it is useful to sort data into Three
Categories
1
STRONG
Why?
2
AVERAGE
LESSONS!
3
WEAK
Why?
33
METHODS OF COLLECTING DATA

• Reports/Tables
• Inspection/Field visits
• Surveys/Questionnaires/Interviews
• Participatory Meetings
• Participatory Techniques (e.g SWOT Analysis,
  Matrix Ranking)

34
MODES
External
Internal
Formal
Informal
Announced
Unannounced
Written
Oral
Individuals
Groups
Participative
Non-Participative
During
After
35
TRIANGULATION OF DATA
36
INTERNAL OR EXTERNAL?
Internal

• A better understanding

• Vested interest in positive reporting

Option ? Triangulate using both!
37
The DOG (A) wags its TAIL (B) A influences
B
What is IMPORTANT (A) is MEASURED (B) A
influences B
But What if
The TAIL (B) wags the DOG (A) B influences
A
What is MEASURED (B) becomes IMPORTANT (A) B
influences A
GOOD! or BAD?
38
Both Possible
Some People Argue Not everything that is
important is measurable!
We can make things important (if we want!) by
attaching importance to their measurement!
A Useful Opportunity to bring about CHANGE!
39
M E is a Four-Stage Process
1
Gather Data
2
Process Data e.g. graphs/tables
3
Interpret Data e.g. trends/patterns
4
Use Findings e.g. to change practice, inform
policy, etc
Real M E only occurs if all four stages are
carried through!
40
FOUR KEY QUESTIONS

• Where are we now?
• Project Baseline Data
• Where do we want to be?
• Objectives
• What do we have to do to get there?
• Activities
• How will we know when we have arrived?
• Performance Indicators

A
B
41
Strong
B
A
Project Baseline
Weak
42
Strong
Indicator Better Usage of Capacity
Minimum 85
Big Improvement!
65
A
Baseline
Weak
2003
2002
2008
43
Strong
B
Control Group
A
Project Baseline
Weak
44
ACCESS TO INFORMATION

• National Resource Database?
• Website?
• Who needs information? What information?
• What is available now? What is not available?