Principles of Educational Evaluation

1
Principles of Educational Evaluation

• Dr E S J Prabhu Kiran
• Professor HOD
• Department of Anatomy
• Father Muller Homoeopathic Medical College
• Mangalore

2
What is Evaluation?

• Evaluation is the process of determining whether
  predetermined educational objectives have been
  achieved

3
Evaluation involves 2 things

• Educational Objectives
• An appropriate measuring equipment

4

• What makes the
• Evaluation complete?

5

• Measurement
• Value Judgment
• (Objectives achieved?)
• (Are the results desired?)

Qualitative
Quantitative
6
Aims of Evaluation

• Formative Evaluation
• Continued monitoring of learning activities to
  give a feed back
• Summative Evaluation
• To certify competence to complete a course

7
Formative Evaluation

• To determine how much and how well students have
  learned, to serve as a feedback to students and
  teachers.
• Helpful in detection of learning difficulties
• Gives a feedback on the efficacy of their
  teaching.
• Should be criterion referenced (Individual based)

8
Summative Evaluation

• To know whether the student is competent enough
  for certification.
• Little scope of feedback to the learner or to the
  teacher.

Should be criterion based but in reality they
are not
9
Characteristics of Evaluation

• Relevance
• appropriateness in the context of needs of the
  society or the system
• Validity
• Is evaluation procedure effective?
• Reliability
• Is it consistent?
• Objectivity
• Can the evaluation of two examiners match?
• Feasibility
• Can the process be implemented in practice?

10
Relevance

• Should be obvious to both the student and the
  teacher
• Relevance refers to the appropriateness of the
  process of evaluation with reference to the jobs
  to be performed by the student after
  qualification and therefore it should reflect the
  health needs of the society.

11
Validity

• In choosing the type of evaluation it is
  important to determine which learning outcome is
  to be measured.
• Written test are valid for measuring the
  knowledge (conginitive domain) but not for
  attidude (affective domain) or Skills
  (Psychomotor domain)

12
Types of Validity

• Content validity
• Criterion validity
• Construct validity

13
Content Validity

• Does the test measure what was intended to be
  taught?
• Content validity can be ensured by the following
  steps
• Prepare a list of content matter or behaviour
  changes to be tested
• Assign weightage to above based on relative
  importance
• Prepare a table of specifications
• Create test according to table of specifications.

14
Criterion Validity

• Do persons who are taught do better than those
  who are not taught.
• Criterion validity is a measure of
• how well the test performance reflects either
  concurrent performance of the same behaviour
  based on some other measuring instrument
• or
• future performance obtained by another test
  given at some future time.

15
Criterion Validity

• Concurrent validity
• Comparision of marks with other marks obtained by
  other measuring instruments at the same time
• Predictive validity
• Comparision with subsequent marks obtained in
  future

16
Construct Validity

• It is a comparison of the test scores with scores
  obtained by other measuring instruments which
  measure a related ability which is usually
  associated with that tested by the current
  measuring instrument.

17
Factors affecting Validity
Test factors
Student factors
18
Factors affecting Validity

• Test factors
• Unclear instructions
• Difficult and ambiguous phrasing of questions
• Poorly constructed items or questions
• Inappropriate level of difficulty
• Inappropriate item or question
• Improper arrangement of items

19
Factors affecting validity

• Test factors
• Identifiable pattern of answers, clues
• Too short a test
• Insufficient time
• Adoption of unfair means by students
• Errors in scoring
• Adverse physical or psychological millieu

20
Factors affecting validity

• Student factors

• Emotional disturbance
• Lack of motivation

21
Reliability

• Reliability is a measure of reproducibility of
  the test.
• Refers to the result of the test and not to the
  instrument it self

22
Reliability

• Measures to improve reliability
• Increase length of test to optimum level
• Use appropriate levels of difficulty and
  discrimination to ensure a widespread of scores
• Maintain conditions of test constant
• Ensure validity of scoring
• Ensure validity of the instrument used.

23
Objectivity

• MCQs
• OSPE/OSCE
• Structuring of questions,
• Preparation of model answers
• Agreeing on the marking scheme
• Valuation by two or more examiners

24
Feasibility

• Factors affecting feasibility
• Time
• Resources
• Availability of an equivalent form of test for
  measuring reliability
• Ease of administration, scoring and interpretation

25
Steps of Evaluation

• Factors to be considered in choosing an
  evaluation tool
• Purpose of evaluation
• Domain to be tested
• Number of students
• Feasibility as regards to resources, time and
  administrative issues
• Ensuring validity and objectivity

26
Flow chart for Evaluation

• Define Objectives
• Provide Teaching / Learning Experience
• Select Measuring Instrument
• Administer Test
• Decide Marking
• Score Test
• Analyse Result
• Take Decision

27
Domain-wise Matrix of Evaluation Methods
28
Domain-wise Matrix of Evaluation Methods
29
(No Transcript)