MULTI CRITERIA EVALUATION OF DISTANCE EDUCATION IMPLEMENTATION MODELS USING ANALYTIC HIERARCHY PROCE

1
MULTI CRITERIA EVALUATION OF DISTANCE EDUCATION
IMPLEMENTATION MODELS USING ANALYTIC HIERARCHY
PROCESS

• Ali Ekrem Ozkul, Zehra Kamisli Ozturk
• Anadolu University, Open Education Faculty,
  Turkey
• aeozkul_at_anadolu.edu.tr, zkamisli_at_anadolu.edu.tr
• Nuray Girginer
• Eskisehir Osmangazi University,
• Business Administration Department, Turkey
• girginer_at_ogu.edu.tr

2
GOAL

• Evaluation of distance education delivery
  modes in implementation model by using a multi
  criteria decision making tool, Analytic Hierarchy
  Process (AHP) for the English Language Teacher
  Training (ELTT) Program at Anadolu University
  Open Education Faculty.

3
English Language Teacher Training Program (ELTT)

• ELTT is 4-year English Language Teacher
  undergraduate education
• Program aims to meet demand English language
  teacher in primary and secondary education in
  Turkey.
• Program in ELTT is a blended learning model.
  Courses in ELTT program offered in the first two
  years are face to face while in the remaining two
  years are through Distance Education

4
Decision Problem

• Determination of the share of different
  delivery models in the implementation model.

5
contrIbutIONs

• To realize group decision making like this multi
  criteria decision problem in distance education
• To suggest a hierarchical approach with four
  levels including tangible and intangible criteria
  to evaluate different delivery modes
• To indicate applicability AHP for this decision
  problem

6
METHEDOLOGY ANALYTIC HIERARCHY PROCESS (AHP)

• AHP has a systematic approach to set priorities
  and trade-off among criteria
• AHP can measure all tangible and intangible
  criteria in the decision process
• AHP is a simple, intuitive approach that can be
  accepted by decision makers
• AHP can easily be used to solve multi-criteria
  decision problems involving multi-actors or group
  decision making with multi-actors

7
STEPS OF AHP

• Define the decision problem and determine its
  objective.
• Define the decision criteria in the form of a
  hierarchy of criteria.
• To make the pair-wise comparisons, structure the
  comparison matrices of criteria and decision
  alternatives.
• Obtain the importance of the criteria and
  sub-criteria by the comparisons.
• Determine the weight of each criterion. By
  hierarchical synthesis, determine the priorities
  of each alternative.

8
Group decIsIon makIng

• There are three basic approaches that a group can
  use to assess weights
• consensus
• vote or compromise
• geometric mean of the individual judgments.
• Geometric mean is the most common approach used
  in AHP with group decision making

9
ELTT

• It is important to determine the weight and
  priorities of each delivery modes (face-to face,
  online etc) to execute the program succesfully
• In the selection of delivery modes for
  implementation phase should be taken into account
  besides the tangible criteria like cost, number
  of personnel and students etc and also intangible
  criteria like interaction, benefit etc.
• These criteria must be considered together and so
  this complex decision problem obligate to use one
  of the multi criteria decision making techniques
  as AHP.

10
Formulating Hierarchy of Delivery Models Ranking

• GOAL Determination of the share of different
  delivery models in the implementation model in
  ELTT
• MAIN CRITERIA AND THEIRS SUB-CRITERIA
• Cost (investment, developing course, operating,
• assessment costs)
• Access (number of enrolled students, status of
  students)
• Staff (number of personnel, status of the
  personnel)
• Interaction (student-student, student-learning
  materials, student-teacher)
• Media of Instruction (design, production,
  edition, control)
• Benefit (experience, number of graduates, pioner
  in ELTT)
• ALTERNATIVES
• Traditional Delivery Model (T-ELTT)
• Videoconferencing based Delivery Model (V-ELTT)
• Web based Delivery Model (W-ELTT)

11
Hierarchy of Delivery Models Ranking
12
Obtaining Pair-wise Comparison Matrices

• For the pair-wise comparisons, a questionnaire
  was prepared.
• While taking the judgment for each expert (There
  are three experts), interviews were made
  separately each other, by using questionnaire
  technique.
• The scale that takes integer values between 1 and
  9 were used in the technique recommended by
  Saaty. The valuation scales in the pair-wise
  comparisons are those, where 1 is equal
  importance, 3 is moderate importance, 5 is strong
  importance, 7 is very strong or demonstrated
  importance, and 9 is extreme importance. Even
  numbered values will fall in between importance
  levels. Pair-wise comparisons were based on
  upper level control criteria

13
Obtaining Pair-wise Comparison Matrices

• First of all, weights of main criteria must be
  determined first. For this reason, the experts
  made their pair-wise comparisons about main
  criteria and notified their judgments according
  to overall goal.
• The paired comparison matrix attempts to capture
  the relative dominance of one element over
  another with respect to an attribute that they
  have in common. The questions to ask when
  comparing two criteria are of the following kind
  Of the two criteria being compared, which is
  considered more important according to the goal
  and how much important? And the same question
  type is also used when comparing alternatives
  according to the criteria.
• Three pair-wise comparison matrices belong to
  each expert have been obtained for pair-wise
  comparisons of main criteria. Each expert gives
  his judgments in 25 pair-wise comparison
  matrices.

14
comparIson of maIn crIterIA
15
Obtaining Pair-wise Comparison Matrices

• The experts data based on pair-wise comparison
  matrices designed by Expert Choice were input.
  The software then calculated relative weights for
  the alternatives with respect to the main
  criteria and their sub-criteria. In the result of
  these, 75 pair-wise comparison matrices were
  obtained from judgments of experts for the whole
  model.
• Relative Weights of Main Criteria Due to Experts

16

• Final Relative Weights

17

18
DISCUSSIONS AND CONCLUSIONS

• A blended distance education ELTT program can be
  based on 41.4 face-to face (traditional), 30.9
  web and 27.8 videoconferencing technologies
• Its an expected result that face-to-face
  education takes the biggest ratio according to a
  foreign language teacher training program.
  Especially the assessment of the teacher
  applicants needs a synchronous delivery model.
• Based on the programs curriculum, the lessons
  can be given based on the ratios either yearly or
  per terms

19
DISCUSSIONS AND CONCLUSIONS

• Based on the developments of information
  technology, determination of the weight of
  delivery modes that includes basic elements of
  distance education is an important decision. So
• We propose a decision model for the institutions
  that will develop or implement a distance
  education program.
• The AHP is a powerful multi-criteria method that
  considers both tangible and intangible criteria.
• We integrate different opinions of different
  experts by applying group decision making
• This methodology in our study can be used
  different decision problems in distance education
  management.