Instructional Design Evaluation, Assessment

1
Instructional DesignEvaluation, Assessment
Design A Discussion (EDER 673 L.91 )From
Calgary With Asst. Professor Eugene G.
Kowchmarch 27(An Synchronous Meeting using
WebCT discussion Thread and WWW Course Home
Page Material)
2
Assessment Instruments Assessing learners

• Criterion Referenced
• Pretest Placement entry level skills
• Readiness Questions
• Placement Questions
• Posttest helps us identify what did not work for
  this learner
• ---------------------
• To make these tests, we need a lot of info
• List of task based behavior objectives or
• Type of learning (verbal, etc, ) and
• Type of conditions for learning (Gagne) and
  events for learning
• Bad criteria test subjects
• Writing a paragraphaffective evaluations..

3
Pretest Method of Inquiry for Placement
From Driscoll, M. (2000). Psychology for
Instruction, 2nd Ed.
Placement Learner Readiness Assessment
(to determine entry performance)
Readiness ConcernDo the students have
the Prerequisite skills?
Placement Concern Have the students
already Achieved the intended Outcomes?
yes
no
yes
no
Provide readiness experiences
Proceed with The instruction
Advance Students To a higher level
Discuss Math vs. Social Studies (criterion?)
4
Formative and Summative Assessments( of your
Design)

• Formative methodology (can be done in order)
• Education Context 1 to 1 evaluation (designer
  learner)
• Questionnaires, interviews - check learning time,
  outcomes
• Focus group
• Nets out the instructor
• Check the effectiveness of instruction
  (viability, attitudes of instr.)
• Field Trial
• Observe a full instruction event with the 2x
  modified course materials and design test for
  achievement.

5
Formative and Summative Assessments( of your
Design)

• Summative Assessment methodology
• Expert Judgment (assess the instructor)
• Congruence Can the instructor meet the needs of
  the organization?
• Content Are the materials complete, updated?
• Instructor followed the design?
• Utility of instructor?
• Design Are the principles of learning,
  instruction and motivation in the materials?
  Instruction?
• Field Trial
• Outcomes Analysis impact to learner on job, in
  school
• Management Instructor, supervisor performance
  test

6
Summative Learner Assmt. Method
Summative Assessment (to monitor learning
progress)
Have the students achieved The intended
outcomes Of instruction?
no
yes
Provide additional learning experiences
Assign grades to certify mastery
Evaluate the effectiveness of the instruction
Discuss
7
Assessment Planning (Magliaro from notes)

• Assessment Learner performance
• Evaluation judging learner decisions
• Program Evaluation Appraising program success
• Assessment Possibilities
• Portfolios
• Physical
• Electronic
• Communicate task and knowledge performance
• Time consuming for all!
• Projects
• Demonstrate action / processes / competencies
  (training)
• Address problem solving (education)
• Require relevant subject matter!

8
Assessment Planning (Magliaro from notes)

• 5 Step assessment process
• Identify goals and objectives from Needs Assmt
• Match goals objectives with assmt tools
• Choose instructional method
• Design tasks and scoring methods (match)
• Think through the implications.
• Guidelines
• Start small
• Develop rubrics
• Communicate purpose of assessment
• Involve participants
• Keep going - recursive design
• Look for blips - unfairness, odd results

9
PostTest Method of Inquiry for Placement
The instructional role of Formative Assessment
During Instruction
(to monitor learning progress)
Are the students achieving the intended Learning
outcomes?
no
yes
Provide Group or Individual remediation
Provide feedback to reinforce learning
Proceed with The planned instruction
DIAGNOSTIC ASSESSMENT (to study persistent
difficulties
Discuss
10
Recall The one and the many
11
Norm
The Distribution Curve Defining Normal
Relative ranking, describes what cannot be
done And what can be done..
12
Great questions
Designers Ask..

• "Can and should instructional designers place
  importance
• on process instead of product? If yes, is this
• desirable and reversible? Discuss"

13
Great questions
Designers Ask..

• Should and can evaluation be both internal and
• external?
• Should and can the evaluator personalize his
• or her findings and put those results into a
  cultural
• perspective? Discuss."

14
Designers Ask..

• 1) There is the sense that "outside the company"
• designers are not as biased as "backyard"
  designers. I
• wonder whether this is in fact true, or whether
  their
• biases are simply different. The "outside"
  designers
• could have biases that are harmful or just as
  blinding
• as "in house" people's.

15
Evaluation/Assessment
Designers Ask..

• There seems to be a belief that objectivity is
• somehow better than subjectivity, and I wonder
  about
• that too. Sometimes, a subjective approach may
  be
• called for. For example, in evaluating a Native
• education program, would it be effective to use a
  white
• person? Maybe.
• Then again, maybe not. Maybe the white
• person would have to spend so much time and
  effort
• learning first about the culture that her work on
  actual
• evaluation would become miniscule in comparison.

16
(No Transcript)