Evaluating Online Courses

1
Evaluating Online Courses
2
What Is Evaluation?

• Per Michael Scriven (1991)
• Process of determining the merit, worth, or
  value of something
• (Process that) leads to evaluative conclusions
• good/bad
• works/ doesnt work
• high/low quality

3
Course Evaluation

• Is the course working as intended?
• Does the course meet its objectives?
• Is the online learning environment as effective
  as it can be?
• Is the course manageable?
• Are learners satisfied?

4
Why Evaluate?

• To improve instructional quality
• To motivate learners
• To motivate instructors

Scriven, M. (1991). Evaluation Thesaurus, 4th
Ed. Sage Publications London
5
Conducted by Administration vs. Conducted by
Instructor

• Administration-Led
• Focuses on learner satisfaction broader issues
  of program
• Access, Course Management System
• University service registration, instructor
• Minimal instructor feedback
• Instructor-Led
• Focuses on course components/design
• Student feedback performance
• Instructor reflection

6
Dick and Carey Design Model
7
Gerlach and Ely Design Model
Determine Strategies
Specify Content
Org. Groups
Allocate Time
Specify Object.
Allocate Space
Select Resources
8
In Reality, Evaluation is.

• Done continuously
• once the course begins.

9
Recommendation

• Formative course evaluation conducted by the
  Instructor.

10
It needs to be

• Informal and Reflective, as opposed to Reactive.

11
Interim vs. Terminal

• Interim
• Evaluating at various points in the course
• Not TOO many points ?
• 2-3 times per semester, mid-term at the minimum
• May affect future weeks of course
• Terminal
• Conducted at the end of the course
• Affects next course offering
• May get more input on interim evaluations than
  terminal evaluation

12
Evaluation Framework
13
AEIOU Course Evaluation(Based on AEIOU Program
Evaluation Method)

• A Access Was the course easily and
  consistently accessible?
• E Effectiveness - How well was the course
  designed/organized? Objectives instructions
  clear? Assessment matched objectives? Level of
  student performance?
• I Interaction Was there a good balance of
  interaction? Level of learner participation in
  discussions? Level of peer support, instructor
  support?
• O Obstacles What hindered the potential
  effectiveness of the course?
• U Unanticipated Consequences What happened
  that was not expected --- both from learner and
  instructor perspective?

14
Evaluation Considerations

• Environment
• Course structure
• Access reliability
• Instructional Strategy
• Objectives
• Activities
• Interaction/Communication
• Assessment

15
Evaluation Considerations

• Learner
• Performance
• Motivation
• Satisfaction

• Instructor
• Performance
• Motivation
• Satisfaction

16
Better to Design the Quality In

• Than to evaluate the problems out

17
Communication Tools for Evaluation

• Email
• Survey attached
• Quick/one question surveys
• Online surveys
• Quick feedback on a new technique What did you
  think? Check the box..
• Discussion board (evaluation summaries)
• Fax
• For handwriters

18
Encouraging Completion Interim

• Make it as short/easy as possible
• Students need to trust interim wont affect their
  grade stay away from instructor components
  (i.e. feedback)
• Evals could be submitted to peer/TA who
  summarizes and forwards to instructor or posts on
  course site
• Could be incremental questioning
• Every student to respond at least once

19
Encouraging Completion - Final

• One peer holds evals until after final grades
  are submitted
• Could be team leaders
• Make it the final assignment and student would
  have to turn it in (to peer/TA) before final
  grade was submitted

20
Summary

• Clear, easy, non-threatening to maximize
  feedback
• Focus on key components of your course and key
  participants in the course
• Include YOUR thoughts as well as your students