Online Assessment Techniques

1
Part I The 3 Ts of Online Assessment Tools,
Techniques, and (Saving) Time
Curtis J. Bonk, Professor, Indiana
University President, CourseShare http//php.india
na.edu/cjbonk cjbonk_at_indiana.edu Vanessa Paz
Dennen, Assistant Professor Florida State
University http//www.vanessadennen.com,
vdennen_at_fsu.edu
2
Session Objectives

• Detail online assessment techniques
• Discuss how to match learning activities with
  learner assessments
• Examine instructor time and comfort issues
• Discuss ways to limit and detect cheating and
  plagiarism
• Document online tools and resources for
  assessment

3
Online Assessment Techniques

• (with some time-saving
• tips added in)

4
Is this motivating? How would you feel?

• You take an online class.
• You read some Web pages.
• Maybe you watch some videos or hear some audio
  clips.
• Maybe you ponder some study review questions.
• You take a multiple choice test online.
• You receive an automated score on the test.
• Class is over.

5
How about this scenario?

• You take an online class.
• You meet your fellow students on the d-board.
• You read some materials. You find and share some
  materials too.
• You participate in some discussions of course
  concepts.
• You take a multiple choice test.
• You receive an automated score on the test.
• Class is over.

6
Commentary on Scenario 1

• No interaction with peers.
• Students dont feel missed if they dont
  participate.
• Not clear why course is online (except perhaps
  for media elements).
• Potential for immediate feedback is nice -- but
  assessment format is limiting.

7
Commentary on Scenario 2

• Interaction with peers is great. Serves as a
  motivator.
• Community is likely to develop.
• Students will feel involved and important if they
  share examples and resources.
• Assessment format may not be well aligned given
  the activities.
• Class lacks closure in a manner appropriate to
  the activities.

8
Assessment and Learning

• Course objectives, activities, and assessments
  should be in alignment
• This tends to be an issue in courses regardless
  of medium.
• Example
• In class students conduct a debate
• Students are tested on their ability to recall
  facts

9
Mis-aligned Online Learning and Assessment

• A not-uncommon scenario
• Discussion is used as a learning activity
• Students are required to participate
• Participation is noted by how many messages were
  composed by a student
• But does this method measure learning?

10
Common Online Assessment Complaints

• Instructor perspective
• Theres too much to assess!
• I dont know what activities to assess!
• I dont know if students really are ready for the
  test!
• How do I know the student actually did the
  work/took the test?

11
Common Online Assessment Complaints

• Student perspective
• If theyre supposed to discuss, why doesnt that
  count as part of their grade?
• If theyre just supposed to do something, why
  does quality matter?
• I just got a number, no feedback.
• I didnt get participation feedback.

12
The Feedback Issue

• Students participating in online activities look
  for feedback to know
• A. the instructor is reading their contributions
• B. their participation is valued
• C. their participation is adequate, in terms of
  quality and quantity
• Feedback need not be individualized to be
  effective
• Whole class commentary provided on a regular
  basis was found to be just as satisfactory from
  the student point of view (Dennen, 2001)

13
The Assessment Issue

• Often, online activities go unassessed
• Add-on syndrome Adding an online activity to a
  previously designed class because it sounds like
  a good idea

14
The Assessment Issue

• Students are more likely to participate when then
  know there is impact on their grade
• Direct impact graded on participation (quality,
  quantity or both)
• Indirect impact participation should bolster
  performance on other assessments
• Students quickly become aware if an online
  activity is not related to assessed learning
  objectives

15
The Assessment Issue, Cont.

• Sometimes the wrong things are assessed
• Examples
• Assessing students online moderation skills when
  the course topic/learning objectives have nothing
  to do with online moderation
• Assessing quantity of participation, but not
  quality
• which, granted, is easier, but encourages sloppy
  message posting rather than thoughtful learning
  dialogues

16
Blooms Revised Taxonomy

• A useful tool for checking alignment
• Also great to guide your course design!

17
To Find Out More

• A Taxonomy
• For Learning
• Teaching and
• Assessing
• By Anderson
• And Krathwohl

18
Assessment Techniques

• Options
• Formative or Summative
• Student-led (Self or Peer) or Teacher-led
• Public or Private
• Process or Product
• Other issues to consider
• Objective or Interpretive
• Rubric-based or Wholistic

19
Formative vs. Summative Assessments

• Formative
• Alleviate student anxieties re expectations
• Seem especially high in online classes
• Encourage working toward mastery
• Can be informal

• Summative
• Used for students grade
• Assumed to be best effort

20
Example Online Formative Assessment

• Paper draft discussion forum
• Start a discussion forum for papers-in-progress
• Have each student start a thread and post
  elements of their papers as they complete them
  (e.g., topic, major claims, research sources)
• A schedule for each element is useful
• Monitor and provide feedback

21
Student vs. Teacher Led

• Teacher-led
• Traditional assessment
• Most often summative

• Student-led
• Students may assess self or peers
• May be formative or summative
• Can greatly relieve instructor burden
• Students reinforce concepts through feedback
  process

22
Examples Online Self-Assessment

• Self-tests Use test tool to create self-tests
  (multiple choice, true false)
• May wish to track student efforts
• Can incentivize use (essentially, use as a
  learning tool)
• Reflection papers Have students submit brief,
  focused papers expressing the strengths and
  assessments of their assignment(s)

23
Example Online Peer Assessment (Formative)

• Feedback groups Assign students in groups to
  provide formative feedback on projects and papers
• Often raises quality of assignments
• Need a structure with clear deadlines
• Need prompts and models to guide students
• May wish to assess feedback process/contributions

24
Example Online Peer Assessment (Summative)

• Conference Presentations Have students present
  their work and ask questions/provide feedback to
  others.
• In d-board, have students attach papers to
  messages post a message with a synopsis or
  attach a powerpoint presentation
• Each student/team should have their own thread
• Feedback should occur during a defined period of
  time.
• May consider allowing students to rate
  assignments on certain dimensions

25
Public vs. Private

• Private
• Work is submitted to the teacher only
• Entire burden of feedback is on teacher
• Important if assessing at fact level

• Public
• Peers can see each others work (either in
  process or completed)
• Peers may comment on each others work
• Often increases quality of work submitted

26
Process vs. Product

• Product
• The end deliverable
• Look for polish, accuracy

• Process
• How the student got there
• Look for thoughtfulness of approach, intent

27
Assessing Process

• Easy to do
• Many technology tools will archive student
  work/interactions
• Students create a document trail in process
• Helps students develop metacognitive knowledge
• Instructors structure/model/encourage productive
  work processes
• Students learn how to manage their own work
  processes

28
Why Assess Process?

• For the instructor
• Provides formative feedback on course (e.g.,
  helps gather data about why students have
  difficulty with product-oriented assessments)
• Clarifies who is doing most work in small group
  assignments
• Helps prevent cheating

29
Why Assess Process?

• For the student
• Typically improves the quality of their products
• Helps them develop productive work processes
• Puts on a schedule
• Shows that you care about individual growth

30
Assessment Project Cycle

• From Classroom Assessment Techniques by Angelo
  Cross (1993)
• Step 1 Plan
• Choose class
• Focus on assessable question
• Design project to answer question

31
Assessment Project Cycle 2

• Step 2 Implement
• Teach target lesson
• Collect assessment data
• Analyze data
• Step 3
• Interpret results
• Communicate results
• Evaluate assessment project

32
I. Term Papers

• How to do online
• Have students each start their own thread and
  post topic of interest
• Peers and instructors give feedback
• Students post thesis statements, research
  sources, etc., with iterations of feedback
• Final paper is posted

33
Term Paper Assessments

• Product the paper
• Process quality and timeliness of student work
  from time when paper is assigned
• Process quality and timeliness of feedback
  provided to peers
• Process responsiveness to feedback received from
  instructor and peers

34
II. Discussion Assignments

• 1. Chain of thought
• Have students develop a solution to a problem
• Have students indicate what led them to a
  particular conclusion, method or approach
• Can be done in a discussion board

35
Discussion Assignments

• 2. Theory to Practice
• Have students match up theories you are learning
  about to actual problems
• Present students with problems and have them
  explain what theories they would use to solve
  these problems and how they would approach it
• Debrief the assignment

36
Discussion Assignment

• 3. Synthesizer (i.e., wrapper)
• Have students take roles being the weekly
  synthesizer of class discussion
• Add a meta level in which students narrate
  their own experiences while reading the weekly
  discussion
• Reflect on how life relates to discussion

37
III. Group Projects

• Tools used
• Chat brainstorming ideas, making group
  decisions, regular way to feel connected (should
  be archived)
• Discussion board commenting on drafts
• E-mail quick feedback
• File exchange sharing project files
• MS Word Track changes

38
Group Project Assessments

• Product project files that are turned in
• Process online archive demonstrating
• Who contributed what
• Who provided peer feedback
• Who worked in a timely manner
• How collaborative a group was
• Process peer ratings
• Process interim instructor consultations

39
Group Project Assignments

• 1. Work Plans
• Have students develop a plan of work for their
  project
• Make them outline topic, schedule, resources
  needed, division of labor and anticipated form of
  final deliverables
• At end of project, have students evaluate how
  well they followed their own plan and how useful
  it was

40
Project Assignments

• 2. Research Trail
• Have students document the steps they took in the
  research process and the results
• Ask for a brief reflection on how effective their
  process was and what they might change the next
  time

41
Project Assignments

• 3. Process Presentations
• Have students focus on their process as well as
  their product in class presentations
• To maintain focus, ask them to share 3 main
  lessons learned
• Might ask for some process documents to be
  shared, like an early draft

42
Project Assignments

• 4. Design Journal
• Have students maintain a journal of all ideas
  related to their project
• Encourage sketches, lists, organizational charts,
  etc.
• Require journals to be turned in with final
  projects

43
IV. Reflection Assignments

• Have students keep a weekly journal of their
  thoughts on readings and course content AND
  real-world related instances that they noticed
• May make these public, with each student having
  their own discussion thread

44
Making it Happen

• Learners need to see that process is valuable
• Model appropriate processes
• Provide students with scaffolding (guide sheets)
  to structure their processes
• Give students feedback on their process
• Require students to reflect on their processes
• Grade students on process

45
Vanessas TopTime-Saving Tips

• Before you assign it, ask yourself can I
  reasonably assess it?
• Rely on students/peers for providing some
  feedback
• Let students know what to expect up front
• Choose the right tool for the job
• Get to know the editing and commenting features
  of your favorite programs
• Use rubrics!
• I make them in Word, and then while grading I
  highlight or bold the section that applies to the
  students projects. Add a few comments at the
  bottom speedy grading!