Title: Online Assessment Techniques
1Part I The 3 Ts of Online Assessment Tools,
Techniques, and (Saving) Time
Curtis J. Bonk, Professor, Indiana
University President, CourseShare http//php.india
na.edu/cjbonk cjbonk_at_indiana.edu Vanessa Paz
Dennen, Assistant Professor Florida State
University http//www.vanessadennen.com,
vdennen_at_fsu.edu
2Session Objectives
- Detail online assessment techniques
- Discuss how to match learning activities with
learner assessments - Examine instructor time and comfort issues
- Discuss ways to limit and detect cheating and
plagiarism - Document online tools and resources for
assessment
3Online Assessment Techniques
- (with some time-saving
- tips added in)
4Is this motivating? How would you feel?
- You take an online class.
- You read some Web pages.
- Maybe you watch some videos or hear some audio
clips. - Maybe you ponder some study review questions.
- You take a multiple choice test online.
- You receive an automated score on the test.
- Class is over.
5How about this scenario?
- You take an online class.
- You meet your fellow students on the d-board.
- You read some materials. You find and share some
materials too. - You participate in some discussions of course
concepts. - You take a multiple choice test.
- You receive an automated score on the test.
- Class is over.
6Commentary on Scenario 1
- No interaction with peers.
- Students dont feel missed if they dont
participate. - Not clear why course is online (except perhaps
for media elements). - Potential for immediate feedback is nice -- but
assessment format is limiting.
7Commentary on Scenario 2
- Interaction with peers is great. Serves as a
motivator. - Community is likely to develop.
- Students will feel involved and important if they
share examples and resources. - Assessment format may not be well aligned given
the activities. - Class lacks closure in a manner appropriate to
the activities.
8Assessment and Learning
- Course objectives, activities, and assessments
should be in alignment - This tends to be an issue in courses regardless
of medium. - Example
- In class students conduct a debate
- Students are tested on their ability to recall
facts
9Mis-aligned Online Learning and Assessment
- A not-uncommon scenario
- Discussion is used as a learning activity
- Students are required to participate
- Participation is noted by how many messages were
composed by a student - But does this method measure learning?
10Common Online Assessment Complaints
- Instructor perspective
- Theres too much to assess!
- I dont know what activities to assess!
- I dont know if students really are ready for the
test! - How do I know the student actually did the
work/took the test?
11Common Online Assessment Complaints
- Student perspective
- If theyre supposed to discuss, why doesnt that
count as part of their grade? - If theyre just supposed to do something, why
does quality matter? - I just got a number, no feedback.
- I didnt get participation feedback.
12The Feedback Issue
- Students participating in online activities look
for feedback to know - A. the instructor is reading their contributions
- B. their participation is valued
- C. their participation is adequate, in terms of
quality and quantity - Feedback need not be individualized to be
effective - Whole class commentary provided on a regular
basis was found to be just as satisfactory from
the student point of view (Dennen, 2001)
13The Assessment Issue
- Often, online activities go unassessed
- Add-on syndrome Adding an online activity to a
previously designed class because it sounds like
a good idea
14The Assessment Issue
- Students are more likely to participate when then
know there is impact on their grade - Direct impact graded on participation (quality,
quantity or both) - Indirect impact participation should bolster
performance on other assessments - Students quickly become aware if an online
activity is not related to assessed learning
objectives
15The Assessment Issue, Cont.
- Sometimes the wrong things are assessed
- Examples
- Assessing students online moderation skills when
the course topic/learning objectives have nothing
to do with online moderation - Assessing quantity of participation, but not
quality - which, granted, is easier, but encourages sloppy
message posting rather than thoughtful learning
dialogues
16Blooms Revised Taxonomy
- A useful tool for checking alignment
- Also great to guide your course design!
17To Find Out More
- A Taxonomy
- For Learning
- Teaching and
- Assessing
- By Anderson
- And Krathwohl
18Assessment Techniques
- Options
- Formative or Summative
- Student-led (Self or Peer) or Teacher-led
- Public or Private
- Process or Product
- Other issues to consider
- Objective or Interpretive
- Rubric-based or Wholistic
19Formative vs. Summative Assessments
- Formative
- Alleviate student anxieties re expectations
- Seem especially high in online classes
- Encourage working toward mastery
- Can be informal
- Summative
- Used for students grade
- Assumed to be best effort
20Example Online Formative Assessment
- Paper draft discussion forum
- Start a discussion forum for papers-in-progress
- Have each student start a thread and post
elements of their papers as they complete them
(e.g., topic, major claims, research sources) - A schedule for each element is useful
- Monitor and provide feedback
21Student vs. Teacher Led
- Teacher-led
- Traditional assessment
- Most often summative
- Student-led
- Students may assess self or peers
- May be formative or summative
- Can greatly relieve instructor burden
- Students reinforce concepts through feedback
process
22Examples Online Self-Assessment
- Self-tests Use test tool to create self-tests
(multiple choice, true false) - May wish to track student efforts
- Can incentivize use (essentially, use as a
learning tool) - Reflection papers Have students submit brief,
focused papers expressing the strengths and
assessments of their assignment(s)
23Example Online Peer Assessment (Formative)
- Feedback groups Assign students in groups to
provide formative feedback on projects and papers - Often raises quality of assignments
- Need a structure with clear deadlines
- Need prompts and models to guide students
- May wish to assess feedback process/contributions
24Example Online Peer Assessment (Summative)
- Conference Presentations Have students present
their work and ask questions/provide feedback to
others. - In d-board, have students attach papers to
messages post a message with a synopsis or
attach a powerpoint presentation - Each student/team should have their own thread
- Feedback should occur during a defined period of
time. - May consider allowing students to rate
assignments on certain dimensions
25Public vs. Private
- Private
- Work is submitted to the teacher only
- Entire burden of feedback is on teacher
- Important if assessing at fact level
- Public
- Peers can see each others work (either in
process or completed) - Peers may comment on each others work
- Often increases quality of work submitted
26Process vs. Product
- Product
- The end deliverable
- Look for polish, accuracy
- Process
- How the student got there
- Look for thoughtfulness of approach, intent
27Assessing Process
- Easy to do
- Many technology tools will archive student
work/interactions - Students create a document trail in process
- Helps students develop metacognitive knowledge
- Instructors structure/model/encourage productive
work processes - Students learn how to manage their own work
processes
28Why Assess Process?
- For the instructor
- Provides formative feedback on course (e.g.,
helps gather data about why students have
difficulty with product-oriented assessments) - Clarifies who is doing most work in small group
assignments - Helps prevent cheating
29Why Assess Process?
- For the student
- Typically improves the quality of their products
- Helps them develop productive work processes
- Puts on a schedule
- Shows that you care about individual growth
30Assessment Project Cycle
- From Classroom Assessment Techniques by Angelo
Cross (1993) - Step 1 Plan
- Choose class
- Focus on assessable question
- Design project to answer question
31Assessment Project Cycle 2
- Step 2 Implement
- Teach target lesson
- Collect assessment data
- Analyze data
- Step 3
- Interpret results
- Communicate results
- Evaluate assessment project
32I. Term Papers
- How to do online
- Have students each start their own thread and
post topic of interest - Peers and instructors give feedback
- Students post thesis statements, research
sources, etc., with iterations of feedback - Final paper is posted
33Term Paper Assessments
- Product the paper
- Process quality and timeliness of student work
from time when paper is assigned - Process quality and timeliness of feedback
provided to peers - Process responsiveness to feedback received from
instructor and peers
34II. Discussion Assignments
- 1. Chain of thought
- Have students develop a solution to a problem
- Have students indicate what led them to a
particular conclusion, method or approach - Can be done in a discussion board
35Discussion Assignments
- 2. Theory to Practice
- Have students match up theories you are learning
about to actual problems - Present students with problems and have them
explain what theories they would use to solve
these problems and how they would approach it - Debrief the assignment
36Discussion Assignment
- 3. Synthesizer (i.e., wrapper)
- Have students take roles being the weekly
synthesizer of class discussion - Add a meta level in which students narrate
their own experiences while reading the weekly
discussion - Reflect on how life relates to discussion
37III. Group Projects
- Tools used
- Chat brainstorming ideas, making group
decisions, regular way to feel connected (should
be archived) - Discussion board commenting on drafts
- E-mail quick feedback
- File exchange sharing project files
- MS Word Track changes
38Group Project Assessments
- Product project files that are turned in
- Process online archive demonstrating
- Who contributed what
- Who provided peer feedback
- Who worked in a timely manner
- How collaborative a group was
- Process peer ratings
- Process interim instructor consultations
39Group Project Assignments
- 1. Work Plans
- Have students develop a plan of work for their
project - Make them outline topic, schedule, resources
needed, division of labor and anticipated form of
final deliverables - At end of project, have students evaluate how
well they followed their own plan and how useful
it was
40Project Assignments
- 2. Research Trail
- Have students document the steps they took in the
research process and the results - Ask for a brief reflection on how effective their
process was and what they might change the next
time
41Project Assignments
- 3. Process Presentations
- Have students focus on their process as well as
their product in class presentations - To maintain focus, ask them to share 3 main
lessons learned - Might ask for some process documents to be
shared, like an early draft
42Project Assignments
- 4. Design Journal
- Have students maintain a journal of all ideas
related to their project - Encourage sketches, lists, organizational charts,
etc. - Require journals to be turned in with final
projects
43IV. Reflection Assignments
- Have students keep a weekly journal of their
thoughts on readings and course content AND
real-world related instances that they noticed - May make these public, with each student having
their own discussion thread
44Making it Happen
- Learners need to see that process is valuable
- Model appropriate processes
- Provide students with scaffolding (guide sheets)
to structure their processes - Give students feedback on their process
- Require students to reflect on their processes
- Grade students on process
45Vanessas TopTime-Saving Tips
- Before you assign it, ask yourself can I
reasonably assess it? - Rely on students/peers for providing some
feedback - Let students know what to expect up front
- Choose the right tool for the job
- Get to know the editing and commenting features
of your favorite programs - Use rubrics!
- I make them in Word, and then while grading I
highlight or bold the section that applies to the
students projects. Add a few comments at the
bottom speedy grading!