Assessment

1
Assessment
CMT105 Lecture 9

• What does who know when?

2
Contents!

• Not the usual structure
• What do we mean by assessment?
• How do we relate this to objectives?
• Why is it hard?
• What has been done?
• What can we reasonably do?

3
What do we mean by assessment?

• Diagnosis
• Evaluation
• Assessment

4
How do we relate this to objectives?

• Diagnosis need not be objective related
• One piece of assessment often relates to multiple
  objectives

5
Why is it hard?

• Knowing what is in someones head is not easy
• for a computer to do it is harder!

6
What has been done?

• Multiple choice
• Skill assessment
• Problem solving
• Restricted formal language
• Restricted natural language
• Open ended language input

7
Multiple choice

• Design issues
• (see printed document)

Question stem
What is the capital of Brazil? 1) Brazilia 2)
Peanuts 3) Chile
Answer
Responses
Distractors
8
Advantages and disadvantages

• Reusable
• self-marking
• Instant feedback
• - Design is hard
• - Restricted usefulness

9
Tools

• Lab rat
• Question Mark
• Web-based question systems
• IRIS

10
Skill assessment

• Not too hard
• Easy for computer-based tasks
• Work with simulations and learning environments
• e.g Dominie

11
Problem solving activities

• Specific restricted tasks which require
  application of certain knowledge

12
Restricted formal language

• Programming language work
• Proust-style systems
• Graceful degradation
• Logics etc.
• Nice graphical tools

13
Restricted natural language

• Constructed language subsets
• Recognition and parsing problems
• Semantic grammars

14
Open ended language input

• This is hard

15
What can we reasonably do?

• Good quality multiple choice
• Restricted formal languages
• skill assessment

16
Activities

• To Do
• Pick a small subject area and design an
  assessment with multiple choice. Aim at about 15
  questions. Include an explanation of which
  concepts are being tested where.
• Readings
• J. Cotton 1995 The theory of assessment An
  introduction
• W.C.Ward (1991) Oportunities for the use of
  Technology Assessment