Computer-Assisted Assessment (CAA)

1
Computer-Assisted Assessment (CAA)

• Bob Rotheram
• Reader in ALT
• Office of PVC (ALT)

2
Session outline

• Review CAA demo
• CAA in your discipline?
• Question-setting (presentation)
• Question-setting and shredding (exercise)
• Uploading questions
• Where next?

3
Did you do your homework?

• Tried this CAA demo? http//questionmark.ntu.ac.uk
  /q/open.dll?SESSION5130550928567055NAMExGROUP
  x
• Comments?
• wording of questions?
• feedback?
• anything else?

4
Some uses for demo

• Summative assessment
• Diagnostic tests
• Formative tests
• Informal quizzes
• Revision aid
• Convey what social policy is

• Individual questions v. flexible, granular,
  learning objects
• insert into lectures
• gather further info via web links
• preparation for classes

5
Computer-assisted assessment

• Much up-front effort needed
• Best where
• large numbers of students on introductory courses
• questions have long shelf-life
• teams can produce shared question banks

6
CAA in your discipline

• Use?
• Potential/advantages?
• Limitations/disadvantages?

7
Writing questions

• Pritchett N, (1999), Effective question design
  in Brown S, Race P and Bull J,
  Computer-Assisted Assessment in Higher Education,
  London, Kogan Page
• Bull J and McKenna C, 2001, Blueprint for
  Computer-Assisted Assessment esp. Chapter 3 and
  Appendix I

8
Blooms Taxonomy (1956)

• knowledge
• comprehension
• application
• analysis
• synthesis
• evaluation

9
Some question types

• multiple-choice
• multiple response
• text match response/fill in the blank
• true/false, yes/no
• Likert scale
• numeric
• essay

10
Knowledge-testing questions

• What word means the same as?
• What is the most important difference between?
• Which one of the following sequences shows the
  correct order of?
• What are the major classifications of?
• Which method is the most useful for?
• What evidence best supports the theory of?

11
Comprehension-testing questions (if not taught or
discussed)

• Which one of the following is closest in meaning
  to the term?
• The statement means that
• (Various facts are presented.) Which of the
  reasons listed below best explains this?

12
Feedback minimum

• statement on whether chosen option correct or
  not
• (if incorrect) statement giving correct answer
  and why correct
• clear up possible misunderstanding about wrong
  choice
• source of info on correct answer
• language clear and not discouraging.

13
Follow up later

• Read Blueprint, Ch.3 Appendix I
• See more examples of CAA, e.g.
• Questionmark Perception
• http//www.questionmark.com/uk/tryitout_college_un
  iversity.htm
• Hot Potatoes, inc. tutorials
• http//hotpot.uvic.ca

14
Now

• Write two questions feedback
• Shred questions

15
Upload questions

• To WebCT development server at
• http//zirconia.lmu.ac.uk8900/webct/public/home.p
  l
• What next?

16
Contact me

• Bob Rotheram
• Office Room 205, Old School Board
• Email b.rotheram_at_leedsmet.ac.uk
• Mobile [phone]
• Office phone [phone] ext. 29045