Designing Questions for Online Assessment Susan Woodger, PassIt Project

1
Designing Questions for Online Assessment
Susan Woodger, Pass-It Project
2
Aims and Objectives

• Benefits to using online assessment
• What we mean by usability and accessibility
• Reasons why web materials need to be accessible
• Issues surrounding the authoring of online
  questions

3
Benefits of Online Assessment

• Automated marking
• Instant feedback
• Different modes of delivery
• Reporting systems
• Improved accessibility
• Customisation

4
Design Considerations

• Usability
• Equivalence of Educational Objectives
• Technical Aspects of using a different medium
• Accessibility

5
What is Usability?

• Equivalence of educational objectives
• Learning objectives
• Language
• Measurement
• Suitable question types
• Revisit the original question

6
Equivalence

• Music question

7
What is Usability?

• The measure of the quality of a user's experience
  when interacting with a product or system..
• Ease of learning
• Efficiency of use
• Error frequency and rate
• Subjective satisfaction
• www.usability.gov

8
Different medium

• Reading times
• Layout
• Fonts
• Verdana, Georgia, Trebuchet MS.
• Navigation

9
What is Accessibility?

• The degree to which someone can access a web
  resource regardless of
• Disability
• Environment
• Browsing technology
• TechDis (Introduction to Web Accessibility)

10
Why is Accessibility Important?

• Legislation
• Disability Discrimination Act 1995 (DDA)
  www.hmso.gov.uk/acts/acts1995/1995050.htm
• Special Educational Needs and Disabilities Act
  2001 (SENDA) www.hmso.gov.uk/acts/acts2001/2001001
  0.htm

11
Why is Accessibility Important?

• The new act came into force on
• 1 September 2002

12
Why is Accessibility Important?

• For an easier read
• http//www.techdis.ac.uk/resources/skill01.html
• http//www.ukcle.ac.uk/directions
  /issue4/senda.html

13
Why is Accessibility Important?

• Statistics
• Higher Education Statistics Agency (HESA)
• http//www.hesa.ac.uk/holisdocs/pubinfo/
  student/disab0203.htm

14
Accessibility

• ALT text and Longdesc
• Images
• Access keys
• Colour and Contrast
• Stylesheets
• Tables
• Screenreaders
• Screen magnifiers

15
Steps

• Steps question

16
Design Process

• Choose Question Type
• Design in line with learning objectives
• Ensure accessible design
• Author question
• Check validity
• Test
• Publish

17
Original Question

• Diagrams 1 and 2 on the worksheet show part of
  the graph yf(x).
• On Diagram 1,
• draw the graph of y-f(x).
• (b) On Diagram 2,
• draw the graph of yf(x2).

• Assessment Engine
• Next

18
Option 1 Multiple Choice
19
Option 1 Multiple Choice

• Advantages
• Easy to create and mark
• Easy for student to answer
• Some randomisation possible
• Disadvantages
• Recognition and Elimination NOT construction
• High screen real estate required
• Limited possible answers

20
Option 2 - Hotspot
21
Option 2 - Hotspot

• Advantages
• Many possible answers
• Construction of an answer
• Lower screen real estate
• Easy to mark
• Disadvantages
• Could be fiddly for student to answer
• Time consuming to create
• Low reusability
• No randomisation
• High server communication and load

22
Option 3 - Flash
23
Option 3 - Flash

• Advantages
• Many possible answers
• Construction of an answer
• Lower screen real estate
• Easy to mark
• Randomisation possible
• High reuse
• Low server and communication load
• Increased accessibility
• Disadvantages
• Creation requires multimedia authoring skills

24
Resources

• W3C http//www.w3.org/
• TechDIS http//www.techdis.ac.uk/
• RNIB http//www.rnib.org.uk/
• RNID http//www.rnid.org.uk/
• Use-It http//useit.com
• Good Practice Guide
• http//www.pass-it.org.uk/resources/031112-goodpra
  cticeguide-hw.pdf

25

Designing Questions for Online Assessment

• Susan Woodger, Pass-It Project
• s.c.woodger_at_hw.ac.uk
• 0131 451 3231