ComputerBased Mathematics Assessment of Engineering Mathematics Students

1
Computer-Based Mathematics Assessment of
Engineering Mathematics Students

• L.E. Davis, M.C. Harrison, A.S. Palipana,
  J.P.Ward
• Mathematics Education Centre

2
Computer-Based Mathematics Assessment of
Engineering Mathematics Students

• Presented by
• Dr Martin Harrison
• HELM Project

3
This presentation

• HELM Project
• Background
• Teaching Mathematics to Engineering Students
• HELM Learning Environment
• Workbooks, CAL and CAA
• HELM CAA Regime
• Trialling Evaluation
• Conclusions

4
The HELM Project

• Helping Engineers Learn Mathematics - HELM
• Major 3-year curriculum development project
• Consortium of five UK universities
• UK government funding 250,000 (about 550,000
  Cdn)
• Oct 2002 - Sept 2005
• To enhance the mathematical education of
  engineering undergraduates by the provision of a
  range of flexible learning and teaching resources

5
HELM Target Group

• Departments and academics teaching mathematics to
  engineering undergraduates
• Engineering undergraduates who are the focus of
  the mathematics problem

6
Mathematics Problem

• Numerous reports testify to the severity of the
  mathematics problem now faced by the UK HE
  sector
• Too many graduate engineers are perceived to be
  deficient in mathematical concepts and fluency
• Students are now accepted on engineering degree
  courses with relatively low mathematics
  qualifications
• A decline in mathematics skills between 19902000
  has been confirmed
• High failure rates and withdrawal rates are
  linked with the mathematics problem

7
Mathematics Problem

• Common problem affecting whole of the UK HE
  sector
• Wide range of ability of students
• Some cant cope with the traditional lecture or
  with the pace
• Weak students may stay hidden
• Unsuitable resources
• Too advanced or too much material
• Too many assumptions about prior knowledge
• Insufficient or inappropriate examples
• Clear need for more flexible mathematics
  curriculum for engineering undergraduates

8
HELM Learning Environment

• Using computer technology to develop
• A range of learning resources (Workbooks, CAL
  courseware)
• Assessment procedures
• Integrate into existing programmes
• Selecting stand-alone units
• Adopting the whole scheme
• Use
• To support lectures and continuous assessments
• To complement existing resources and texts
• For independent or group learning
• Useful resource for science students and even
  specialist mathematics students

9
HELM Workbooks

• High quality written materials distributed to
  students as workbooks.
• Paper based and electronic versions (PDF) are
  available
• 46 workbooks cover UK engineering mathematics
  syllabus
• Calculus Algebra
• Fourier analysis
• Laplace and z-transform methods
• Ordinary partial differential equations
• Complex analysis
• Numerical methods
• Probability statistics
• 3 workbooks of engineering case studies
  applications
• Students guide Tutors guide

10
HELM Workbooks - Sample
11
HELM Workbooks

• Each Workbook contains
• Key points and contents in manageable sizes
• Guided exercises and worked solutions
• Students can insert their solutions
• Engineering examples

12
HELM CAL Resources

• Web-delivered CAL courseware
• Multimedia interactive lessons (Authorware)
• About 70 segments related to many workbooks
• Audio, interactivity, revision exercises
• Self-assessments

13
HELM Assessment Regime

• HELM Computer Aided Assessments
• To reduce burden on staff involved in continuous
  assessment
• To check if students are ready to learn an
  engineering concept involving specific
  mathematical topics
• To encourage self-assessment (formative)
• To drive student learning
• Essential to gain the full potential of the other
  learning resources
• A regular pattern of short periods of study
  followed by assessment drives learning along at a
  steady pace

14
CAA Implementation

• Question Mark Perception (QMP)
• Integrated web-delivered CAA regime
• Self-testing (Formative)
• Formal assessment (Summative)
• CD based CAA regime
• Self-testing is straightforward
• Formal assessment may be more difficult
• Requires a mechanism for marks to be processed
  and stored.
• Successfully used at LU for RAE BestMaths project

15
CAA Screenshots

• CAA QM Perception Screenshots

16
CAA Resources

• 4500 questions in about 150 question banks
• Most have feedback as worked solutions, examples
  or generic instruction
• Each bank contains clones of a question
• This maintains the same difficulty level within a
  bank
• Justifies the random selection of a question each
  time a test is presented
• Tests can be custom-made by selecting questions
  from more than one bank of questions

17
Likes and Dislikes

• Students like flexibility
• Taking tests when they are ready
• Taking tests where they want
• Taking tests as many times as they need, on their
  own
• Students dislike
• Unforgiving nature of CAA
• No marks for the method and intermediate steps
• Staff concerns
• Question banks for practice formal testing
• Unsupervised testing / cheating

18
Usage by Time of Day
19
Usage by Year
20
Load 2003-04
21
Trialling Evaluation

• High level of interest
• Over 60 academics from UK HE or FE institutions
  either trial HELM resources or have shown
  interest in trialling
• Evaluation of resources is needed
• To ensure they are of a high standard
• To check appropriateness for the needs of
  students and academics
• To check accuracy
• To find out better ways of using them and good
  practice

22
Trialling Evaluation

• Evaluation via
• Information gathered from trialling academics and
  students
• Focus groups
• Critical readers
• Outcomes
• Workbooks
• lecturers use them both as core notes
  supplementary material
• students use them for independent learning
• CAL - mainly used as additional resource
• CAA - significant interest
• Implementation overheads of web delivery lead to
  CD version

23
Conclusions

• Need
• More flexible mathematics learning resources
• Due to increasing diversity of intake standards
• HELM Learning Resources
• Potential to be enhance the mathematical
  education of engineering undergraduates
• Provide an alternative to lectures
• Can be used in distance learning mode

24
Conclusions

• HELM Workbooks
• Encourage student engagement during lectures
• HELM CAL
• Complement workbooks aid understanding
• HELM CAA
• Flexible access via web delivery
• Facilitates regular testing of large numbers of
  students
• Random question selection
• Instant feedback
• Incorporates formative and summative testing
• Drives student learning

25
Contact HELM

• Phone [phone]
• Email helm_at_lboro.ac.uk
• Web http//helm.lboro.ac.uk