Enhancement of student engagement utilising a minimal tracking technique

1
Enhancement of student engagement utilising a
minimal tracking technique

• Eric Bodger Mike Hart David Rush
• Winchester Business School
• Faculty of Social Sciences
• The University of Winchester

2
Introduction

• Pervasive technology makes it easy to track
  people
• Our movements are tracked by CCTV, mobile phone
  cell, Internet site visits, etc.
• Students have long been tracked on paper
• attendance, tests
• Now we can also track their engagement with
  e-Learning
• Virtual Learning Environments (VLEs),
• Forums, e-mails, and blogs
• Personal Response Systems (PRS)
• This session explores the utility of such
  tracking, using
• reported experiences
• results of an investigation at Winchester

3
The Background

• Recent studies have reviewed current state of the
  art
• Analysed detail of student electronic
  interactions in forums in terms of models of the
  learning process
• Used the data to compare the delivery and
  outcomes between traditional and electronic
  delivery of the same course
• Our study seeks more empirical approaches in a
  blended learning environment
• Is there a simple way to enhance student
  performance?
• First indications are favourable

4
Approach of this paper

• Check if more engagement is reflected in improved
  performance
• Focus is on simple measures from tracking data
• Assume two items signify engagement.
• physical attendance.
• Total number of interactions with a VLE
• If link is established, tracking data can improve
  course delivery and student attainment, including
• diagnostic information to be used in supporting
  students,
• helping tutors in reflecting on their own
  practice
• tracking may alter personal relationships between
  students and tutors for better or for worse

5
Previous Tracking Studies

• Several studies indicate link between attendance
  and success
• Colby (2004) showed very clear linkage for 100
  students on 1st year programming module
• Burd and Hodgson (2005) supported this on larger
  sample
• Different demographic from Colby similar results
• 5 years, 2nd year cohort of 70, 5 Computer
  Science modules
• Contrary indication Barrett et al. (2007) found
  no correlation
• 1st year programming module
• Results limited to tutorials/practicals and early
  assessments
• Suspicion that VLE was removing perceived need to
  attend
• Should make us cautious about implications of VLE

6
Measures for Early Alarm Points

• Colbys 70 rule those attending less teaching
  have
• 67 chance of failing
• 80 chance of getting below 60
• Colbys two-week rule
• Non-attendance in first two weeks is cause for
  concern
• Could relate to progressive nature of the subject
• Provides good metric to take remedial action
• Since 2001, Winchester Business School
  administrator has tracked attendance in first 4
  weeks across four modules
• Makes contact with students showing poor
  attendance
• This has revealed some serious student problems
• and cut attrition and failure rate

7
Tracking VLE Interactions

• Two studies analysed contributions to online
  discussion forums
• 5 factor model of Pozzi et al. (2007)
• Complex study in terms of participative,
  interactive, social, cognitive and
  meta-cognitive, and teaching
• Lally (2002) used content analysis
• Group of 8 PG professional students on distance
  learning
• Complexity (60 categories) limited study to 10
  of posts
• Paanen and Simpson (2007) studied high-function
  VL
• small group, postgraduate Accounting
• Better performance on blended learning components
• Use of VLE increased significantly in these parts
• Difficult to apply complex metrics operationally

8
Tracking in Practice

• Our goal to find simple measures that assist
  management
• Early detection of at risk students
• Encouragement of practices that increase success
• Environment at Winchester
• Small University fewer than 4,000 full-time
  students
• Tradition of active approach to care for students
• Business School correspondingly small
• 450 full-time equivalent, including foundation
  degrees, combined studies, and Tourism and Event
  Mgt
• 50 Business Management students in cohort
  studied 41 students combining Business with
  other subjects
• 12 Business tutors known well by Business students

9
Results to be presented

• Correlation of attendance and performance at
  level 1
• Computing module for Business students
• 91 Students split between four sessions
• Correlates marks with number sessions attended
  out of 12
• Strong correlation except for smallest group
• Prior knowledge could be useful on second
  assessment
• Attendance and VLE usage at level 2
• Module with IT focus
• 16 students subset of first cohort
• Part of Moodle pilot at Winchester
• Some VLE usage mandatory

10
L1 Attendance and Performance
11
L2 Attendance and Performance

• Supports year 1 findings despite smaller sample
• Correlation not strong for second assignment

12
L2 Electronic interactions and Performance

• Simple count of interactions with Moodle
• Ranged from 61 to 1585, mean of 345

13
Caveats

• Could alter student behaviour when we use such
  metrics
• No concern about enhancing attendance,
• But reliance on VLE statistics could encourage
  students to generate worthless interactions
• Perception of being watched could diminish
  students relationship with staff
• Forms of interaction could be changed
• If students perceive higher value in posting
  versus e-mail
• Could make personal aspects of study less
  prominent

14
Conclusions

• Even very simple metrics can be useful
  predictors
• Useful information can be collected unobtrusively
• We analysed these data from existing records
• Need attendance lists anyway for health and
  safety
• Moodle records interaction data by default
• Potentially considerable saving of effort

15
References

• Barrett R, Rainer A and Marczyk O (2007) Managed
  Learning Environments and an Attendance Crisis?
  The Electronic Journal of e-Learning Volume 5
  Issue 1, pp 1 - 10, available online at
  www.ejel.org
• Beaudoin, M (2002) Learning or Lurking? Tracking
  the invisible online student, The Internet and
  Higher Education 5 pp147-155
• Burd, E and Hodgson, B. (2005) Attendance and
  Attainment Revisited, 6th Annual Conference of
  the ICS HE Academy, 30th -1st September,
  University of York (2005).
• Colby J., (2004) Attendance and Attainment, 5th
  ICS-LTSN. Annual Conference, 31st-2nd September,
  University of Ulster.
• Dougiamas, M. (2007) Welcome to Moodle, online
  http//moodle.org
• Kirkpatrick, G. (2005) Online chat facilities
  as pedagogic tools, Active learning in Higher
  Education, 6 (5) 145-159
• Lally, V. (2002) Elaborating collaborative
  interactions in networked learning a
  multi-method approach, in V. Lally (Ed.)
  Proceedings of the Networked Learning Conference
  2002, University of Sheffield.
• Paananen, M. and Simpson, M.(2007), An
  Exploratory study of the Effects of Web-based
  Technology on Student Performance, Second
  International Blended Learning Conference,
  University of Hertfordshire.
• Pozzi, F., Manca, S., Persico, D. and Sarti, L.
  (2007), 'A general framework for tracking and
  analysing learning processes in
  computer-supported collaborative learning
  environments', Innovations in Education and
  Teaching International, 442, 169 - 179
• Wang, M., (2004) Correlational Analysis of
  Student Visibility And Performance in Online
  Learning. Journal of Asynchronous Learning
  Networks 8 (4)
• Woodfield, R., Novell, S. E., and Solomon,
  L.,(2005) Gender and mode of assessment at
  university should we assume female students are
  better suited to coursework and males to unseen
  examinations? Assessment Evaluation in Higher
  Education (30), 1, 35 - 50

16
Questions?