Evaluation of the SFC eLearning Transformation Programme

1
Evaluation of the SFC e-Learning Transformation
Programme

• Veronica Adamson
• Jane Plenderleith
• Glenaffric Ltd

2
Nature of Transformation

• Transformational change will require a conscious
  and deliberate decision made by (one or more)
  institutions to do something differently, in a
  systematic way, across the whole institution, on
  a defined timescale of two or more years

3
Key Principles

• The process of change will mean that certain
  aspects of learning and teaching are conducted in
  a new way
• The process of change is consistent with, and
  embedded in, institutional strategies, and is not
  a peripheral process driven solely by the
  possibility of external funding
• The intended outcome is sustainable, and is
  expected to result in long-term change in
  activities beyond the period of external funding
• The process will yield measurable benefits to the
  institution and its learners

4
Key Influences

• Center for Academic Transformation at Rensselaer
  Polytechnic Institute
• Its mission is to serve as a source of expertise
  and support for those in higher education who
  wish to take advantage of the capabilities of
  information technology to transform their
  academic practices
• MIT Transformational Model
• CITSCAPES Development Tool
• Becta Matrix

5
Pew Grant Programme

• Supported by an 8.8 million grant from the Pew
  Charitable Trusts, the Program in Course Redesign
  was created in April 1999
• Program is supporting thirty colleges and
  universities in their efforts to redesign
  instruction using technology to achieve quality
  enhancements as well as cost savings
• Preliminary results show that all institutions
  reduced costs by about 40 on average, with a
  range of 20 to 84

6
Program in Course Re-Design

• Continuum from fully F2F to fully online
• Five design models identified
• Supplementalretains basic course structure but
  add technology-based activities
• Replacementreduction in class meeting time
• Emporiumbest time to learn is when students want
  to not when tutor wants to teach
• Fully onlinestaff perform tasks alone
  translated to online but labour intensive
• Buffetall of the above treat students the same
  but could use menu based on student needs with
  varying learning styles

7
Becta Matrix Self-assessment Tool

• Developing an e-Learning Strategy
• College Strategic Plan

8
e-Learning Environments Study

• Listed below are functions of processes, services
  and systems that support learning and teaching.
  To what extent will these systems, services and
  processes be developed or integrated in your
  institution by September 2005, and to what extent
  do you want them to develop in the future?

9
(No Transcript)
10
(No Transcript)
11
Evaluation Model
12
Evaluation Aims

• Assess the extent to which the projects met their
  aims and objectives and implemented new models of
  delivery in FE and HE institutions
• Provide constructive feedback to the Councils
  executive and to project teams on the progress of
  the projects
• Assess the extent to which the programme as a
  whole (including the arrangements for monitoring
  and co-ordination) met its objectives
• Inform future Council policy on e-learning

13
Evaluation Key Tasks

• Develop an evaluation framework/methodology,
  present this to the projects and undertake the
  evaluation of the programme
• Review progress by the six projects and identify
  emerging issues
• Report to the SFC executive on key emerging
  issues and to make recommendations (formative
  feedback) for maximising the impact of the
  programme in the short and the long term
• Report to the SFC on the effectiveness of the
  arrangements for programme monitoring and
  co-ordination, and dissemination.

14
Project Visits

• Core project documentation (eg project initiation
  document and project plan)
• Project management arrangements (strategic and
  operational)
• Communication strategy and reporting
• Project evaluation plans and baseline data
  gathering
• Cultural/organisational issues
• Project support needs

15
Project Visits
CeLLSUniversity of Dundee, Tuesday 17
May e-ConstructionGlasgow Metropolitan College,
Wednesday 18 May BlendEdReid Kerr College,
Paisley, Wednesday 1 June (am) ISLE,University
of Paisley, Wednesday 1 June (pm) Re-engineering
Assessment PracticesUniversity of Strathclyde,
Glasgow, Thursday 2 June TESEP,Napier
University, Edinburgh, Friday 3 June
16
Evaluation Framework

• Programme overview and context
• Purpose of evaluation
• Evaluation objectives
• Methodology
• Key informants and other stakeholders
• Evaluation outputs and outcomes
• Logframe

17
Modified Logframe
18
Project Support Needs

• What skills, knowledge and experience are
  required in the project team in order to complete
  the project successfully?
• Does the project team currently have these
  skills, knowledge and experience?
• What kinds of external advisory support will you
  need to complete the project successfully?
• What form would you like this advice and support
  to take?
• How will you review your project team development
  and advisory support needs?

19
History of the Future Exercise

• Logic Models - tool to describe the effectiveness
  of a project
• Logic Maps - simplified version for articulating
  implicit theories of change
• Distinction between outputs and outcomes
• Surfacing assumptions allows the project team to
  identify differences and reach a consensus

20
History of the Future Exercise

• Imagine that your project is completed and that
  it has succeeded in all of its goals.
• You are being interviewed tomorrow on a local
  radio show to talk about the project.
• In order to avoid fluffing your words on live
  radio the questions have been sent to you in
  advance so that you can jot down your main points.

21
What to do

• On the form note
• What did your project actually do
• Whos going to benefit from this
• Why is it important
• How did you help to make the project a success
• You have 10 minutes

22
Now share with your team

• Form a group with the others in your team
• Share your interview notes
• Identify any key differences
• Briefly feedback to everyone - any surprises?