JISC Elearning Programme

1
JISC E-learning Programme

• Evaluation

2
This morning

• Stuff about evaluation perspectives
• Possible evaluation models
• What weve done so far
• Some tentative observations from interviews - a)
  evaluation b) issues
• Even more tentative observations - our issues
• Things to be resolved - questions

3
Tomorrow morning

• Evaluation approach - strengths and weaknesses
• Evaluation plan
• Timetable
• Tools
• Report to JISC
• Next steps

4
Evaluation approach

• Why a theoretical base?
• Evaluation can only be effective if you know what
  you want it to do before you start!
• Different approaches to evaluation serve
  different purposes
• Different approaches generate different models,
  tools and practice

5
Theoretical divides epistemological
6
Theoretical divides ethical
7

A classification system
8
Other possible divides

• Methodological
• Disciplinary background
• Different metaphors
• Different client groups
• Differences in practice

9
Process to date

• Interviews
• Observation at various events a)JISC b)wider
  arena
• own knowledge of the e-learning field
• Documentation from JISC
• Library search

10
Outputs

• Stuff about what the evaluation should look like
• Stuff about the what the evaluators should be
  doing /spending their time
• Matters of concern to the JISC E-learning
  programme team
• Matters of concern to us

11
Key messages from interviews about evaluation
process

• Contradiction between focussing on summative
  confirmation of achievements and formative
  process
• Unanimity on a lot of issues
• light process at project level - Go easy on the
  surveys!

12
Key messages from interviews about evaluation
process (cont.)

• Interventionist and formative
• Pushing out the boundaries of the evaluator role
• Evaluator as integrated part of programme team
• Products should be fast and accessible - content
  not packaging

13
Evaluation priorities and tasks

• Providing updates on what is happening in the
  outside and inside world and making sure we know
  AND are acting on it AND we have a strategy for
  doing it.
• Co-ordination of reporting activities and other
  evaluation activities
• Helping to facilitate the communication between
  strands and between strands and the CLT and other
  parts of JISC - suggesting strategies and
  providing an overview.
• Risk assessment - what happens if something goes
  wrong? (strategic and operational)
• Finding out what other people think about us!

14
(No Transcript)
15
Tender priorities

• Responsiveness and flexibility
• Achievement and outcomes
• Timeliness and relevance
• Impact analysis

16
Concerns (yours)

• Programme coherence
• Improving communication between strands
  especially integrating the pedagogy strand
• Accessibility will the wider community
  understand?
• How do we explain the framework and what it will
  do?

17
Your concerns (cont)

• Workload of programme staff and speed of work
• How to capture, optimise and develop skills
• The maturity of the development projects (T)
• Ownership and adoption of ELF
• Technical implementation of SOA

18
Tish raised these extra ones!

• Combining committee demands and funding pressures
• Breadth of consultation needed

19
Key concerns (ours)

• Sustainability (T)
• Iterative development and the move between
  development and demonstration phases
• Focus on teaching or learning?
• Ambitions vs reality?
• Relationship between projects and strands

20
More of our concerns!

• Use of us-cases
• Organisational commitment
• Internal and external communication
• Presentation and dissemination - language barrier
• Jisc overall commitment

21
Yet more of our concerns!

• Levels of integration with European developments
• Relationship with educational community
• Relationship with industry
• Operational stuff - time, money, support
• Sophisticated level of self criticism

22
And even more!

• Need for working software
• New type of programme, new outputs (T)

23
Exemplars

• Distributed vs centralised repositories?
• Use of CMS applications as learning platforms
• Games /blog and non-learning software for
  learning
• Teachers or learners defining learning processes?
• Limited metadata sets as basis for description

24
Messages for CLT

• The programme staff can only do so much - the CLT
  has an operational role as well as a policy role.
• Dont expect results overnight - its a long road
  and we have to get it right.
• Invest some time thinking about how to take some
  of the daily pressure off staff - e.g. pay and
  rations issues

25
Questions?

• Communication confidentiality vs transparency -
  web site
• Meetings with evaluation people timing,
  responsibility, remit, attendance
• Tools

26
(No Transcript)
27
(No Transcript)