Self Evaluation of Programmes

1
Self Evaluation of Programmes

• Experience of the Internal Evaluator
• County Dublin VEC

2
Approach to Self Evaluation

• Decision taken to carry out Self Evaluation on a
  programme basis over a 5 year cycle VTOS,
  Youthreach STTC, PLC, Literacy, Community Ed.
• Initial self-evaluation to be carried out using
  the Self Evaluation Checklist and other sources
  of information.
• Evaluation Report to be drafted by County
  Co-ordinator.
• External Evaluators to verify the findings
  through a process of
• sampling and contribute suggestions for
  improvement.
• Programme Improvement Plans to be written up and
  
• implemented.

3
VTOS in Co. Dublin VEC

• VTOS operates in 15 centres across the county 8
  schools or FE Colleges and 7 Adult Ed. Centres.
  VTOS QA system implemented from 2000
• Guide to VTOS in County Dublin VEC
  (Co-ordinator Handbook)
• 2. VTOS Network Meetings
• 3. Annual reporting system VTOS Programme
  Review and Evaluation incorporating evaluation
  by tutors, participants and VTOS Co-ordinators

4
Steps in Self Evaluation

• Introductory briefing on Self Evaluation to VTOS
  Network and meeting with External Evaluators
  (County Wicklow EDOs)
• 2. Development of Self Evaluation Plan and
  Template Self Evaluation Checklist (adapted)
• 3. Completion of adapted Self Evaluation
  Checklist by all VTOS Centres

5
Self Evaluation Checklist (adapted sample)
6
Steps in Self Evaluation (continued)

• Preparation of VTOS Programme Evaluation Report
  by County Co-ordinator using Checklists and
  annual VTOS reports (VTOS Programme Review and
  Evaluation)

7
Steps in Self Evaluation (continued)

• Review of VTOS Programme Evaluation Report by
  External Evaluators and agreement of procedures
  for Sampling
• Implementation of sampling activities e.g.
  review of learner evaluations
• 7. Development of Programme Improvement Plan by
  County Co-ordinator incorporating feedback from
  External Evaluators

8
Steps in Self Evaluation (continued)

• 8. Development of Programme Improvement Plans by
  individual VTOS Centres
• 9. Implementation of Programme Improvement Plans
  at local centre level and at county level

9
Self Evaluation Cycle
10
Experiences in External Evaluation

• Catherine Byrne
• Education Development Officer
• County Wicklow VEC

11
Experiences of External Evaluation

• Reciprocal peer review approach
• Respect for the process, practitioners and
  participants essential
• Conducting own self evaluation gave a great
  insight
• Opportunity for an exchange of good practice
• A learning process in its own right

12
Some challenges

• Developing an agreed structure and procedure
• Managing time constraints
• Remaining focused on verification
• Developing a template for the draft External
  Evaluation Report with minimal guidelines
• Maintaining objectivity

13
Recommendations

• Use peer review approach
• ensures good understanding
• informs self-evaluations carried out in home
  provider
• Support practitioners and participants to develop
  the expertise needed through training
• Provide appropriate resources finance, time,
  facilities, services to support the process
• Ensure that good practice is shared throughout
  the sector