New Opportunities for

1
New Opportunities for PE and Sport Initiative
Evaluation Seminar
2
Structure of Seminar

• Introduction
• NOPES programme
• Evaluation why?
• Programme evaluation
• Big Lottery Fund Monitoring
• Self-evaluation
• Baseline and Annual Surveys
• Summary and Conclusions

3
Introduction to NOPES
4
Why Evaluate?
5
Why Evaluate?

• Demonstrate the impact of an initiative
• Identify ways to improve existing practice
• Influence future policy

6
Why Evaluate?
7
Why Evaluate?
8
Why Evaluate?
9
Why Evaluate?
10
New Opportunities for PE and Sport Initiative
Programme Evaluation
11
NOPES Evaluation Team
Loughborough Partnership

• Institute of Youth Sport
• Institute of Sport and Leisure Policy
• Centre for Research in Social Policy

• PE and Sport Pedagogy Group
• British Heart Foundation National Centre
• Centre for Developing and Evaluating
• Lifelong Learning, Nottingham University

• Led by Dr Mary Nevill, Director, Institute of
  Youth Sport
• Three Research Associates at Loughborough
  University

12
Programme Evaluation
Three key dimensions

• Partnership effectiveness
• Impact on participation
• Impact of participation on wider social change

13
Programme Evaluation
Methods

• 7 Fast Track projects
• 11 Core Case Study
• Telephone Case Studies
• Baseline and Annual Surveys

14
Programme Evaluation
Dissemination of Results

• Reports to the Big Lottery Fund
• Evaluation Newsletters
• Evaluation Website www.nopesevaluation.org.uk

15
Findings Fast Track projects

• Partnership
• Operational realities taken precedence over
  strategic aims and objectives
• Role seen as being instrumental in achieving
  longer term goals

16
Findings Fast Track projects

• Participation
• Dramatic impact on both quality and quantity of
  PE recognised by both staff and young people
• Very positive impact on the provision of
  extra-curricular activities
• Impact on the community limited to date and
  presents a much greater challenge

17
Findings Fast Track projects

• Impact on Wider Social Behaviour
• Links with wider social outcomes are based on
  very general notions about the benefits of sport
  and exercise
• Few specific programmes designed to target wider
  social behaviour

18
New Opportunities for PE and Sport Initiative
Big Lottery Fund Monitoring
19
New Opportunities for PE and Sport Initiative
Self-Evaluation
20
Self Evaluation Why?
Task 1 In pairs or small groups consider What
are the main benefits of monitoring and
evaluating the projects within your portfolio?

21
Self Evaluation Why?

• Demonstrate the impact of NOPES on local schools
  and communities
• Identify ways to improve existing practice within
  your LEA and within your schools
• Influence the future direction of PE and Sport
  policies for your LEA and schools

22
Self Evaluation Purpose

• To consider how your local authority will
  demonstrate the impact of your NOPES portfolio
• To consider how the projects within your
  portfolio can be supported in undertaking
  self-evaluation

23
Gathering evidence Evaluators role

• New Opportunities for PE and Sport
  Self- Evaluation Handbook
• Includes
• Survey guidance
• Step by step guide

24
Self Evaluation Step by Step
Step 1 Identify aims, objectives and success
indicators Step 2 Collect and record
data   Step 3 Make sense of your data   Step
4 Share your findings
25
STEP 1 Aims, Objectives and Success Criteria

• Aims - Aims are general statements you intend
  to achieve
• Objectives - More specific measurable outcomes

26
NOPES Outcomes 1- 3

• 1.Improved physical education and sport in
  schools 
• 2. Higher standards across the whole school
  through PE, sport and other forms of structured
  activity
• 3. Better opportunities to increase the levels of
  physical activity among the school age population
  and, more generally local communities

27
NOPES Outcomes 4 - 6

• 4. Improved collaboration, co-operation and
  partnership between schools and their communities
• 5. Promotion of social inclusion through access
  to, and use of, sports and outdoor adventure
  facilities by all groups in society
• 6. Innovation and best practice in the design and
  management of facilities including consideration
  of the principles of sustainable construction and
  development.

28
STEP 1 Aims, Objectives and Success Criteria
Success indicators - Criteria to determine
whether the process is successful or not in
achieving the aims originally identified. Success
indicators need to be SMART (Specific,
Measurable, Attainable, Relevant and
Time-limited)
29
SMART Success Criteria
Task 2 Using the following aim and objectives
taken from a development plan identify possible
SMART Success Criteria. SMART Specific,
Measurable, Attainable, Relevant and
Time-limited Aim NOPES Outcomes 3 and 4
Objective Develop inter-school competitions
organised and umpired by students
30
SMART Success Criteria

• Task 2 - Examples of SMART Success criteria
• Quantitative measures of impact
• Summer term 2005 - 5 Year 10 students involved
  in JSLA to organise a one day kwik cricket
  event for 4 local primary schools in Summer term
  using the new facility.
• Also need to consider
• Quality of provision
• How will future policy and practice be informed?

31
STEP 2 Collect and record data

• Data collection
• What data need to be collected?
• How will the data be collected?
• How often will the data be collected?

32
STEP 2 Collect and record data

• Task 3
• In pairs or small groups consider
• How will the required data be collated within
  your projects?
• What assistance will they need in order to do
  this?
• Is any support available?

33
STEP 3 Making sense of the data

• Has your project or portfolio achieved the
  intended aims and objectives?
• Do the results match your expectations?
• What implications does this examination of the
  data have for the future of the project?
• What will need to be done to ensure the project
  achieves the aims and objectives identified?

34
STEP 4 Sharing the findings

• Audiences
• Stakeholders and partners
• Young people
• Parents
• Colleagues
• Reporting the findings
• Posters
• Newsletters
• Reports

35
Sharing the findings Evaluators role

• Sharing examples of effective practice and
  evaluation findings through
• Newsletters
• Website - www.nopesevaluation.org.uk
• Evaluation progress
• Resources
• Effective practice

36
New Opportunities for PE and Sport Initiative
Baseline and Annual Surveys
37
Baseline and Annual Surveys

• Development
• Extensive development phase
• Piloted with fast track projects

Big Lottery Fund Monitoring
38
Baseline and Annual Surveys

• Purpose
• Quantitative Data
• Change Over Time

• Tools
• Baseline Survey NOPES funding under 125,000
  and / or not based at school site
• Baseline Survey (Extended) NOPES funding over
  125,000 based at school site

39
Baseline and Annual Surveys

• Distribution
• Directly to Schools / Projects
• Follow Up
• Annual Survey

• LEA Role
• Access to returned surveys
• Loughborough Report (50 of surveys returned)

40
Baseline Survey Distribution - England
41
Baseline Survey (Extended)
42
Baseline Survey (Extended)
Section 2 Background Information
Q5

• Quality of existing facilities
• Can be used as evidence for Outcome 3 6

43
Baseline Survey (Extended)
Section 2 Background Information
Q6

• Importance of other initiatives
• Can be used as evidence for Outcome 4

44
Baseline Survey (Extended)
Section 3 Typical Week
A week in which there is minimal disruption to
school timetables and plans as a result of
activities weeks, poor weather, exams and tests,
plays, visits or concerts. In a typical week,
pupils are familiar with the timetable and
pattern of the day.
45
Baseline Survey (Extended)
Section 3 Information about the pupils at your
school
Q11

• Curriculum PE Time
• How to collect
• Can be used as evidence for Outcome 1

46
Baseline Survey (Extended)
Section 3 Information about the pupils at your
school
Q12

• Pupils participating in 2 hours PE and Sport
• How to collect
• Can be used as evidence for Outcome 1

47
Baseline Survey (Extended)
Section 3 Information about the pupils at your
school
Q14

• Availability of extra-curricular activities
• Participation in extra-curricular activities
• How to collect?
• Can be used as evidence for Outcome 1

48
Baseline Survey (Extended)
Section 4 Usage by other schools and community
Q16

• Other schools use
• Definition
• How to collect?
• Can be used as evidence for Outcomes 3 4

49
Baseline Survey (Extended)
Section 5 Range of Activities
Q19

• Available Activities
• How to collect?
• Can be used as evidence for Outcomes 1 3

50
Baseline Survey Larger Projects
Section 6 Target Users
Q20

• Identify Target Groups
• Target groups users
• How to collect?
• Can be used as evidence for Outcome 5

51
Support for Completion

• Self-Evaluation Handbook
• Website
• www.nopesevaluation.org.uk
• Phone and email
• I.A.Lindsey_at_lboro.ac.uk / C.L.J.Mason_at_lboro.ac.uk
  
• 01509 226379 / 01509 226378

52
Summary and Conclusions

• Why evaluate?
• NOPES programme evaluation
• Self-evaluation
• Baseline and Annual Surveys

53
New Opportunities for PE and Sport Initiative
Evaluation Seminar