Project Networking Event, Birmingham

1
Reaching Communities Programme Evaluation

• Project Networking Event, Birmingham
• Monday 29th October 2007

2
Welcome

• Allice Hocking
• ECOTEC Research Consulting

3
Introductions

• Evaluation Team
• Project Director
• Project Manager
• Research Team
• Research Team
• Research Team

• Peter Bailey, Big Lottery
• Allice Hocking, ECOTEC
• Nicola Hall, ECOTEC
• Zoey Breuer
• Rebecca Handley
• Alison Murray

4
Todays networking event

• 2.00 pm Welcome and Introductions, Overview of
  the Evaluation
• 2.15 pm Introduction to Evaluation The Toolkit
• 3.00 pm Afternoon tea and networking
• 3.15 pm Parallel breakout sessions (coloured
  stickers)
• 4.30 pm Final session Round up
• 5.00 pm Close

5
Purpose of the day

• Introduce us and the programme level evaluation
• Introduction to evaluation and explaining the
  Toolkit
• Inform you about other support available
• Projects meet each other
• Learning from each other and sharing experiences
• Answering any queries about evaluation

6
Reaching Communities Evaluation

• ECOTEC Research Consulting has been
  commissioned to evaluate Reaching Communities in
  England and Northern Ireland
• Introduction to ECOTEC
• Working with Boyd Consulting in Northern Ireland
  to cover that aspect.

7
Reaching Communities Evaluation

• Four year programme evaluation 2007-2010
• Various strands of the study including
• Evaluation Toolkit
• Networking events for projects (annual)
• Website
• Survey (annual)
• Case studies
• Annual reports
• Events to enable projects to network, meet and
  share experiences

8
Aims of Programme Evaluation

• How have projects identified need, most in need
  and hard to reach groups?
• Have projects considered local strategies to
  provide evidence of need?
• Have beneficiaries been involved in identifying
  need?
• Have beneficiaries been involved in project
  planning/ delivery?
• Have projects met the needs they identified?
• Have projects and the programme reduced
  disadvantage and exclusion?
• What different approaches have projects used to
  identify and address need and reduce
  disadvantage?

9
Introduction to Evaluation The Toolkit

• Nicola Hall
• ECOTEC Research Consulting

10
The Evaluation Toolkit

• We have developed a Toolkit to help you to
  evaluate your project
• Toolkit in your packs, by post and on the web
• Ideas of methods and approaches to use
• Examples of tools to use
• How to analyse and write up your findings
• Enable you to input to the programme evaluation
• Details of other sources of support

11
Benefits of Evaluation

• Improve the planning and management of your
  project
• Help understand how your project is working and
  how to make changes so it achieves even more
• Events to enable projects to network, meet and
  share experiences
• Involve your beneficiaries in feedback
• Help you try to secure extra funding by providing
  evidence about successes

12
How your evaluation fits in

• Not compulsory but BIG strongly urge you to
  undertake evaluation
• We can help!
• Your evaluation reports (at end of project or
  annual) can be incorporated into our final
  reports to Big Lottery Fund
• Please send your evaluations to us (details at
  end)

13
What is Evaluation?

• Different from monitoring
• Provides an assessment of your project based on
  the information you collect
• Monitoring tells you whether you have met your
  objectives, evaluation tells you how or why
  you met them
• Helps you to understand what worked and what
  didnt work as well in meeting the need you
  identified
• Self-evaluation involves staff and beneficiaries

14
(No Transcript)
15
Getting Started

• Who will do the evaluation?
• What do you want to find out? (could use the
  programme evaluation objectives)
• What has the project achieved?
• Inputs (what goes in)
• Activities (what you do)
• Outcomes (what comes out)
• You have all agreed SMART outcomes with BIG so
  evaluating them should not be too much extra work
• Use Explaining the Difference for information
  on outcomes

16
Getting Started (contd)

• Can your outcomes be sustained/ continued?
• Was your project cost-effective?
• Identified any best practice?

• Have you successfully involved beneficiaries?
• What longer impact did your project have on
  beneficiaries (tracking)?

17
Doing your evaluation

• Who needs to be consulted?
• stakeholders, staff, beneficiaries
• Involving all or a sample?
• random or targeted sample
• 10 rule of thumb
• Setting a baseline
• Collect data/surveys from the start
• Measure progress or development
• For example numbers attending, levels of
  confidence

18
Doing your evaluation (contd)

• Start with what you have already done!
• Any data/evidence you used during your
  application (e.g. local reports or statistics)
• Recent research or data on the area you are
  working in
• Data you already collect (e.g. registration
  forms)
• Feedback forms/happy sheets
• Use this to provide the context for your
  project
• Why is the project needed?
• What needs does it meet (evidence of needs)
• Why did you design it the way you did?
• Did the needs change?
• Has it worked?

19
Types of research

• Qualitative
• feelings, perceptions and thoughts
• explore the reasons why
• words and pictures
• depth interviews, focus groups, diaries

• Quantitative
• numbers and counting
• measuring how many
• surveys
• data analysis

20
Types of research (contd)

• Qualitative
• In-depth interviews
• Focus groups
• Diaries
• Peer research
• Photographic research
• Workshops

• Quantitative
• Postal survey
• Telephone survey
• Face-to-face survey
• Email survey
• Data analysis

21
Quantitative research
22
(No Transcript)
23
Surveys hints and tips

• Keep it short and simple
• Closed questions / tick boxes / scales
• Give clear instructions
• Use clear language
• Ask only relevant questions
• Collect background information (e.g. gender, age)
  for analysis
• Check data protection

24
Qualitative research
25
Topic guide example
26
Diary example
27
In-depth interviews hints and tips

• Accessible, comfortable , confidential location
• Take notes or record the interview (get
  permission!)
• Build the rapport, be attentive
• Use open questions and probe
• Focus on how? why?
• How did this affect you?
• How did this make you feel?

28
Peer research

• A great way to involve your beneficiaries in your
  project
• Enable beneficiaries to interview each other
• Provide them with skills and tools to do it
• Provide guidance / advice
• Consider some form of incentive?
• Unique feedback to include in your reports and
  self-evaluation

29
(No Transcript)
30
Interpreting your findings Surveys
Q. Do you feel more confident as a result of
attending this project?
31
Interpreting your findings Surveys
Q. What is your age? Q. Gender?
32
Interpreting your findings Qualitative
33
(No Transcript)
34
Sharing your findings

• Writing an evaluation report
• Summary
• Introduction / context
• Summary of method (what did you do?)
• Key findings
• Conclusions and recommendations
• Send us your report !
• Presentation, events
• Media and newspapers
• Newsletters

35
Sources of help

• www.reaching.communities.org
• reaching.communities_at_ecotec.com
• Send us your evaluation report to include in the
  annual report by December 15th this year

36
Whats next?

• Afternoon tea
• Then at 3.15pm go into three breakout groups
  Bham
• Zoey Breakout room
• Becky Main room
  (front)
• Nicola Main room
  (back)

37
Final session Round up

• Key themes emerging
• Bullet point 02
• Bullet point 03
• Bullet point 04
• Bullet point 05
• Bullet point 06

38
Final session Round up

• Thank you for coming and joining in
• Visit our website www.reaching.communities.org
• Email us reaching.communities_at_ecotec.com
• Give us your feedback on this event (feedback
  form in packs)
• Send us your final evaluation reports
• Fill in the annual survey for all projects
• Look out for the end of year one evaluation
  report (January 2008)

39
Thanks for coming!

• www.reaching.communities.org
• reaching.communities_at_ecotec.com