State of the Sector Panel: Design and Execution of a Panel of NGO Organisations in England

1
State of the Sector PanelDesign and Execution
of a Panel of NGO Organisations in England

• Presentation of findings
• to Invited Session ICES-3
• 24th May 2007

2
Introduction

• UK NGOs (voluntary sector) represents growing
  element in the UK but only limited information is
  available
• In 2003, a need arose to monitor progress towards
  government targets within the voluntary sector
• The State of the Sector Panel was set up to meet
  this need and to fill many of the knowledge gaps
  about NGOs
• But the project was limited by the available
  information required to structure the universe
  and weight the data
• Panel was constructed for a specific purpose so
  not all NGOs were eligible
• Needed organisations who deal directly with the
  public
• Wanted to exclude organisations whose sole
  function is grant making or umbrella bodies who
  deal only with other NGOs
• Wanted to exclude other NGOs such as trade
  unions, professional bodies, places of worship

3

• The Government are very keen on amassing
  statistics - they collect them, add them, raise
  them to the nth power, take the cube root and
  prepare wonderful diagrams.
• Josiah Stamp

4
Non charitable sector

• NGOs in England fall into two categories
• Registered Charities
• Regulated by the Charity Commission
• Obtain tax advantages but incur regulatory
  obligations
• Structure of sector well known through annual
  returns to Charity Commission
• Non-Registered NGOs
• No reliable data on number, size or activity
• Separate desk research exercise confirmed lack of
  reliable national information (though some local
  studies have been done)
• Best estimates suggest that unregistered NGOs
  account for around 20 - 30 of organisations (by
  number but less by income)

5
Panel Design

• Total sample of 5400 for robust analysis and
  sub-analysis
• Design limited by lack of available universe data
• Three stages of stratification
• Region
• Geographic locality based on organisations HO
• Only information available based on government
  funding invested in NGOs in that region
• Service sector
• To provide spread across all types of NGO
  (health, education, arts etc)
• Many organisations work across multiple sectors
• Region
• Income (proxy for size)

6
Panel Design (2) Income

• Good data on registered charities but data on
  non-registered universe confined to small scale
  regional studies.
• Structure therefore broadly based on registered
  charities though most non-registered are smaller
• Selection of five income bands based on
  compromise between pragmatism (too many is
  complex) and need to identify differences by size
• Ideal sample size for each income band also
  compromise

7
Panel Design (3) Final

• Final structure based on
• 5 income bands x 9 regions (interlocking)
• 10 service sectors (overall minimum quotas)
• Minimum number of organisations targeting
  specific groups (e.g. women, disabled people,
  older people)
• Optimistic target of 20 registered charities
• Actually achieved 15

8
Sample Sources

• A mixture of sample sources were used to try to
  obtain varied mix
• Again, good information available only for
  registered element of sample.
• Sources used included
• National advertising (reaching registered and non
  registered NGOs)
• Regional government sources (included small
  proportion of non-registered NGOs)
• Use of Charity Commission database (registered
  charities only)

9
Weighting and its Limitations

• Lack of knowledge about universe shape was major
  problem
• Inability to gross up and represent all universe
  has meant some applications are of reduced
  value/not feasible
• Need for internal consistency to track key
  variables over time
• Have chosen to weight all surveys to response to
  first postal survey
• Uses location and income as weighting matrix

10

• Errors using inadequate data are much less than
  those using no data at all.
• Charles Babbage

11
Data Collected (1) Surveys Undertaken

• Main programme each year includes
• Survey 1 Funding and Activities
• Survey 2 NGOs relationship with its clients and
  with other NGOs
• Survey 3 NGOs relationships with government
  (outside of funding)
• Survey 4 Human resources, IT and accommodation
• Survey (1) is postal and sent to all panel
  members
• Response rates in first three years 67 - 81
• Surveys (2) (4) undertaken by telephone with
  3600 respondents per survey

12
Data Collected (2) Meeting our Objectives

• Main findings relate to measuring increase in
  activity
• Activity identified as comprising 23 variables
  but critically
• Total funding from government
• Number of employees
• Number of volunteers
• Use index to measure change in activity based on
  median figures
• Based on median figures for key variables
• Set at 100 for year 1

13
Data Collected (3) Using the data

• Most of our information has never been
  systematically collected before
• Analysis is showing variations by size (most
  significant variable) but also by service sector,
  by region and by proportion of funding
• Panel also provides NGOs with an opportunity to
  feed their views and expectations back to
  government
• Panel members also approached for number of ad
  hoc studies in addition to main programme
• E.g. recent study of (small and medium)
  organisations operating locally explored their
  knowledge and experience of available funding
  sources

14
Examples of findings (1) Obstacles to activities

Unweighted Base All Respondents (4482)

• Shows that, after funding, accommodation is key
  problem
• Further investigation reveals that main reasons
  are
• Lack of space
• Lack of disabled facilities
• Insufficient funding to resolve difficulties in
  full

15
Examples of findings (2) Government Funding

Unweighted Base All Respondents (4482/3687)

• Shows shift of funding source from central
  government to local authority
• Provides evidence that government policy to
  devolve funding responsibilities is having a
  positive effect

16
Summary

• Main source of difficulties is lack of
  information on key element of NGO universe
• Pragmatic approach required to overcome
  limitations
• Have been able to track key variables over time
• Have not been able to gross up and provide much
  needed information about all NGOs
• Separate work now underway to obtain more
  structured information on non-regulated part of
  sector
• Main spin offs have resulted and government
  policy has firmer evidence base