Indicators to characterize public funding systems

1
Indicators to characterize public funding systems

• Benedetto Lepori, 28th November 2008

2
The issue

• Is it possible to produce indicators to
  characterize/compare public funding systems
  including
• The role of different funding agencies
• The allocation methods
• The streams of money
• The beneficiaries
• Beyond RD statistics which is essentially
  focused on performers
• With limited ability to map funders performers
  flows
• With a too simple description of funders (by
  sector)
• Not taking into account the existence of
  intermediaries
• Obvious relevance for public policies of these
  indicators
• See for example the OECD work on steering and
  funding of public research where no data on
  allocation could be collected
• Two PRIME project on the field
• Exploratory project on public project funding
• Analysis of CEEC funding systems and of their
  change

3
An overall view of public funding

• Four relevant system layers
• State
• Funding agencies
• Performers organisations
• Research groups/individuals)
• Two main allocation streams
• core funding
• project funding
• Key issues
• Interactions accross layers and funding modes
• Increasing role of performers in driving funding

4
National state
Policy layer
Funding agencies (national)
International agencies (EU, ESA, etc.)
Agencies layer
Core funding
Project funding
Organisations layer
Higher Education Institutions and PROs
Research units
Research groups layer
Internal allocation
5
Project funding analysis

• Try to compare countries concerning the role and
  composition on project funding
• Identify in each country the instruments which we
  can identify as project funding
• As well as the main funding agencies
• A basic definition a lot of comparative work to
  treat in the same way the same instruments
• Collect the data from different sources
• Ministries reports, agencies reports, etc.
• Per year and main beneficiary group
• Data collection and cleaning procedures
• Produce aggregations and international
  comparisons
• As well as analyses of the evolution in last 30
  years

6
Criteria

• We consider project funding if
• Attributed mostly for research purposes
• Limited in time and scope (not recurrent)
• Attributed by an external agency to the research
  organization
• No reference to competition/modes of allocation
• Issues / problems
• Use for research activities (since we are not
  looking to performers) difficult to assess for
  example for contracts
• Long-term competitive schemes (centres of
  excellence)
• Internal competitive schemes to research
  organizations (or vertically integrated
  organizations like academy of sciences)

7
Problem cases

• European Space Agency contracts
• Mostly to industry for development of rockets,
  satellites, etc.
• Some funding for scientific programs
• It is nearly impossible to ascertain how much
  should be considered as RD
• We adopted a broad delimitation
• Including probably more that what would be
  considered as RD funding

8
French CNRS

• A very large organization with a double role
• Funding its own laboratories (10)
• Funding joint laboratories with universities
  (thus partially outside the CNRS perimeter)
• A partially competitive allocation mechanism
• Labelling by CNRS gives access to its resources
  and there is a rather high turnover
• Mostly in form of personnel, but no mandatory
  allocation
• It is something intermediary between general and
  project funds
• Accounting for nearly 10 of public research
  funding in France, thus impossible to put in a
  footnote
• Our picture of the French system largely depends
  on the choice we do concerning these funds
• Considering it as project gives a view of France
  much nearer to other Western European countries
• See Theves et al. paper

9
Discussion

• Overall the distinction works and is usable but
• One needs much care in the comparisons especially
  for time series
• We should try to develop a finer typology of
  funding instruments in the future
• Intensive discussion was needed to solve dubious
  cases
• Ensuring some comparability
• In-depth knowledge of national systems was
  essential for this exercise
• this is not the end of the story

10
Categories

• Comparative analysis clearly needs common
  categories
• Beyond individual list of instruments/agencies at
  national level
• We devise three classifications
• By type of agency (international, ministry,
  intermediaries)
• By instrument type
• By beneficiaries

11
Instrument types

• Intuitively it is clear that there are different
  instruments concerning their orientation / type
  of research they fund
• But designing a clear classification is very
  difficult since these concepts are largely
  multidimensional
• We end up with a simple distinction between
  academic / thematic / innovation-oriented
  instruments
• However, this is just a rough approximation of
  policy intentions, but not necessarily of the
  research done
• Different features need to be combined to better
  understand the significance of instruments for
  the research system
• Granularity is a problem since the level of
  aggregation is rather high

12
Data sources and data collection issues

• There is no unique source of these data, but they
  had to be complied from
• State accounts (France)
• Research ministry reports
• Agency reports
• Some existing databases
• Surveys (for example the Swiss RD survey)
• Direct inquiry to ministries
• Most data are available but with
  limitations/problems in many cases
• In most cases it was possible to come back to the
  70

13
Data problems

• Funding to companies through loans
• Estimate of cash value
• Project decisions instead of allocations
• One needs some kind of averaging
• European Space Agency
• Use national contribution as proxy (national
  return rule)
• Contracts from ministries
• Coverage is problematic outside the formal
  programs
• even if the methdology is simple there is a lot
  of dirty work to get usable data

14
Some selected results

• PF as the second stream of research funding in
  the six considered countries
• ¼ to 1/3 of funding volume
• Strong increase in the role in the last years
• A composite model of funding instruments
• Some academic, some thematic, some innovation
• Try to accomodate the different goals instead of
  chosing a single rationale
• Differences between countries remain quite large
• In the organisation of funding agencies and the
  type of funding
• Strong dependence on history of organisational
  structures
• One needs to consider carefully this context when
  designing European funding policies

15
Type of PF instruments
16
PF as of BIP
17
From academic to thematic and back
Thematic instruments
Academic (bottom-up) instruments
18
From steering to markets

• Todays highly differentiated project funding
  system resembles increasing to a market
• Where loosely coordinated agencies buy research
  services from performers
• With strong strategic behavior of applicants
• Depending on their strenghts, needs, etc.
• Building stable market structures
• Needed to support long-term research, but also
  predictable allocation of funding
• Crucially based on shared representations on
  others actors behaviour
• Of funding agencies
• Of potential competitors
• Some evidence that core markets are relatively
  small-scale
• Strong segmentation by domain/type of
  research/topic
• Few key players with high success rates, many
  marginal players

19
CEEC project

• To analyse the changes in the organization of
  public funding in CEEC
• Three countries as a test (Czech Republic,
  Estonia, Poland)
• Look also for availability of data (including
  time series)
• Approach
• Identify the main funding streams and to
  characterize them
• Draw structural diagrams
• Understand the role of funding agencies and
  allocation criteria
• See Karels presentation for more details on
  specific results

20
Poland
21
Structural models

• A step towards identifying the main structural
  features of national funding models
• Taking into account the interactions between
  modes and layers (institutional
  complementarities)
• Different models related to different national
  contexts
• Which can be stable in an evolutionary
  perspective
• Characterize the models in terms of some key
  features
• Ability of the State of steering the system
• Levels of delegation
• Performance vs. innovation
• Three basic models seems to emerge
• Project-funding based
• Centralised
• Mixed models

22
The project-funding based model

• Most of research funding coming from highly
  differentiated set of non coordnated agencies
• High share of project funding and low of core
  funding
• State  buys  research services from groups
• With limited attempt of central coordination
• Mission agencies funding also basic research
  (DoD)
• Reliance on competition rather than on
  coordination to get the required results
• stratified university system with concentration
  in a few players
• International excellence as the main criterion
• Limitations of this model when the system is too
  small
• Cumulative effects suppress diversity of the
  system (the Estonian case)

23
The centralised model

• A single large research organisation funded by
  the state (outside the university sector)
• The old CNRS model before joint laboratories
• The old system of Academy of Sciences in
  communist countries
• The todays Polish system
• The PRO has more functions than just funding
• Decision on priority areas
• Creation and restructuring of laboratories
• Mostly coopting scientists in the PRO board
• A strongly centralised system
• Define priorities and concentrate effort
• Central planning, but with all its rigidities in
  face of new scientific development
• Less adapted to new science dynamics more based
  on complementarities than on heavvy investments
  (new  search regimes  Bonaccorsi)

24
The mixed model

• Some balance between core and project funding
• Say 60 to 40
• A differentiated performers sector
• Large university sector alongside a number of
  non-university research institutes
• Trying to achieve some balance between
• Targeting the best groups and spreading out
  resources (also because of the link with
  education and regional development)
• Differentiating the performers sectors to answer
  to different missions vs. creating competition
  between performers
• Typical of most Continental European countries
  including Switzerland
• Best performing in small rich countries rather
  that on large ones
• Is the efficiency loss and the lack of central
  coordination acceptable?
• The Czech republic as a typical case

25
Conclusions (1)

• It is basically feasible to characterize funding
  systems using existing data at national level
• Mostly detailed budgetary data
• Complemented with data from agencies and RD
  statistics
• A very good knowledge of national systems is
  required
• To identify and classify funding streams
• To identify the really relevant features
• National experts are required for these tasks

26
Conclusions (2)

• This works provides very interesting insights on
  the structure of funding systems
• Quantitative indicators are essential to
  characterize them and to distinguish variants
• The next avenue is to go towards data on
  micro-structures of funding systems

27
References(see www.enid-europe.org)

• Lepori B., van den Besselaar P., Dinges M., van
  der Meulen B., Potì B., Reale E., Slipersaeter
  S., Theves J., (2007), Comparing the Evolution of
  National Research Policies what Patterns of
  Change?, Science and Public Policy 34 (6),
  372-388.
• Lepori B., Dinges M., Potì B., Reale E.,
  Slipersaeter S., Theves J., van d. (2007).
  Indicators for Comparative Analysis of Public
  Project Funding. Research Evaluation, 16 (4),
  243-255.
• Lepori B., Masso J., Jablecka J., Sima K.,
  Ukrainski K. (2008). Research funding system in
  Central and Eastern European countries a
  comparative analysis. Paper presented at the
  ENID-PRIME Indicators Conference, Olso, May 2008.