Monitoring R

1
Monitoring RD as a Methodological Tool for
Impact Studies

• Stig Slipersæter
• Norwegian Institute for Studies in Research and
  Higher Education
• Centre for Innovation Research

2
Background

• Lack of international standards for output and
  impact indicators
• Small steps to integrate outcome data relevant
  for impact studies into Norway's national RD
  statistics

3
Monitoring RD

• An example on monitoring
• An example on use of data
• Lessons learned

4
Example Research institutes

• 65 independent research institutes
• Annual budget about 1 billion US
• Approximately 7000 employees
• Monitoring scheme run by NIFU on behalf of the
  Research Council of Norway
• Purpose Accountability and policy analysis
• Integrated with regular RD statistics

5
Input to research activities

• Financial resources split on
• Government funds
• Normal research contracts
• Programme allowances from research council
• Public sources, split on various ministries and
  their agencies, local and regional authorities
  etc.
• Domestic industry, split into three categories
• Contracts from abroad, split into seven
  categories
• Detailed data on human capital

6
Output from research activites

• Indicators for output
• Enlarged set of variables for printed
  dissemination
• Articles in scientific journals
• Different kinds of reports
• Books and chapters in books
• Articles in magazines of professional
  associations
• Popularization
• Patents applied for and granted
• Spin-offs
• Partcipatons at conferences, seminars etc

7
Indicators for diffusion

• In- and outgoing mobility
• Shared positions (with universities, industry and
  others)
• Cooperative research projects
• Supervising of students
• Visiting students
• Visiting scholars (in- and outgoing)

8
Advantages of the scheme

• Details on income
• An extended set of variables on outcome
• Variables on different diffusion processes

9
Utilising the institute monitor

• Users evaluation of applied social research at 31
  research institutes
• Analysis of financial data showed
• 12 institutes serving regional and local
  authorities
• 19 institutes serving national authorities and
  international organisations
• How is differences in customer base affecting the
  outputs of the institutes?

10
Differences in written output
11
What purposes were RD used for?

• User survey shows two main categories of use
• Instrumental New facts and policy support
• Conceptual New ideas, concepts, methods and ways
  of understanding
• Output from national and academically oriented
  institutes considered being of largest
  instrumental value for users

12
Lessons learned for indicators 1

• There are many ways of transferring scientific
  results Broad set of transfer mechanisms should
  be included
• Use of output is far from obvious Impact can
  come in unexpected ways
• Ask the user of scientific results what is
  important before making the indicator.

13
Lessons learned for indicators 2

• When setting up a system of indicators, consider
• A broad set of types of output
• Include at least these groups in the process
• Producers of output (i.e. scientists) and the
  ones reporting (i.e. administrators)
• Users of output
• Users of indicators
• The importance of time series