Diapositiva 1

1
Session 4. Evaluation of publicly funded
research development and innovation The
Integral Monitoring and Evaluation System
(SISE) Alfonso Beltrán García-Echániz Director
General Adjunto Fundación Española para la
Ciencia y la Tecnología
2
The Integral Monitoring and Evaluation System
(SISE)
WHAT IS SISE?

• SISE was launched by the Central Government
  together with the National Plan for RDI
  2004-2007
• It is a tool for
• Publicizing their activities and making them more
  transparent, thus providing the Spanish society
  with a better understanding of RDI activities
  funded with public money
• Controlling the management of publicly funded
  RDI programs
• Producing all data required for evaluating the
  RDI public policy
• The rationale behind SISE is the need for
  decision makers to count in with a constant
  feedback about the implementation and the
  outcomes of their policies

3
The Integral Monitoring and Evaluation System
(SISE)
WHAT IS SISE?

• SISE works as the permanent observatory on the
  Spanish Science and Technology System (SECYT)
• SISE promotes analysis and studies that provides
  decision makers with a deeper and more accurate
  understanding of the Spanish RDI System

4
The Integral Monitoring and Evaluation System
(SISE)
MAIN GOALS

1. Identifying which are the main science and
   technology indicators
2. Evaluating the Spanish RDI public policies and
   monitoring their impacts on the evolution of the
   science and technology system
3. Developing recommendations and proposals for
   improving the efficiency of these public policies
4. Disseminating all SISE outputs among the actors
   of the Spanish Science and Technology System
5. Contributing to a better data provision,
   including information related to regions
   performance, and womens participation in the
   RDI system

5
The Integral Monitoring and Evaluation System
(SISE)
STRUCTURE
6
The Integral Monitoring and Evaluation System
(SISE)
SISE in the National Plan for RDI 2008-2011
7
The Integral Monitoring and Evaluation System
(SISE)
SISE in the National Plan for RDI 2008-2011

• SISE has been included in the National Plan for
  RDI 2008-2011 as the key mechanism for
  monitoring and evaluating research and innovation
  policies, in order to review current activities
  and identify the need for new initiatives
• Three evaluation levels operate
• Ex-ante evaluations Focused on evaluating
  proposals
• On-going evaluations Through the RDI National
  Plans Annual Report, and the Working Programs
  Monitoring Report
• Ex-post evaluations Focused on the outcomes and
  the outputs of publicly funded RDI activities.
  ? PROEVAR

8
The Integral Monitoring and Evaluation System
(SISE)
SISE OUTPUTS

• Three SISE Reports have been produced so far
• The SISE Report 2005
• The SISE Report 2006
• The SISE Report 2007

Each of these Reports contained specific
recommendations for the Interministerial
Commission for Science and Technology (CICYT).
Most of the recommendations were converted into
specific political measures.
9
The Integral Monitoring and Evaluation System
(SISE)
Main Outputs and Outcomes - SISE Report 2005

• To create a reflection forum for the promotion of
  a National Strategy on Science and Technology
  that encompass a new National Plan for Science,
  Technology and Innovation. (December 2006 ENCYT ?
  July 2007 National Plan 2008-2011)
• To design an All-Round Popular Science
  Communication Plan that fostered scientific and
  technological culture. (? 2007 Science Year)
• To publish before January the annual Working
  Program of the National Plan, containing all the
  forthcoming RDI calls and official
  announcements, as well the amount of money
  awarded to them, and what are the management
  units responsible for each of them. (? Working
  Program 2006 and Working Program 2007)

10
The Integral Monitoring and Evaluation System
(SISE)
Main Outputs and Outcomes - SISE Report 2005 (II)

• To take into consideration a division between
  civil RDI entries and military RDI entries in
  the National Budget. (? National Budget 2007)
• To try to increase the participation of public
  research centres and private enterprises in the
  European Framework Program (? EUROINGENIO 2007)

11
The Integral Monitoring and Evaluation System
(SISE)
Main Outputs and Outcomes - SISE Report 2006

• The National Plan for RDI 2008-2011 should
  include a set of indicators to measure its
  management and its outputs.
• The National Plan for RDI 2008-2011 should be
  submitted to the Permanent Commission before the
  end of July 2007, in order for it to be passed in
  accordance with the Advisory Board and the
  Science and Technology Governing Body. (?Passed
  through the Permanent Commission, the Advisory
  Board, and the Standing Commission in June and
  July 2007 Passed through the Cabinet in
  September 2007)

12
The Integral Monitoring and Evaluation System
(SISE)
Main Outputs and Outcomes - SISE Report 2006 (II)

• To develop an electronic platform for collecting
  data regarding all public calls for RDI
  projects, both from the Central Administration
  and from the Regional Governments. (? SISE
  Platform 2007)
• To include in the annual Working Program all
  Regions RDI public policies, in order to
  promote a fluent dialogue between the Autonomous
  Regions and the Central Government. (? Working
  Program 2008)

13
The Integral Monitoring and Evaluation System
(SISE)
Main Outputs and Outcomes - SISE Report 2007

1. To carry out a pilot study on public sector
   technology purchase under the legal figure of
   public-private collaboration contract. (The
   Secretary of State for Telecommunications and the
   Information Society, together with the Public
   Administration Ministry).
2. To develop an All-Round Popular Science
   Communication Plan (PICODI) for disseminating
   scientific and technological culture. (The
   Spanish Foundation for Science and Technology -
   FECYT).
3. To develop an in-depth evaluation analysis on the
   2000-2003 National Plan. (FECYT).

14
The Integral Monitoring and Evaluation System
(SISE)
Main Outputs and Outcomes - SISE Report 2007 (II)

• To analyse which are the most suitable mechanisms
  to integrate the Autonomous Regions within the
  National Plans actions towards human resources,
  infrastructures, and institutional strengthening.
  (Ministry for Science and Innovation).
• To evaluate the use of the financial expenditures
  allocated to RD through the National Budget
  (Ministry For Economy and Finances).

All these recommendations have been included in
the Working Program for 2008
15
The Integral Monitoring and Evaluation System
(SISE)
The Working Program for 2008

• The Working Program is the annual updating tool
  of the National Plan. It includes
• A full timetable with all forthcoming public
  calls, including deadlines and management units.
• An expenditures distribution per knowledge areas
  and priority programs.
• A review on the strategic actions.
• The evolution of the carry out of the Working
  Program in followed through the Monitoring
  Report. It includes
• A follow-up of the public calls published on the
  National or Regional Bulletins
• An analysis of each call content.
• Half-yearly reports on the Working Program
  development.