The Regional Dimensions of Evaluation

1

• The Regional Dimensions of Evaluation
• Issues, Particularities and potential Pitfalls
• 6 CP Spring Workshop 2007, Dublin
• JOANNEUM RESEARCH-Institute for Technology and
  Regional Policy
• Christian Hartmann
• christian.hartmann_at_joanneum.at
• Wolfgang Polt
• wolfgang.polt_at_joanneum.at

INNOVATION aus TRADITION
2
Overview

• Recent trends in Evaluation
• The Regional Dimension
• The typical regional Evaluation Project
• Particularities and potential Pitfalls

3
Recent Trends in Evaluation

• Increased emphasis on impact assessment
• ex-ante (? priority setting, ? links to
  foresight)
• ex-post (? economic effects, ? behavioural
  additionality)
• Widening the scope IA of basic research
• Concept evaluations (ex-ante, interim)
• Checking the rationale and early implementation
• Systemic evaluations
• OECD (CSTP Oct 2006, Seoul) asked countries to
  improve the capacity to carry out Systemic
  evaluations
• Addressing Innovations systems as a whole
  (TrendCHART ERAwatch, EIS, OECD STI
  Scoreboard,..)
• Addressing the Policy Mixes of countries
• Addressing portfolios RTDI policy instruments
  (e.g. direct vs indirect support for private RD)
• Asking for the impossible..?

4
Evaluation in the policy cycle
5
Evaluation needs a systemic (mulit-actor,
multi-level) perspective
international
national
regional
6

• The Regional Dimension

7
Why does the regional dimension of evaluation
matter?

• Regional Innovation Systems
• Globalisation leads to a growing importance of
  localized innovation capabilities, that are hard
  to copy / to transfer
• The growing importance of regional innovation
  systems in the economic literature (Asheim et al.
  2003, Cooke 2003, Malmberg and Maskell 2002)
  leads a stronger focus on the regional level in
  the policy domain
• The concept of regional innovation systems offers
  effective (low cost) roads for policy delivery
• Regionalisation of RTDI Policy
• Devolution of (innovation) policy competencies in
  some EU countries (e.g. France, England)
• Growing interest in RTDI policy at regional level
  in federal provinces or autonomous regions in EU
  countries
• Growing importance of regional RTDI policy at
  EU-Level (i.e. RIS/RITTS, Regions of knowledge)

8
Strategic Policy Intelligence in the Regional
Dimension

• Different Levels of Development among European
  Regions
• While some European regions already have a long
  existing practice (like Styria, Basque Country,
  etc.)
• in particular regions in the NMS are just at the
  beginning to develop respective structures and
  processes.
• Different Degrees of Freedom for SPI in European
  Regions
• Size of the region and corresponding capacities
  for SPI
• i.e. Catalunya vs. Burgenland
• Degree of regional autonomy for RTDI policy
• i.e. Friuli Venezia Giulia (IT) vs.
  South-Transdanubia (HUN)

9

• The typical regional Evaluation Project

10
Is there such a thing as a typical regional
evaluation project?

• How typical is typical - regional evaluation
  projects do actually cover a wide range of issues
  and policy areas
• Regional Evaluation is often Evaluation of
  Regional Policy (i.e. Structural Funds)
• Evaluation in the regional dimension does often
  mean Evaluation of small and medium sized
  programmes with regional funding
• Regional Cluster Initiatives
• Regional/Municipal Economic Programmes
• Evaluation of regional strategic projects
• Regional Innovation Infrastructures (i.e.
  Technology / Science Parks)
• Key projects in the framework of Community
  Initiatives

11
Are typical regional evaluation projects
following the general trends?

• Typical regional evaluation projects do reflect
  general trends in evaluation - but with varying
  degrees of intensity
• Concept evaluations do already play a big role on
  the regional level
• Structural Funds
• Increasing emphasis of impact assessments on
  regional level
• Regional RTDI Programmes
• Strategic Projects
• Systemic evaluations are at the moment not (yet)
  of relevance

12
The typical regional evaluation project and
Strategic Policy Intelligence

• No (complete) policy cycle for RTDI policy in
  most European regions
• Reluctance to confront the past
• Evaluations are often motivated by the necessity
  to letigimate the past
• Evaluations as tools in regional political games
• There are natural constraints for SPI
  corresponding to the Size and political status of
  the Region
• Lack of policy capacities
• Lack of degrees of freedom for SPI

13

• Potential Pitfalls

14
Potential Pitfalls of evaluation at the regional
level

• Roughly the same that face evaluations on all
  levels, but more accentuated
• Lack of capacity to carry out evaluations and
  to absorb and implement the results of
  evaluations
• Difficulties of role delineation the evaluator
  as policy maker? Too close to policy
• Potential to influence the evaluation results
  because of closeness of policy makers to
  stakeholders
• Expectations too high (especially with respect to
  impact assessment)