The Policy makers' evaluation use

1

• The Policy makers' evaluation use between
  rationalizing and legitimating
• Dr. Daniel Tsygankov
• Head of the Center for Program and Policy
  Evaluation
• (Russia, Moscow)
• European
  Evaluation Society Conference - 2008
• Lisboa, 2. October 2008

2
Introduction

• In recent decades the number of evaluation
  research carried out is rapidly growing now this
  trend is also referred to states of the former
  Soviet block and China.
• A lot of countries in transit have already
  entered the influence orbit of countries and
  international organizations (e.g. EU) with formed
  evaluation institutes.
• Which risks exist for Russia / CIS states?
  Evaluation is developing here "wildly",
  fragmentarily, with own (not yet clear)
  scenarios.

3
Cooperations models between policy maker and
expert / scientist

• Technocracy (expert adapts himself to policy
  maker demands)
• Meritocracy (policy maker has no idea, and
  expert decides everything from theme
  formulation up to the research design)
• Policy Learning (both sides find mutual
  understanding by multistage negotiations).

4
Knowledge utilization? (1)

• Simplifying the policy makers taking political
  and administrative decisions pursue two groups of
  objectives
• scientifically centralized objectives (knowledge
  may increase the rationality of political
  practice)
• applied objectives
• the legitimating of decisions / proposals
• political win.

5
Knowledge utilization? (2)

• In accordance with this fact
• the supporters of amplification of
  use/integration of evaluation into public sector
  speak of challenges of modern administration
  efficiency(eg. NPM, Performance Audit, etc.)
• the critics refer to the verification rituals
  of modern auditing society (Michael Power) and
  forming of new audit culture (Marylin Strathern)
  which can affect social reality (e.g. by means of
  forming of performance indicators system) .
• Here we put aside those evaluations and surveys
  which are objectively useless due to their
  methodological mistakes.

6
Evaluation percolation through fields of power

• On the one hand, evaluation results use in
  political practice is still not a general rule.
• gt The information on evaluation, its
  concepts and arguments is gradually percolating
  through society by means of force groups, expert
  influence and mass-media.
• On the other hand, evaluation as new
  cross-border discipline beside management and
  marketing research compares favourably with
  policy science, sociology and philosophy.
• The advocates of evaluation implementation
  themselves perceive both weak professionalism in
  some countries and shapeless evaluation
  (everything that names itself in such a way is
  accepted just so by those as well) with an
  anxiety, and neatly characterize them as
  pseudo-evaluations or quasi-evaluations.

7
Rationalizing vs. Legitimating

• Therefore the development of evaluation as public
  practice and research strategy can be
  analytically placed on the axis between the
  poles
• Rationalizing ltgt Legitimating
• For further analysis of chances of its
  institutionalization the new dimensions can be
  introduced
• country institution environment
• waves of public sector reforms
• development of boundary science disciplines
• structure of the power field / the intellectual
  field, etc.

8
Attributes of the "non-rational" intentions

• The peculiarities of formulating of conditions
  and evaluation topics by policy makers can point
  out such a removal from rational pole
• the topic is contradictorily, confusingly and at
  length formulated
• policy maker (or his representative) avoids
  explaining research topics
• research reports are never published
• the companies which are absolutely unknown in the
  market participate in tenders
• the price of the contract obviously contains a
  corrupt constituent.

9
Conclusion Acknowledgment

• The understanding of consistent patterns of such
  development possesses not only pure scientific
  interest, but it is also important in principle
  for the countries in transit which did not enter
  the influence orbit of the formed evaluation
  culture. For such countries there is a risk the
  relegation of evaluation to the Legitimating
  pole is high.
• Therefore there is a question whether the
  existing approaches (creation of national
  association and work-out of evaluation standards,
  development of master programs and programs of
  further education, publication of magazines and
  books in mother-tongue, etc.) will be enough for
  risks reduction? Or
• Whether it will be necessary to take non-standard
  steps which will contribute to more rational use
  of evaluation as cross-border discipline?

10
Contact information

• Head of CPPE Dr. Daniel Tsygankov
• Address in Russia 125493, Moscow, Smolnaya
  street, 10.
• Phone [phone]
• E-mail mail_at_iopp.ru
• Web www.cppe.info (English site)
• evaluationrussia.wordpress.com
  (English blog)
• www.iopp.ru (Russian site)
• European office (Paris) Maria Smirnova
• E-mail smirnova_at_iopp.ru