EXPERIENCES OF THE EVALUATOR

1
EXPERIENCES OF THE EVALUATOR

• RNDr. Zuzana BOUKALOVÁ
• CROSSCZECH, CCSS, GEO Group

2
Professional Experience

• Charles University, Faculty of Science, Dept. of
  Hydrogeology, Civil Engineering
• Geological Institute of Czechoslovak Academy of
  Science - hydrogeologist
• University Politecnica de Valencia, Spain
  groundwater modelling
• HR Wallingford Ltd. water management

3
Back in Prague from 1996

• GEO Group, a.s.
• participation in the management, design and
  realization of groundwater and non-saturated zone
  pollution survey and remediation projects
• supervising expert in the team for government
  remediation programmes in groundwater and soil
  pollution.
• PHARE, PHARE-CREDO, ISPA and SAPARD project
  coordinator
• 5- th and 6-th FP projects managing director
  (CEGRMOMA, IRON CURTAIN, IMPACT, TRANSCAT,
  FLOODsite)

4
From the year 2000

• VZ GLS, a.s. as the 5.FP projects LOWRGREP and
  ENERGY FOREST managing director
• CROSSCZECH a.s. Head of the Department,
  Hydrogeologist, Consultant
• Czech Center for Strategic Studies (CCSS), the
  member of the Managing Board, ARMONIA project WP
  leader
• EDUCEUM the pool of 6 experts having a long
  lasting experience in different areas of European
  research and EU funding.

5
Evaluator, Rapporteur

• Marie Curie Actions (2003, 2004)
• eContent (2003, 2005)
• Member of the Sounding Board of the
  Commissioner Janez Potocnik, responsible for
  Science and Research within the European
  Commission (simplification of Framework Programme
  procedures, FP7), 2005

6
Marie Curie Actions

• Marie Curie Excellence Grants Excellence Grants,
  Awards, Chairs
• (remote evaluation)
• Marie Curie Conferences and Training Courses
• (remote evaluation)

7
Marie Curie Actions

• For the evaluation, the proposals are divided to
  8 areas (disciplines)
• CHE chemistry
• ECO economics
• ENG engineering
• ENV environment
• LIFE life sciences
• MAT mathematics and information society
• PHY physics
• SOC social sciences and humanities
• Section Z multidisciplinar projects

8
Marie Curie Actions

• Year 2004

Gender M F Total
Heiger Education 34 25 59
RES 13 12 25
IND 4 3 7
OTH 3 2 5
Total 54 42 96
9
MC Actions SCORE (evaluations by experts)

• 0 the proposal fails or cannot be judget
  against the criteria due to missing or uncomplete
  information
• 1 poor
• 2 fair
• 3 good
• 4 very good
• 5 excellent

10
MC actions EVALUTION

• Remote evaluation (3 evalators -independent
  experts- per 1 proposal) confidentiality and
  non-conflict of interest individual report forms
  (IRF)
• A rapporteur will be nominated to prepare the
  Consensus Report (CR) for a given proposal and
  obtain approval from the other evaluators
• Panel meeting in Brussel for each proposal a
  consensus should be reached and a CR will be
  prepared and signed by the triplet. One CR form
  per proposal.
• The experts will make a list of the proposals
  ranked
• The experts conclusions are examined by the EC
• Evaluation Summary Report (ESR) giving the
  opinion of the evaluators to the Coordinator of
  the proposal, on the basis of the Consensus Report

11
eContent program

• A multiannual Community programme to simulate the
  development and use of European digital content
  on the global networks and to promote the
  linguistic diversity in the Information Society
• eContenplus programme (May 2005)
• Call for experts!!!

12
eContent evaluation

• LUXEMBOURG
• 1 week evaluation
• Score 1 5
• Triplet per 6 10 proposals
• Panel meetings
• Evaluation and reports, all finished in
  Luxembourg

13
Review Procedure of the eContent proposals

• 1Unsatisfactory,
• 2Poor,
• 3Satisfactory,
• 4Good,
• 5Excellent

14
Agenda of the review

• Introduction, Objectives, context and purpose of
  a final review by PO
• Presentation Aim and progress of work
  (coordinator)
• Individual Work-packages Presentation and
  discussion (WP leaders)
• Demonstration of the Web/CD-ROM product
  (prototype) consortium
• Questions and answer session
• PO reviewers evaluation
• PO close, recommendations
• Successful completion Modify Rejected
• Review reports finished by experts from home till
  1 month

15
Sounding Board

• smaller actors in the context of simplification
  of Framework Programme procedures and
  implementation
• to incorporate the views of experienced
  stakeholders into the development of the Seventh
  Framework Programme (FP7)
• 3 4 meetings per year
• 1-st meeting 17.3. 2005

16
SB meeting

• Commission first ideas to achieve substantial
  simplification of the FP7
• FP Action Plan on Rationalisation and
  Acceleration
• I. Actions to simplify and accelerate
• II. Actions to improve quality and
  effectiveness

17
Actions to improve quality and effectiveness
1

• The established sets of uniform beings will be
  the basis for briefing all external evaluators to
  ensure consistency of approach
• DONE standard briefing available on internal
  website

18
Actions to improve quality and effectiveness
2

• Evaluation Summary Reports sent to proposers MUST
  always be of high quality
• EC will closely monitor the output from consensus
  groups and panels
• DONE introduced in evaluation workshops and
  briefings

19
Actions to improve quality and effectiveness
3

• Quality of evaluators!!!
• may include contacts by senior officials with
  industrial umbrella groups, requesting them to
  nominate highly qualified individuals experts
• ONGOING

20
Actions to improve quality and effectiveness
4

• Further use of two stage proposal submission and
  evaluation (for IP, NoE,? STREP ?)
  more experts for evaluations
• Guidelines - revised first stage is to be as
  light as possible, based on limited number of
  criteria with limited administrative data
• The second stage will be based on FULL SET of
  evaluation criteria
• DONE

21
Actions to improve quality and effectiveness
5

• Rules on annual reviews of the projects will be
  developed
• ONGOING
• Review guidelines to be finalised and put on
  CORDIS

22
THANKS for your attention !

• Zuzana Boukalova
• Contact
• Zuzana.boukalova_at_crossczech.cz
• boukalova_at_geo-praha.cz