Proposal submission and evaluation procedures in FP6

1
Proposal submission and evaluation procedures in
FP6

• Marina Marchetti
• DG Research

2
Proposal evaluation in FP6

• New approaches, new instruments for FP6
• Re-examined evaluation procedures from first
  principles
• Examined best practice in other systems
• Sought advice (EURAB)
• Kept best features from FP5, improved quality
  overall
• Require fast, transparent, equal and impartial
  process, understood by all.

3
Overview Evaluation Process
Panel with optional Hearings
Submission
Consensus
Individual reading
Commission Follow-up
Final ranking list
Evaluators
Evaluators
Evaluators
Criteria
Criteria
Criteria
Rejection list
Questions
Proposals in suggested priority order
Eligibility
4
Overview Evaluation Process
Panel with optional Hearings
Submission
Consensus
Individual reading
Commission Follow-up
Final ranking list
Evaluators
Evaluators
Evaluators
Criteria
Criteria
Criteria
Rejection list
Questions
Proposals in suggested priority order
Eligibility
5
Full / short proposal

• One-stage submission of proposals
• Full proposal with all details
• Set of criteria defined in the work programme
• Two-stage submission of proposals (optional)
• First stage
• short proposal (about 10-20 pages)
• use of limited set of criteria
• successful proposers invited to complete
  proposals
• Second stage - like one-stage submission
• defined in the work programme

6
The Parts/Content of a proposal

• PART B - PROPOSAL CONTENT
• Objectives and expected impact
• Work plan and associated budget
• Consortium and the project resources
• Project management
• Exploitation and dissemination plans
• Ethics, safety and other issues (where
  relevant)
• See Info-Pack/Guide for Proposers for precise
  details

PART A - FORMS
7
The proposal forms part A - what they contain

• A1 - General info on the proposal
• A2 - Info on the coordinator and other partners
• One form per partner
• A3 - Cost breakdown
• One sheet for the whole consortium

8
Form A1 General info on the proposal
Title, abstract, keywords etc.
9
Form A2 Info on the co-ordinatorand other
partners
(1 sheet per partner)
organisation name, address, legal status,
activity type, SME yes/no etc.
10
Form A3 Cost breakdown by type of activity and
by partner
type of activity research, innovation,
demonstration, training, management
(one line by partner, i.e. only one sheet for
the consortium)
11
Proposal submission

• Electronic proposal submission system
• Web-based online tool
• Offline tool
• format for electronic files PDF
• 2 means of submission
• Prepare on-line, submit on-line
• Prepare off-line, submit on-line

12
Eligibility checks

• Date and time of receipt of proposal on or before
  deadline for receipt
• Firm deadlines
• Minimum number of eligible, independent partners
• As set out in work programme/call
• Completeness of proposal
• Presence of all requested forms

13
Overview Evaluation Process
Panel with optional Hearings
Submission
Consensus
Individual reading
Commission Follow-up
Final ranking list
Evaluators
Evaluators
Evaluators
Criteria
Criteria
Criteria
Rejection list
Questions
Proposals in suggested priority order
Eligibility
14
Evaluators (1)

• High quality independent evaluators as core of
  evaluation system
• Wider pool of evaluators
• Call for applications from individuals
• Call addressed to institutions with a view to
  establishing lists of suitable independent
  experts
• Commission may select individuals with the
  appropriate skills from outside these lists

15
Evaluators (2)

• Qualification
• Highest level professionals from science,
  industry and/or innovation and internationally
  recognised authorities
• Some will participate in the hearings with the
  consortia
• Sign confidentiality and conflict of interest
  declaration
• Names published after the evaluations

16
Evaluators (3)

• EC target for women evaluators 40

17
Evaluators (4)

• Selections criteria for experts
• - An appropriate range of competencies
• - An appropriate balance between academic and
  industrial expertise and users
• - A reasonable gender balance
• - A reasonable geographical distribution
• - A regular rotation of independent experts.

18
The evaluation criteria

• Criteria adapted to each instrument
• may vary within the different areas
• are specified in the work programme
• Codification of ethical review procedures
• Gender issues fully integrated
• Science and society issues
• Different weights and thresholds may be applied
  to the criteria

19
Basic set of issues for Networks of Excellence

• Relevance
• Potential impact
• Excellence of participants
• Degree of integration and the Joint Programme of
  Activities (JPA)
• Organisation and management

20
Basic set of issues for Networks of Excellence

• Relevance
• The extent to which the proposed project
  addresses the objectives of the work programme.

21
Basic set of issues for Networks of Excellence

• Potential Impact
• The extent to which the proposed project
• Addresses the EU strategic need to strengthen ST
  excellence on the topic, by means of a
  restructuring of existing research capabilities
  and the way research is carried out
• The goals of the network are suitably ambitious,
  particularly in terms of achieving EU leadership
  and acting as a world force on this topic
• The proposal shows a clear added value in
  carrying out the work at EU level and takes
  account of researches activities at national
  level and under European initiatives (Eureka).

22
Basic set of issues for Networks of Excellence

• Potential Impact (2)
• The extent to which the proposed project
• Has an effective plan for spreading excellence,
  exploiting results and dissemination knowledge to
  those outside the network
• Presents as approach likely to have a durable
  structuring impact on European research.

23
Basic set of issues for Networks of Excellence

• Excellence of the participants
• The extent to which
• The participants are currently conducting
  excellent research relevant to the topic of the
  network or are capable of important contributions
  to the JPA
• The participants are well suited to the tasks
  assigned to them
• They collectively have the necessary critical
  mass of expertise and resources to carry out the
  JPA successfully.

24
Basic set of issues for Networks of Excellence

• Degree of integration and the JPA
• - The extent to which
• - The expected degree of integration justifies
  supporting the proposal as a NoE
• - The Joint Programme of Activities is
  sufficiently well designed to achieve the
  expected degree of integration
• - The participating organisations have made a
  convincing commitment towards a deep and durable
  integration continuing beyond the period of EC
  support

25
Basic set of issues for Networks of Excellence

• Organisation and Management
• The extent to which
• The organisation a structure of the network
  provides a secure framework for any necessary
  structural decisions to be taken
• The management of the network is demonstrably
  high quality
• There is a well considered plan for promoting
  gender equality in the network.

26
Basic set of issues for Integrated Projects

• Relevance
• Potential impact
• Scientific and technological excellence
• Quality of the consortium
• Quality of the management
• Mobilisation of resources

27
Basic set of issues for Integrated Projects

• Relevance
• The extent to which
• the proposed project addresses the objectives
  of the work programme in the areas open for the
  particular call.

28
Basic set of issues for Integrated Projects

• Potential Impact
• The extent to which
• the proposed project is suitably ambitious in
  terms of its strategic impact on reinforcing
  competitiveness or on solving societal problems
• the innovation-related activities and
  exploitation and/ or dissemination plans are
  adequate to ensure optimal use of the project
  results
• the proposed project demonstrates a clear added
  value in carrying out the work at EU level and
  takes account of research activities at national
  level and under European initiatives (e.g.
  Eureka).

29
Basic set of issues for Integrated Projects

• ST Excellence
• The extent to which
• the project has clearly defined objectives
• the objectives represent clear progress beyond
  the current state-of-the-art
• the proposed ST approach is likely to enable the
  project to achieve its objectives in research and
  innovation.

30
Basic set of issues for Integrated Projects

• Quality of the Consortium
• The extent to which participants
• - Constitute a high quality consortium
• Are well-suited and committed to the tasks
  assigned to them
• Are complementary their profiles, including of
  those to be included later, have been clearly
  described
• The real involvement of SMEs has been adequately
  addressed.

31
Basic set of issues for Integrated Projects

• Quality of the Management
• The extent to which
• - the organisational structure is well-matched to
  the complexity of the project and to the degree
  of integration required
• - the project management is demonstrably of high
  quality
• - there is a satisfactory plan for the management
  of knowledge, of intellectual property and of
  other innovation-related activities.

32
Individual reading

• By 3 or more evaluators (5 for NoE and IP)
• May take place remotely
• Marks and comments for each block of criteria
• Individual evaluation form

33
Consensus

• Build on the basis of the individual assessments
  of all the evaluators
• A discussion may be convened
• Agreement on consensus marks and comments for
  each of the blocks of criteria
• Overall Consensus Report
• After the consensus weightings and thresholds
  applied, when used
• as set out in the work programme

34
Panel with optional Hearings
Submission
Consensus
Individual reading
Commission Follow-up
Panel (with optional hearings)

• Panel Meeting
• Final marks and comments for each proposal
• Suggestions on order of priority, clustering,
  amendments, etc.
• Hearings with proposers may be convened
• As set out in the work programme
• Questions to the invited proposal coordinators
• Small number of proposal representatives

35
Overview Evaluation Process
Panel with optional Hearings
Submission
Consensus
Individual reading
Commission Follow-up
Final ranking list
Evaluators
Evaluators
Evaluators
Criteria
Criteria
Criteria
Rejection list
Questions
Proposals in suggested priority order
Eligibility
36
Commission Follow-up

• Evaluation summary reports sent to proposers
• Final ranking lists
• Information to and discussion with the Programme
  Committee
• Commission decisions on rejected proposals
• Contract negotiation
• Commission decisions on proposals selected for
  funding

37
Further information

• Guidelines on proposal evaluation and selection
  procedures(http//www.cordis.lu/fp6/find-doc.htm
  evalproc)
• Calls for expert evaluators(http//www.cordis.lu
  /experts/fp6_candidature.htm)
• Guidance notes for evaluators(http.//www.cordis.
  lu)