Bild 1

1
(No Transcript)
2
Phoenix Fostering the rebirth of social
sciences and humanities in Central Asia

• 2nd PHOENIX Workshop
• Road to Excellence Research Evaluation
• in the Social Sciences and the Humanities
• Issyk-Kul (Kyrgystan), 20-22 May 2007
• Supranational Peer Review
• in the Social Sciences and the Humanities
• some European Experiences
• Dr Rüdiger Klein
• European Science Foundation
• Dep Head Humanities
• SSO Research and Foresight

3
2nd PHOENIX Workshop Research Evaluation

• Supranational Peer Review in the Social Sciences
  and the Humanities some European Experiences
• Presentation of institutional background (ESF)
• Discussion of lead questions on peer review
• - notion of peer review
• - examples of peer review
• - characterstics of peer review
• - steps towards international peer review
• - challenges of supranational peer review
• European Reference Index for the Humanities
  (ERIH)

4
ESF Membership

• 75 Member Organisations in 30 countries, also
  beyond the European Union
• Research funding organisations (e.g. research
  councils)
• Research performing organisations (e.g. national
  research institutes)
• Academies
• Combat fragmentation create critical mass
  advance science
• Partnerships with agencies outside Europe

5
ESF History and Role

• Established in Strasbourg in 1974
• An independent, non-governmental organisation
• Offices in Strasbourg and Brussels
• Budget (2006) 41 Mio.
• Science budgets networked 1-3 Mrd.
• Workshops and small research programmes (15.000
  ca. 500.000 )
• 30 large EUROCORES programmes
• (ca. 6-12 Mio. )
• EURYI young researchers award (1.25Mio.)
• Research foresight (incl. research
  infrastructures)
• ESF Member Organisation Fora

6
ESF Mission

• ESF provides a common platform for its Member
  Organisations in order to
• advance European research
• explore new directions for research at
  supranational level
• Cooperation between ESF Member Organisations,
  e.g.
• - coordinated research programmes
• - joint peer review processes and benchmarking
• - research infrastructures
• ESF serves the needs of the European research
  community in a global context through
  collaboration with
• Non-European, national research funding agencies
  (NSF, JSPS, CASS ministries academies RAN)
• UN (UNESCO etc.) ICSU UAI OECD NATO etc.

7
European COoperation in the field of Scientific
and Technical research

• Origin
• Started 1971 Ministerial Conference, 19
  Member States
• Current Participation
• 2006 34 COST Member States 1 cooperating
  state (Israel)
• COST Actions
• Concerted Actions (Networks) of nationally funded
  RD projects
• all fields of research

8
ESF Scientific Standing Committees Expert Boards

• Standing Committees
• Humanities
• Life, Earth Environmental Sciences
• Medical Sciences
• Physical Engineering Sciences
• Social Sciences
• Expert Boards
• Committee on Radio AstronomyFrequencies
• European Polar Board
• European Space Science Committee
• Marine Board
• Nuclear Physics European Collaboration Committee

9
Social Sciences at ESF

• Fields
• Business Administrative Sciences
• Communication Sciences
• Demography
• Economics
• Environmental Sciences
• Geography
• International Relations

• Law
• Pedagogy Edu-cational research
• Political Sciences
• Psychology Cognitive Science
• Social Anthropology
• Social Statistics Informatics
• Sociology
• Womens Studies

10
Humanities at ESF

• Disciplines
• Anthropology
• Archaeology
• Art art history
• History
• History philosophy of science
• Languages philologies

• Linguistics
• Literary studies
• Musicology
• Pedagogy Edu-cational research
• Philosophy
• Psychology
• Religious studies Theology

11
Humanities at ESF

• Fields such as
• Area studies
• Classical studies
• Cognitive science
• Communication media studies
• Culture, develop-ment, environment
• Gender studies
• Heritage studies
• Urban studies etc.

• Research questions such as
• Consciousness
• Evolution of cooperation
• Global change
• Health welfare
• Human dignity
• Landscape research
• Migration
• Security
• Technology, culture and society etc.

• Research Infrastructures

12
Peer Review lead questions

• Discussion of lead questions on peer review
• - notion of peer review
• - examples of peer review
• - characterstics of peer review
• - steps towards international peer review
• - challenges of supranational peer review

13
Peer Review lead questions

• Examples of Peer review
• Assess track-record and potential
• Application for a research grant (individual
  institutional)
• Application for a position (individual)
• Assess product
• Submission of a manuscript (article, book etc.)
  assess quality (methods, results, presentation
  etc.)
• Evaluation of research programmes, research
  institutes (universities, academies), research
  funding agencies assess performance over time
  benchmarking
• Sub-category
• Examination of a degree candidate

14
Peer Review lead questions

• Peer review as a social process
• Formal tradition from 17th century (at least)
• market in scientific goods (knowledge) control
  over access to resources for research
• competitive struggle for scientific authority
  (epistemological conflicts)
• Risk denial and elite privilege
• Yet notion of expertise often conveys idea of
  objective knowledge (and possibility of progress)
• Distortions rarely accounted for or made
  explicit economic bias (mafia) professional,
  academic bias (research interests) personal bias
  

15
Peer Review lead questions

• Peer review as a social process (cont.d)
• Peers agree / disagree which is best?
• Scientific merit BUT
• Conventional research (interdisciplinarity)
• Institutional conservatism (status of researcher
  or department)
• Ethical Issues
• Personal bias / research profile
• Sexism
• Old Boys Networks
• Institutional traditions of peer review

16
Peer Review lead questions

• Suggestions for criteria for successful peer
  review
• Success likely selection of best proposals
  according to scientific merit
• Openness transparency of the process
  (procedures, criteria, sometimes names known to
  applicants and public)
• Multi-level selection (external mail review
  panel)
• Clear conflict of interest guidelines
  (published) training of panel members written
  commitment of external reviewers
• Knowledgeable secretariat (suggestion of peer
  group grasp of science procedures IT)
• Use of support tools where and as appropriate
  (e.g. scientometrics recognition of
  outcome/outreach)

17
Peer Review lead questions

• Suggestions for criteria for successful peer
  review
• Success likely selection of best proposals
  according to scientific merit
• IMPORTANT
• recognise fallibility (struggle for capital)
• maintain flexibility (interdisciplinarity
  internationality)
• monitor group dynamics
• improve accountability
• For international peer review
• - acknowledge cross-cultural differences
  (description of science fields
  interdisciplinarity)

18
Peer Review lead questions

• Forms of international peer review
• international composition of juries for selection
  in national competitions
• international composition of research groups
• competitions of international grant-making
  agencies
• international coordination of competitive funding
  from different national grant-making agencies to
  support international research groups
• juste retour
• - agencies pays only successful researchers from
  own country
• - funding partly according to ability of agency
• common pot
• - all agencies contribute to joint funds
• - funding exclusively according to excellence of
  researchers

19
Peer Review lead questions

• Steps towards international peer review
• Specific challenges in the Humanities
• International research perceived as new
  phenomenon (but academies learned societies)
• Role of national research traditions (incl.
  language)
• Contribution of Humanities research to national
  identity debates (e.g. history, literature,
  anthropology)
• Support tools lacking (indicators)
• Overall Languages considered not as a barrier,
  but language diversity considered a cultural and
  intellectual asset

20
European Reference Index for the Humanities (ERIH)

• Starting point
• - assessment unsuitability of existing
  databases (e.g. ISI) for journal output of
  European and other non-Anglo-Saxon Humanities
  research
• - access poor visibility of much of
  journal-based European scholarship in Humanities
• N.B. Internationalisation of research (careers
  collaborations ERC etc.) requires comparability
  of quality across borders and languages
• Objective 1 identify and categorise good quality
  research journals in the Humanities
• - New mapping of Humanities research in all
  languages located in an international context
• - Encourage best practice (peer review)
  dialogue with publishers and editors
• Objective 2 improve access to and visibility of
  Humanities research

21
European Reference Index for the Humanities (ERIH)

• Process
• involve research community (different levels of
  peer review)
• - 30 national funding agencies (Europe) collect
  input through consultation of research community
• 14.000 suggestions
• - 15 international expert panels unpaid
  analyse input and categorise journals (18
  months)
• - international subject associations and
  specialist libraries widely consulted 6.000
  comments
• - international journals from the entire world
  included AB language-specific journals so
  far only from Europe C
• - open feedback form and continuous updates
  (April 2007 onward) structured interaction with
  publishers and editors (and other stakeholders)
  through workshops etc.

22
European Reference Index for the Humanities
(ERIH) disciplinary coverage 2006
Initial lists are highlighted in yellow draft lists are not highlighted Initial lists are highlighted in yellow draft lists are not highlighted        
Discipline Initial national submissions 1st draft lists Consolidated comments Consultation1a Cnsolidated comments Consultation 1b Initial lists/ draft lists
    Mar-Apr 2006 Mar-Apr 2006 Jul-2006 Oct-2006
Anthropology (social and evolutionary) 752 154 133 17 242
Archaeology 1310 524 290 25 420-440
Art and Art History 938 445 292 16 550-600
Classical Studies 619 256 321 11 260-280
Gender Studies 155 181 37 6 119
History 1419 874 508 71 900-1000
History and Philosophy of Science 806 145 64 4 166
Linguistics 1093 680 391 34 586
Literature 1453 1481 706 10 900-1000
Music and Musicology 204 n/a 187 4 166
Oriental and African Studies 196 588 386 14 600-650
Pedagogical and Educational Research 666 404 271 92 450-470
Philosophy 658 320 153 22 305
Psychology 1198 1201 159 4 600-650
Religious Studies and Theology 745 n/a 580 10 371
23
European Reference Index for the Humanities (ERIH)

• Next steps
• - include new areas applied ethics
  communication media archives, museum, library
  studies
• - consider subdivision of large fields
• - testing of robustness of the lists requested
• - ERIH National contact points improve national
  feedback
• - consider models to include monographs,
  conference proceedings etc.
• - widen geographical consultation through
  partnerships with non-European agencies and
  initiatives
• (East Asia Russia/NIS/Caucasus Latin America
  Africa)
• - widen discussion on use to include other
  perspectives Research Infrastructure Virtual
  Learning Environment
• (Permanent Access to the records of science)

24
2nd PHOENIX Workshop Research Evaluation

• ...thank you
• for your attention
• For further information ,
• please visit www.esf.org,
• or write to rklein_at_esf.org

25

• www.esf.org