The use of bibliometry for evaluation in Physics

1
The use of bibliometry for evaluation in Physics
D. Jérome Université Paris XI-CNRS, Past Editor
in Chief of Europhysics Letters Académies des
Sciences
2
The need for evaluation
Basic Research activity is supported to a large
extent by States Decide for support positions,
equipment, running money Evaluation (for
individuals, labs, domains, universities, research
organisms,..) of the past (recognition, ranking
of scientists) and of the future (positions,
grants,..) Evaluation committees, peer reviews
analysing scientific output Use of bibliometry,
What and how?.
3
Databases
Existence of web-databases with inverted
bibliographic data for citation analysis Google
Scholar scholar.google.com
free Scopus (Elsevier) scopus.com NASA
ADS adsdoc.harvard.edu
free ISI Thomson http//portal.isiknowledge.com
Web of Sciences JCR
ISI---gt Journals, Google scholar---gtBooks,
conferences (Belew 2005)
4
10 most cited articles 2000-02up to now
Squew distribution
5
Impact factor of journals
IF (year n) means that on average each of the
papers for years n-1 and n-2 are cited IF times
in the year n. Calculated every year in JCR.
2007 will give IF(2006)
Arbitrariness of the two years period for the
IF Are all journals listed? 6000 in Science and
Technology IF is an indicator for editors Misuse
of IF for the evaluation of individuals
6
Physics papers at Science/Nature
Time span 1975-2007
IF30.92
IF 29.27
First 100 papers Mostly Bio and Medicine, a small
minority of physics papers
7
Distribution of citations
Nature1852 Science1713
gt2/3 of the papers are below average citations
8
Distribution of citations at Phys.Rev.Lett
Papers 2003-2004
6972 papers
2/3 are below average
9
Distribution of citations at Europhysics.Lett
1054
lt 2/3 below average
10
Need for other indicators
IF informs about the popularity of a journal.
Ban IF factor for evaluation. What is needed is
an indicator related to the importance, quality,
influence, impact of the scientific work, not to
the popularity of a journal Physicists publish
in a small number of journals nearly all listed
in WoS (test done with four major labs at
University Paris XI) May not be true for other
domains (language problem) Use of h and g
indicators
11
Definitions for h and g
J.Hirsch, 2005
L.Egghe, 2006
g
h
12
Citations profile for the anonymous researcher !
h 41 g 83 G/H2.02
13
Pros
h -- is smoothing big hits and review articles
-- Problem when a small number of papers even
if they are brillant -- Good
correlations between h and total production
-- Extension to journals or domains is possible
-- Time dependence for comparison of different
seniorities is possible
g -- Allows cumulative advantages on bright
papers even with a small number of
papers -- Reveals precursor papers,sleeping
beauties
14
and cons
Caution Technical problems Homonymy for common
names, hyphened names Self citations ( no
dramatic changes) Team work (the normalized h
factor) Negative citations to incorrect
works Cronyism, network of friends, circle of
citations Must be handled by scientists
15
Examples (ISI database 1950-2007)
16
(No Transcript)
17
Conclusions

• Database are rather exhaustive in Physics
• Use bibliometry with much caution and expertise
  (by scientists)
• Results must be validated and agreed by persons
  being evaluated
• Ban on the use of IF for assessment of individual
  performance
• Use several indicators related to citations, h,
  g, g/h

18
Solving the problem of evaluation should solve
the crisis of publications