Title: Evaluation of Process, Value, and Influence of Scholarly Book Reviews in LIS Journals
1Evaluation of Process, Value, and Influenceof
Scholarly Book Reviews in LIS Journals
- Jeppe Nicolaisen
- jni_at_db.dk
- Department of Information Studies
- Royal School of Library and Information Science
2Agenda
-
- Motivations and objectives of the research
project - Research questions
- Organizational framework
- The scholarliness of published peer reviews
- Scholarly communication and strategic research
materials - Results of a bibliometric study of book reviews
in selected social - science fields
3Motivations and objectives
- The secrecy involved in so much
- of the peer review process
- precludes a comprehensive,
- systematic study of the subject.
- Gordon Moran (1998)
4Motivations and objectives (2)
- Peer review is the process whereby authorities in
a given field determine the validity and assess
the relative significance of a particular
contribution of a scholar or scientist within
that field. - Editorial refereeing
- Evaluation of grant applications
- Awards
- Promotions
- Hiring
- Book reviews
- etc.
5Motivations and objectives (3)
- The peer review process is criticized for lack of
reliability and validity - The process is influenced by socio-psychological
mechanisms, which make the process unreliable. - Predicting the eventual importance of scientific
work would require one to predict an
unpredictable future nothing forces the
conclusion that it would be the least bit foolish
to make many of our choices by drawing lots.
6Motivations and objectives (4)
- A large-scale investigation of the peer review
process in LIS. - Reliability
- Validity
- Scholarly book reviews Published peer reviews.
- Published peer reviews ? The peer review process.
- Scholarly book reviews in LIS journals ? The peer
review process in LIS.
7Motivations and objectives (5)
- Bornsteins hypothesis
- If a relationship between citation frequency and
research quality does exist, this relationship is
not likely to be linear. The relationship between
research quality and citation frequency probably
takes the form of a J-shaped curve, with
exceedingly bad research cited more frequently
than mediocre research (e.g. as an example of an
idea or line of research that turned out to be a
blind alley, or as an example of what not to do
in a particular area). - Bornstein (1991)
8Citation- frequency
Peer review
9Research questions
- To what extent is the evaluative content of
scholarly book reviews influenced by
socio-psychological mechanisms? - To what extent is the evaluative content of
scholarly book reviews valid? - What is the relationship between peer review and
citations? - How, and to what extent does scholarly criticism
contribute to shape the relationship between peer
review and citations?
10Organizational framework
- 6 research papers
- The J-shaped distribution of citedness, Journal
of Documentation, Vol. 58 No. 4, in press. - Structure-based interpretation of scholarly book
reviews a new research technique, Proceedings
of the Fourth International Conference on
Conceptions of Library and Information Science,
in press. - The scholarliness of published peer reviews a
bibliometric study of book reviews in selected
social science fields.
11Organizational framework (2)
- The reliability of book reviews in LIS journals
- The validity of book reviews in LIS journals
- The relationship between peer review and
citations - Final thesis
12The scholarliness of published peer reviews
- Book reviews serve a number of vital functions
- They are widely used to help estimate the
quality and importance - of books published by academicians and thus
to help make - decisions about hiring, promotions and salary
increases (Glenn, - 1978)
- Developing library and information center
collections - Allow members of academic communities to keep
up with the - latest professional progress despite of the
eternal growth and - dissemination of recorded knowledge
13The scholarliness of published peer reviews (2)
- Book reviews differ in regard to their
prototypicality - Some are short, while others are longer. Some are
more focused on providing a general view of how
the book is organized rather than an actual
evaluation. Some reviewers insert the book in the
field while others concentrate on making topic
generalizations or inform about the author or
potential readership. - What are the characteristics of a scholarly
book review?
14Scholarly communication
- Scholarly communication is a communicative
practice anchored in three dimensions (Kling
McKim, 1999) - Publicity
- Accessibility
- Trustworthiness
15Scholarly communication (2)
- A trustworthy book review
- should describe and characterize not only the
book in question, - but also the subject with which it is
dealing. - should answer the question is the book a real
addition to our - knowledge, and if so, what exactly has been
added?. - should position the book in the literature
devoted to the same - subject and evaluate the new contribution in
relation to related - works.
- George Sarton (1960)
16Strategic research materials
- are the empirical materials that exhibit the
phenomenon to be explained or interpreted to such
advantage and in such accessible form that it
enables the fruitful investigation of previously
stubborn problems and the discovery of new
problems for further inquiry - (Merton, 1987)
- Publication counts
- Citation counts
- Textual indicators
- Colonic titles
- References
17Strategic research materials(2)
- The practice of citing other works is second
nature to anyone - writing a scholarly or scientific paper
(Kaplan, 1965) - Explicit references are essential in order to
communicate - effectively and intelligently about
scientific and technical subjects - (Garfield, 1977)
- The practice of footnoting is as old as
scholarship itself - (Price, 1970)
18Results of a bibliometric study
- Table 1. Definition of six social science fields
based on ISIs Subject Category assignments
Modified from Glänzel (1996)
19Results of a bibliometric study (2)
- Table 2. SSCI Search for scholarly book reviews
in the field of Economics, 1997-2001
20Results of a bibliometric study (3)
- Table 3. Selected social science fields Number
of book reviews in six equal sized periods
1972-2001, number of scholarly book reviews,
percentage of scholarly book reviews
21Results of a bibliometric study (4)
- Figure 2. Selected social science fields
Percentage of scholarly book reviews in six equal
sized periods - 1972-1976, 1977-1981, 1982-1986, 1987-1991,
1992-1996, 1997-2001
22Results of a bibliometric study (5)
- Table 4. LIS core journals Number of book
reviews in six equal sized periods 1972-2001,
number of scholarly book reviews, percentage of
scholarly book reviews
(White McCain, 1998)
23Results of a bibliometric study (6)
- Figure 3. LIS journals and LIS core journals
Percentage of scholarly book reviews in six equal
sized periods 1972-2001
24Results of a bibliometric study (7)
- Price (1963) maintains that if any sufficiently
large segment of science is measured in any
reasonable way, the normal mode of growth is
exponential. He even stated that he had no
hesitation in suggesting it as the fundamental
law of any analysis of science. - If science in fact is growing exponentially, then
we are able to predict the future growth of
scholarly book reviews in the six fields under
study. - Mathematically, the law of exponential growth
follows from the simple condition that at any
time the rate of growth is proportional to the
size of the population or to the total magnitude
already achieved.
25Results of a bibliometric study (8)
- Figure 4. Selected social science fields The
expected exponential growth of scholarly book
reviews 2002-2086
26References
- Bornstein, R.F. (1991), The predictive validity
of peer review a neglected issue, Behavioral
and Brain Sciences, Vol 14 No. 1, pp. 138-139. - Garfield, E. (1977), To cite or not to cite a
note of annoyance, Current Contents, Vol. 35
(August 29.), pp. 5-8. - Glänzel, W. (1996), A bibliometric approach to
social sciences. National research performance in
6 selected social science areas, 1990-1992,
Scientometrics, Vol. 35 No. 3, pp.291-307. - Glenn, N. (1978), On the misuse of book
reviews, Contemporary Sociology, Vol. 7 No. 3,
pp.254-255.
27References (2)
- Kaplan, N. (1965), The norms of citation
behavior prolegomena to the footnote, American
Documentation, Vol. 16 No 3, pp.179-184. - Kling, R.K. McKim, G. (1999), Scholarly
communication and the continuum of electronic
publishing, Journal of the American Society for
information Science, Vol. 50 No. 10, pp. 890-906. - Merton, R.K. (1987), Three fragments from a
sociologists notebooks establishing the
phenomenon, specified ignorance, and strategic
research materials, Annual Review of Sociology,
Vol. 13, pp. 1-28. - Moran, G. (1998), Peer review and academic
paradigms, Journal of Information Ethics, Vol. 7
No. 2, pp. 19-29.
28References (3)
- Price, D.J.S. (1963), Little Science Big Science,
Colombia University Press, New York, NY. - Price, D.J.S. (1970), Citation measures of hard
science, soft science, technology, and
nonscience, in Nelson, C.E. Pollock, D.K.
(Eds) Communication Among Scientists and
Engineers, Heath, Lexington, MA, pp.3-22. - Sarton, G. (1960), Notes on the reviewing of
learned books, Science, Vol. 131 (April 22.),
pp. 1182-1187. - White, H.D. McCain, K.W. (1998), Visualizing a
discipline an author co-citation analysis of
Information Science, 1972-1995, Journal of the
American Society for Information Science, Vol. 49
No. 4, pp. 327-355.