CAR PURCHASE COMPARI

1
CAR PURCHASE COMPARI

Trust and authority in scholarly
communications David Nicholas and Anthony
Watkinson, CIBER

• text
• text

2
CAR PURCHASE COMPARI

Trust and authority under massive attack

• Trust, authority and reliability are the very
  watchwords of scholarly communication. System
  built upon quality assurance
• In todays crowded, dynamic, diverse and
  dis-intermediated digital scholarly environment
  it is ever more difficult to establish and
  authority of information
• More sources, channels, platforms, environments,
  players and consumers all muddying the water.
• Difficult to even know whose information it is
  anymore
• More metrics and proxies more gaming all brought
  on by increasing competition among academics
• Certainly some VERY challenging behaviour coming
  along from the Google Generation picture on
  right and smartphone users

3
CAR PURCHASE COMPARI

Background the project

• CIBER and UT with funding from Sloan Foundation
  researching how emerging digital
  behaviours/platforms are (possibly) challenging
  and changing concepts of trust and authority in
  scholarly world. In particular to discover
• How academic researchers assign and calibrate
  authority and trustworthiness to the scholarly
  sources and channels they choose to use, cite and
  disseminate
• Whether social media and open access are having
  an impact on conventional practices of
  establishing the authority and trustworthiness
• Scope 09/2012 11/2013 global with focus on UK
  USA science, social science
• Working with publishers to identify, recruit and
  access researchers. TF , Elsevier, Sage, BMC,
  Wiley, PloS and Thompson Reuters.
• Methods a) 14 focus groups to scope boundaries
  and identify important issues and questions
  (DONE) b) 80 critical incident interviews to
  supply the personal detail (DONE) c)
  international questionnaire to obtain
  quantitative data (PLANNING STAGE)

4
CAR PURCHASE COMPARI

General characteristics of a reliable and
trustworthy source/channel
Few researchers really dealt head-on with what
was meant by reliability, quality,
trustworthiness discussions usually turned into
a debate about relative merits of personal vs.
proxy methods of establishing reliability and
quality Trustworthiness was said be down to
reputation of the author or journal in their
research specialism Quality could be best
determined by personal inspection and judgment,
but because of a shortage of time, too much
material to get through and the strictures of
their managers and institutions (in respect to
Impact Factors, for instance) this was not always
feasible. While everyone thought it was an
important issue nobody really thought it was a
big or pressing issue. Was it just a question of
what is useful?

5
CAR PURCHASE COMPARI

General personas
Researchers function as authors, editors,
reviewers, citers and users of the literature.
Thus can bring different trust judgments to bear
for each function Editors seemed to take a
different stance their thinking was dominated by
their own picture of themselves as information
providers, people offering trustworthy
collections for others to trust. They appeared
much more traditional Citation behaviour tends to
be much stricter, more focussed and complex than
use Researchers have considerably more freedom
as to what they use - they can use blogs, OA
etc. to their hearts content Most researchers
undertake a variety of scholarly roles. However,
when they talk about trust and authority in
scholarly communication they rarely preface what
they say by stating that, for instance, as user
or citer I do this. It might be so because the
journal is so central to their views on quality

6
CAR PURCHASE COMPARI

General peer review
Provides a degree of certainty about the quality
of product. Shows that someone has put in an
effort and that it had been validated by a
community of scholars. Therefore, in theory, it
is an important scholarly attribute which enables
researchers to search, use, cite and disseminate
with confidence On the one hand researchers want
to be published in journals that have robust peer
review mechanisms (despite the heartache
involved) and on the other they want to feel
secure in citing peer reviewed material. While
there is a strong attachment to peer review most
people preface their expression of trust with
recognition that there are problems with the way
it is undertaken. However, when you drill down to
the mechanisms of peer review there is no
consensus at all on how it might be
improved. Biggest impact REF has had is in
creating an institutional peer review in
universities throughout the UK a system by which
researchers are constantly monitored.

7
CAR PURCHASE COMPARI

General metrics
Evidence that the scholarly world being driven
and governed by algorithms and the consequence
was that creativity and new ideas were being
driven out by a (high) metric-driven culture,
which was standardising scholarly communication
behaviour Some differences in attitude according
to discipline of researcher. Scientists largely
unquestioning about merits of metric system
social scientists slightly uneasy but felt there
was no choice the few humanities scholars
clearly felt culturally uncomfortable and
alienated, but uncomfortable or not were part of
it Most extreme case of force-fed metrics was
found in Business/Economics, where researchers
told where to publish and what to cite by the
academic journal quality guide produced by the
Association of Business Schools Early career
researchers in social sciences/humanities thought
themselves as slaves to a metric-based/journal
focussed system they have to adhere to rules to
climb academic ladder but thought the ladder was
broken. Journals a manifestation of all that was
wrong with scholarly communication system. Yes,
its all crazy, but I am not here to reshape
academe.

8
CAR PURCHASE COMPARI

General social media
Only a few mostly young, thought social media
nothing but social and transient no question it
should be trusted, and certainly not a substitute
for the peer reviewed journal While expressing
lack of interest in social media themselves many
knew people that used it and felt guilty about
lack of involvement Lack of interest explained by
trust and validity problem but also other
reasons a) many were novices b) they were
antagonistic towards it c) had no free time d)
were put off going down that route by the current
HE climate e) the (informal) language of social
media not suitable for scholarly
discourse. Recognition that social media most
valuable for a) getting new ideas and
stimulation b) self-promotion of publications,
especially in regard to outreach. What most
interested participants (when told) was the fact
that social media could increase their
citations. Early career researchers made
extensive use of social media but scared to
embrace it fully and camouflaged its use.
Elsevier research suggests they are getting
bolder. Many academic benefits for them a)
develop a personal network b) facilitates
collaboration c) finding fellow researchers to
work with (in real-time) d) staying in touch
with what is going on e) following authors
(stalking!) f) easier to find someone with a
particular point of view. Fast-track development.

9
CAR PURCHASE COMPARI

General open access
Distrust of open access from an author and reader
perspective evident. Distrust diminishes
considerably (but not quite evaporates) when OA
journals are published by an established
publisher Few researchers aware of the pioneering
efforts of PloS1, although they did like, in
principle, what it has achieved (quick and
popular). Researchers from teaching intensive
universities more positive on openness grounds.
Felt also that OA journals might help your
career Universities might rank IF journals more
highly but Google ranks OA higher in hits list,
so if you want to be listed first to impress head
hunters then OA provides a good career route.
Early career researchers liked principle of open
access , but scared to embrace it because felt
academe has not made up its mind about it. If
published in an OA journal, or cited one, might
have backed the wrong horse and found, for
instance, that they had published in a
second-rate journal. There might be a reputation
threat.

10
CAR PURCHASE COMPARI

General better/worse than a decade ago?
Researchers acknowledged that there was more bad
stuff around, because it is just more accessible
and there are more opportunities to publish, but
accepted , overall, quality had risen over the
years. The rise in quality meant they could live
with the bad and boring stuff Rise in quality
result of more people entering the field and the
greater competition that comes with it. It is the
niche/specialist journals that have taken full
benefit of an abundance of run-of-the-mill
material So not OA journals that are being blamed
for rise in poor or mediocre work, but the big
increase in subscription journals There is a
massive sea of mediocrity now because it is just
easier to publish, but at the higher end the
quality is better because of better training,
greater competition and rewards for publishing

11
CAR PURCHASE COMPARI

Usage
Low key. Played down usage and trust, because, if
something was out there, their long-established
networks would tell them about it they did not
need to go looking. Intuitive where you turned
for information. But at the heart of it all was
networks of people that you had developed over
the years. Establishing worth. If I dont
recognise the author, then be careful if
additionally you dont recognise the
institutional affiliation be even more careful,
and if you dont recognise the journal as well,
it is definitely not even worth looking at.
Impact Factor. Not that important determining
what to read a) too narrow a view of research
literature b) attached to a journal, not an
article c) lottery as to which journals get IFs
d) an IF meant journal bombarded by low grade
authors trying to enhance careers e) high IF
journals tend to be stylized, lack innovative and
fresh papers f) not aware of what the IF is for
a journal Bigger fields the more important IFs -
do not know all authors. IFs useful in peripheral
fields. High IF journals get best referees so
papers of merit Peer review. If knew the journal
or the editorial group researchers were fairly
confident but cannot always guarantee quality.
Some dubious peer reviewed stuff out there.

12
CAR PURCHASE COMPARI
a.
Usage
Usage factor. Researchers not aware of
possibilities on offer. Negatives 1) too easily
gamed 2) highly used articles not in the best
ones (by editorial opinion) 3) downloads do not
represent readings 4) media exposure raises an
articles profile greatly. Altmetrics? Interesting/
original sometimes trumps authority/ranking.
Occasions when authority/ ranking are of
secondary concern in determining usage, when
researchers looking for new, fresh and creative
content. Mainly social scientists. In these cases
a quick and dirty search in Google/Google Scholar
productive, especially in delivering
inter-disciplinary material. Good writing a
determinant. Reading/role of abstracts in trust
/authority judgments. Too much published, too
little time to read it and widespread adoption of
skittering leads to a dependence on proxies
(abstracts). Screening and cross-comparisons made
at the abstract level. Few researchers read
full-text when writing/reviewing an article. In
fact reading an article means looking 5-10 of
it. Wanted them quality-controlled, properly
reviewed. We need to be able to trust the
abstract. Abstracts offered free to view adds to
their value in this role. Other bibliographical
trust points. Methodology (more so in science
social scientists thought to be too mechanistic a
measure), conclusions, bibliography and
theoretical stance.

13
CAR PURCHASE COMPARI

Usage types of sources
Digital sources. Trust enhanced if something also
available in print. Merits in paper and some
associated with quality/reliability. Thus a)
paper was thought to provide more confidence
because less ephemeral b) because there were
clear length limitations this focussed the mind
and produced better copy. Early career
researchers thought the opposite could not
believe anyone would think like that and thought
digital was the real. Data. Some trust concerns
because of a presumed absence of peer review
needed because could not decide for yourself
given size and complexity of data. Idea of data
being attached to an article, the article giving
it its authority, thought to be a good idea. Also
felt that name of the author was even more
important in case of data. Conferences. Not REF
material. Also a) peer reviewing mechanisms
poor, absent or done on the back of an abstract
b) increasing number of sharks out there that
sully the name of conferences c) despite the
best indexing efforts of Web of Knowledge et al
proceedings do not have the same visibility and
standing. But they can attract large number of
citations.

14
CAR PURCHASE COMPARI

Usage searching
Search platforms. Trusted big fat information
pipe researchers connected to not the publisher
platform or library big deal it is in the
Internet. Hence popularity of Google. Google
Scholar thought to be surprisingly good.
Librarians/JISC argue discovery terrain becoming
increasingly difficult, with OA, but not
according to researchers. Did not mention their
publishers website nor for that matter
university catalogue or federated search engine
(JISC project!). Libraries mentioned in a
negative and nostalgic fashion. Role of
librarians bound up with the buildings in which
they work. Libraries, once guardians of quality
have no role at all to play today. Just did not
see libraries as the point of entry to the
information they are looking. Libraries seen as
incomplete sources of information and researchers
do not trust librarians to make the critical
decisions on what is and what is not in the
walled garden on their behalf. Trusted searches
strategies 1) follow-up citations from trusted
sources and then you check the abstracts to
establish worth and quality 2) enter the name
of a classic work in the field in Google Scholar
and then look at the citations to it. Any barrier
will put you off chasing something. Early career
researchers did follow people and publishers, but
the difference did not do this via a publisher
database, but via social media

15
CAR PURCHASE COMPARI


Usage open access and social media
Open access. All the researchers we spoke to were
beneficiaries of the big deal, which gave them
easy access to journals. So OA no big deal.
Nobody actually said they would rather not use OA
material. However, even if they wished to, what
is not clear as to how they could discriminate in
the first place Concerns of poor (or absent) peer
reviewing of OA articles did not result in
widespread checking of OA journals peer review
polices. But the interesting thing was that some
actually did say that they did check policies
which we bet they do not do for subscription
journals Social media. Scientific tweets
almost always point to a journal article they
are just to link. They all use Wikipedia, love it
and happy to say so. Academe.edu mentioned
positively for accessing difficult to access
material. Young career workers talk about social
media enabling them to have a conversation.

16
CAR PURCHASE COMPARI

Citing

• Not as liberal as use. Authors weigh up sources
  carefully (reject social media). There are
  political issues need to cover their backs
  people you have to cite to get accepted only
  have real choice over a few of your citations. A
  lot of window dressing
• Citation practices/gaming
• 1) cite your own work to raise your H index
• 2) cite papers in journal to which you submit
• 3) reviewers ask you to cite their own papers
• 4) cite very high impact articles because they
  set the agenda/benchmark and represent the very
  pinnacles of science (halo effect)
• 5) cite post hoc use citations to support your
  position, give your ideas more weight this is
  particularly important if your ideas are novel
• 6) cite the first source on the topic and the
  most recent one (bookending)
• 7) cite review articles as a bibliographical
  shorthand. We know that people are gaming because
  they say they do not do it but know people that
  do

17
CAR PURCHASE COMPARI

Citing

• Early career researchers. Pressure on them from
  supervisors to cite peer-reviewed articles. You
  see interesting things elsewhere (i.e. social
  media) but you cannot use/cited them. Cited
  social media sources as personal communications
• Academics from teaching intensive universities
  different could not cite something they have not
  read and would cite anything, including trade
  publications.
• Social media. twitter not used as an information
  source it would be like citing a conversation in
  the bar. Blogs not to be sourced many blogs
  were just streams of consciousness stuff.

18
CAR PURCHASE COMPARI
a.
Publishing/Dissemination

• Pressures to publish and the impact on quality.
  Biggest and most important concern was not so
  much growth in the literature, but what was
  responsible for driving this growth, the
  increased pressure to publish. Trend is creating
  an avalanche of bad/mediocre journal articles.
  It is publish or perish multiplied. And there
  is a huge amount of junk floating around the
  scholarly system. This manifests itself in
  massive journal rejection rates 50 are
  rubbish, coming from everywhere, but mostly
  developing countries
• The massive influence of REF (in UK)
• REF is all-pervading and persuasive. REF says
  citation scores not everything but no-one
  believes this and carry on attempting to publish
  in high impact journals. Who is going to question
  an article in Nature? Leads to a negative impact
  on creativity and a distortion in where articles
  really should be placed. Penalising inspirational
  and creative academics
• REF is guilty of forcing people to publish more
  than they otherwise would, something which leads
  to higher levels of poor content being published.
  More and more universities are prescribing
  metrics as a proxy for quality, meaning lower
  ranked universities joining the game, when they
  really should not.

19
CAR PURCHASE COMPARI

Publishing peer review
Few thought that the review process was broken.
Early career researchers least fond Publishers.
Organising peer review central role of
publisher. Rejection rates. Badge of
honour/quality. The practice. a) liked blind
reviewing - reviewers freer to comment b) mixed
opinions about benefits of author-suggested
referees (bad suggest their friends good avoid
referees who suspect of foul-play) c) light
touch peer review not liked heavy leads to
better papers d) open refereeing not popular as
inhibits reviewer e) not sure about post
publication peer review e) editors should be
ultimate judges be proactive and not always heed
reviewers f) editors should function as a
release valve for peer-review process, when fails
to allow for difference, freshness and
innovation g) referees improve an article even
if they rejected it. It is worth submitting to
Nature, even if you had no chance being accepted,
just to get quality feedback. Dislikes too slow.
Need to obtain decision within two months
weakness predatory OA publishers take full
advantage of (in their advertising, if not in
reality). Concerns Quality of reviewing thought
to be variable and explained by the pressures on
reviewers to get the job done quickly as a
consequence, quality is being sacrificed.      

20
CAR PURCHASE COMPARI

Publishing Plagiarism and unethical practices
Plagiarism. With the article avalanche comes
dodgy content. Thought to be more rife and
widespread than generally thought. While early
career researchers agreed that plagiarism was a
no-no, they were less antagonistic towards
cut-and-paste behaviour, providing attributions
were given. While academics from teaching
intensive universities were not supportive either
they were not that hard on it. Thought it was a
fuzzy area and understood the reasons/pressures
for doing it. Self-plagiarism they seemed to
think was a less serious offence maybe not an
offence at all.   Publisher data shows plagiarism
and duplication detected down from 30 to 10 in
last 3 years. CrossCheck Making it
up/fabrication. Everyone knew cases senior
academics included. Big pressure to do so in some
parts of the world. It was alleged it could be as
much as 10-20 in biological sciences. Difficult
to establish.    

21
CAR PURCHASE COMPARI

Publishing Open access

• Few researchers admitted to OA publishing deep
  seated dislike of OA for many. Being imposed
  (political reasons), rather than something
  actually needed. Its open anyway. Few realised
  that traditional publishers produce OA articles
  and journals
• Worries
• 1) OA material poor quality. Crap was one term
  used to describe it other terms used were
  vanity publishing, self-deluded authors. Why
  would you want to (pay to) publish in something
  in a start-up, which is easy to get into and has
  no reputation or pedigree. The journal would have
  to have a brand
• 2) Business model liable to undermine rigorous
  review concerns about possible two track peer
  review process, with OA articles being treated
  more leniently because of payment
• 3) Uneasy about the author pay model that
  underpins gold OA publishing. Passing of money
  seems to sully the transaction. Some concern that
  you could pay your way into publishing
• 4) Poorly run and unprofessional. Fails to
  recognise that there is a good deal of
  professionalism and standards behind publishing
• 5) Concerns that academics might have to publish
  in OA journals and that subscription journals
  might end up featuring different content and
  authors.
•  

22
CAR PURCHASE COMPARI

Publishing Social Media
Nobody saw it being an alternative to journal or
book publishing. Raised the results of a
previous CIBER focus group on social media which
revealed that in one discipline at least younger
researchers saw social media giving them a way of
communicating ideas and information which they
could not publish in journals in the hands of a
cabal of old white males, fixated on impact
factor scores. While there was some sympathy for
this view, nobody came forth with similar
examples