Anne Asserson

1
How can you assess your organisation through a
CRIS?The Norwegian national CRIS, FRIDA, and
evaluation by means of research output
2

• Structure of Presentation
• Ministry Model for Assessing Research Output
• Research Output Publications
• Requirements of a CRIS in this context
• System Solution
• Evaluation of Output
• Ensuring Quality
• Conclusion

3

• Aim of Ministry for Assessment
• Model for assessment based on indicators to
  Stimulate research and Allocate resources
• The best researchers (by allocating money based
  on publications)
• The best ideas (allocating money to the
  competition-based financing of projects)
• Allocate money to special thematic areas

4
St.prp.nr.1, UFD, pg. 115 Financing of
Research 2003

• New efforts
• to use result indicators for academic/scientific
  publishing as a basis to budget the research at
  Universities and State Colleges. The Publications
  from 2004 will be included in the budgets for
  2006
• Main issues
• Provide better data on Academic Publishing
• Take in consideration different practices thus
  avoiding unfortunate effect on the publication
  pattern.
• Divide in Quality levels to avoid reduction in
  quality of the publications pattern and avoid
  volume of low publications
• Develop indicators for activities within
  dissemination

5

• Ministry of Education and Research wish to
  improve research results in the universities
• Measure through indicators that say something of
  quality of results
• Results will trigger resource allocation to the
  universities
• 14 of total budget allocated based on research
  output indicators

6

• Result Indicators that have been chosen
• Doctoral theses
• EU funding
• National Research Council Funding
• Publications
• These are quality indicators all subject to peer
  review

7

• Institution receives an allocation from the
  ministry
• The model is NOT recommended for use internally
• The ministry stresses the need for creating
  internal models to reward / stimulate
• Necessary to think complementarily in order to
  support particular subject areas

8

• Structure of Presentation
• Ministry Model for Assessing Research Output
• Research Output Publications
• Requirements of a CRIS in this context
• System Solution
• Evaluation of Output
• Ensuring Quality
• Conclusion

9
(No Transcript)
10

• Definition of an academic Publication
• The Publication must
• Present new insight/knowledge
• Be in a form that make it possible to validate or
  useful in new research
• Be in a language and have a distribution that can
  make the publication available for the
  researchers of whom it may be of interest
• Be in a publication channel ( journal, series,
  publisher, website) with routines for Peer review.

11

• Publications
• Peer reviewed articles
• Academic monographs
• Chapters / parts of anthologies
• (Proceedings)
• Only a few classes of publications used in
  assessment

12

• Structure of Presentation
• Ministry Model for Assessing Research Output
• Research Output Publications
• Requirements of a CRIS in this context
• System Solution
• Evaluation of Output
• Ensuring Quality
• Conclusion

13

• Functionality Required
• Must be able to express
• Person belonging to one or more organisational
  unit
• The output produced at one or more organisational
  units
• from one year or a temporal interval
• From one or several projects
• And these in any combination

14

• Functionality Required 2
• Person must be linked to publication,
  organisational unit and project
• Reference format
• Vancouver, APA etc
• Person must be linkable to two or more
  organisational units

15

• Structure of Presentation
• Ministry Model for Assessing Research Output
• Research Output Publications
• Requirements of a CRIS in this context
• System Solution
• Evaluation of Output
• Ensuring Quality
• Conclusion

16

• Structure Required
• relational model as basis with dynamic role-based
  temporal links between base entities
• Extended with elements of publication
• Journal name, ISSN, shortname, acronym, pagefrom,
  pageto, volume, issue

17
The Whole Thing
Funding Programme
Classification
18
Secondary Base Entities example
RESULT_PUBLICATION
Can Express
(where DT-date/time) Person A (DT1 -
DT2) (is author of) Publication X Orgunit O (DT1
- DT2) (is owner of IPR in) Publication X Person
A (DT1 - DT2) (is employee of ) Orgunit O Person
A (DT1 - DT2) (is project leader of) Project
P Person A (DT1-DT2) (is member of) Orgunit
M Person A (DT1-DT2) (is member of) Orgunit
N Orgunit M (DT1-DT2) (is part of) Orgunit
O Orgunit N (DT1-DT2) (is part of) Orgunit O
19

• National system for the four universities
• Funded and owned by the universities
• Central database
• Based on CERIF (with extensions)
• 300,000 records in 2006
• In production use for 1 year

20

• Structure of Presentation
• Ministry Model for Assessing Research Output
• Research Output Publications
• Requirements of a CRIS in this context
• System Solution
• Evaluation of Output
• Ensuring Quality
• Conclusion

21
A publications is divided in author and
institutional parts within co-authoring
22
We find Academic Publications in the intersection
between Publications Channel and Publication
Form.
Channel
Eexample scientific article in Nature or
scientific monografi at Aschehoug forlag
Form
23

• Level 1 and Level 2
• At level 2 there are specified certain
  publication channels within all subjects.Level 2
  are expressing the desirable direction for the
  publication pattern.
• Level 1 is the normal and includes all
  publishing according to the definition.

24
Level 2 ca. 1/5 of the publications
Level 1 ca. 4/5 of the publications
25

• Guidelines for nomination at level 2
• In general the publication channel must be
• Considered as leading within broader scientific
  community
• Publish the most important publications for
  researchers from different countries

26

• Structure of Presentation
• Ministry Model for Assessing Research Output
• Research Output Publications
• Requirements of a CRIS in this context
• System Solution
• Evaluation of Output
• Ensuring Quality
• Conclusion

27

• Quality
• Import of ISI records
• Problem person and organisational unit not
  connected
• Routines and responsibility for quality assurance
• Researchers
• Organisational unit
• Researcher as end-user validates records

28
Bibliographical sources

• ISI-indexes will cover 62 of all journal
  articles.
• Norart can cover further 32 of the articles by
  adjusting the application and by Nordic
  cooperation.
• Bibsys could possible cover 70-80 of book
  publishing

29
(No Transcript)
30

• Structure of Presentation
• Ministry Model for Assessing Research Output
• Research Output Publications
• Requirements of a CRIS in this context
• System Solution
• Evaluation of Output
• Ensuring Quality
• Conclusion

31

• Quality
• Using selected publication channels, not only
  publication volume, gives quality
• Requirements of System
• Cannot be done by a library system
• Requires a flexible structured CRIS