Research Assessment and Funding: the experience of other countries

1
Research Assessment and Funding the experience
of other countries

• Dr Lisa Lucas
• Graduate School of Education
• University of Bristol

2
Change in RAE gradings of top grades (5 5)
from 1992 to 2001 by subject area
3
The Research Game in Academic Life (Lucas, 2006)

• Intensification of the management and
  organisation of research activities
• Differentiation of academics within departments
  over status/workloads
• Struggles over classification of research
  active/research inactive
• Lack of value perceived for teaching and
  associated work
• Prioritising of research areas that can attract
  high levels of research funding and would be
  worthy of submission to high ranking journals

4
UK University Core Funding for Research and
Teaching 2005-6Source Higher Education Funding
Council (HEFCE)
5
Focus of Presentation

• What can be learned from other national systems
  of funding and evaluating research in
  universities that utilise metrics?
• Can other national systems provide alternative
  ideas to inform the development of the UK system
  of funding and evaluating research?

6
University State Funding and the Higher Education
Landscape in the 21st Century

• Reduction in State funding to universities
• Resource Dependency Theory (Slaughter Leslie,
  1997)
• Global Competition
• Creating World Class Universities (global
  league tables)
• Globalisation and Higher Education
• Convergence of Higher Education Policies?

7
Measuring the Scientific Impact of Nations?
8
National Case Studies of Research Funding and
Evaluation

• Hong Kong
• The Netherlands
• Australia and New Zealand

9
Higher Education System in Hong Kong

• Eight institutions
• Funded by the University Grants Committee (UGC)
• As far as the UGC is concerned, the UGC
  advocates role differentiation among
  institutions, and the concentration of resources
  to reward performance and encourage the growth of
  centres of excellence. (UGC, Facts and Figures,
  2002)

10
Hong Kong RAE 2006

• Based originally on the UK RAE
• Single quality threshold at attainable level of
  excellence, no grading
• Carnegie Classifications
• scholarship of discovery
• Scholarship of integration
• Scholarship of application
• Scholarship of teaching

11
Hong Kong System key issues

• Single quality threshold no differentiation
  between research active and inactive staff.
• Allowing for multiple dimensions of excellence by
  broadening the definition of research to forms
  of scholarship.
• BUT
• Role differentiation amongst institutions
  mandated.
• Greater compulsion for EVERYONE to do research.
• Multiple dimensions of evaluation less easy in
  practice.
• Pressure to publish internationally.

12
Higher Education System in the Netherlands

• The universities prepare students for
  independent scientific work in an academic or
  professional setting and the hogescholen prepares
  students to practise a profession and enable them
  to function self-consciously in society at
  large.
• 13 universities and 50 hogescholen
• Source Boezerooy, P (2003) Higher Education in
  the Netherlands Country Report, CHEPS

13
Research Funding and Evaluation in the
Netherlands (VSNU)

• Universities are funded for teaching and research
  as a block grant based primarily on historical
  circumstances
• Development of Research Institute and Schools
  Structure (separate within Faculties)
• Research (mainly self-evaluation) based on
• Quality
• Productivity
• Relevance
• Vitality and Feasibility

14
The Dutch System key issues

• Funding not linked to evaluation of research work
• Self-evaluation of research
• Research School/Institute structure means that
  academics can be either research or teaching only
• BUT
• Binary system/small university sector
• Institutions use evaluation to manage, control
  and direct research activities
• Possible separation of teaching and research
  staff within faculties but also more
  collaboration of staff across institutions

15
Research Funding and Evaluation in Australia
(2008?)

• 40 universities
• Imminent change to the Research Quality Framework
  (RQF) combining quantitative and qualitative
  evaluation
• On going debates on the formation of the RQF
• Concern with basket of metric indicators
  metrics working group
• Concern with Impact of research (social,
  economic, environmental) impact working group

16
Distribution of publication output by field,
Australian universities, 1999-2001 (Butler, 2006)
17
Research Funding and Evaluation in New Zealand
(2006, 2012)

• Performance-based Research Funding (PBRF)
  quantitative and qualitative indicators
• Quality of researchers (60)
• Reflect research degree completions (25)
• Reflect external income (15)
• 45 institutions
• Some Key Aims of the PBRF
• Ensure that research continues to support degree
  and postgraduate teaching
• Prevent undue concentration of funding that would
  undermine research support for all degrees

18
What can be learned from other national systems
that utilise metrics?

• Quantitative measures should inform qualitative
  judgement not replace it.
• A basket of metrics needs to be utilised rather
  than simplistic single indicators.
• Perverse research practices can be encouraged
  where simplistic measures are utilised (Butler,
  2003).

19
Can other national systems provide alternative
ideas?

• Widening the definition of research and
  encouraging the integration of research and
  teaching
• Encouraging greater collaboration across the
  sector to build capacity
• Research for whom? Impact and communication of
  research
• Importance of combined quantitative and
  qualitative indicators and variety of indicators
  to be utilised
• Indicator utilised that are appropriate for
  different subject areas
• Holistic approach concern to support the whole
  higher education sector

20
Dr Martin (Biology, Golden County University)

• I would rather see all the vice chancellors
  lined up for a hundred yards dash and just assign
  money on that basis, because that exercise would
  take at the most two minutes. Even the weakest
  vice chancellor could do a hundred yards in two
  minutes and then get on with life. It is about as
  rational as that. At least you could train your
  vice chancellor and pick a healthy one. At least
  you would have a use for a vice chancellor at
  long last, youd be able to select on a rational
  basis. It might televise well and you might get
  money from the rights on watching it. And you
  might get rid of a few each time.

21
References

• Butler, L. (2003) Explaining Australias
  increased share of ISI publications the effects
  of funding formula based on publication counts,
  Research Policy, 32 143-155.
• Butler, L. (2006) Research Assessment moving
  beyond journal outputs, SPRU 40th Anniversary
  Conference The Future of Science, Technology
  and Innovation Policy, University of Sussex,
  11-13 September.
• DEST, Research Quality Framework, Australia
  http//www.dest.gov.au/sectors/research_sector/pol
  icies_issues_reviews/key_issues/research_quality_f
  ramework/default.htm
• King, D.A. (2004) The Scientific Impact of
  Nations, Nature, 430 (15th July) 311-6.
• Lucas, L. (2006) The Research Game in Academic
  Life, Maidenhead SRHE/Open University Press
• Slaughter, S. Leslie, L. (1997) Academic
  Capitalism politics, policies and the
  entrepreneurial university, Baltimore The John
  Hopkins University Press.
• Tertiary Education Comission, Performance-based
  Research Fund, New Zealand, http//www.tec.govt.nz
  /funding/research/pbrf/pbrf.htm
• UGC (2002) Facts and Figures, Hong Kong SAR.