Workshop on the Use of Evaluative and Comparative Information

1
Workshop on the Use of Evaluative and
Comparative Information in the Management of
Tertiary Education Systems Bratislava Slovak
Republic June 16 17, 2005 Don Thornhill
2
Background
Commission from ARRA / World Bank
?
Provide consultancy advice, with Lewis Purser
?
3
Original Proposal Two roles for ARRA
ARRA publish rankings of HEIs in Slovakia
?
Framework for quality assessments
?
4
Important and exciting project
5
Information availability
Enhances choice
?
Stimulates competition
?
Leads to improved outcomes
?
6
Benefits
Strengthens advocates of change and reform
(including institutional leaders)
?
Weakens status quo
?
Punctures complacency
?
7
The question of ranking ?
8
Considerable growth
Nationally
?
Internationally
?
Controversy
Outcomes
?
Methodologies
?
But here to stay ?
9
Limitations of ranking - 1
Subjective judgement?
?
Quantifiable criteria only?
?
lemons, apples and oranges
?
Input focus
?
10
Limitations of ranking - 2
Aggregation and averaging at institutional level
may tell little about departments, faculties or
schools
?
Conflating scores weightings can be subjective
?
May not capture value added
?
11
Limitations of ranking - 3
Incentives for capture positive role of media
?
Perverse incentives within institutions hide
rather than confront problem issues?
?
12
Our conclusion
Primary focus information supply and system
?
Caution about ranking
?
Diversity of needs
?
Provide a ranking tool?
?
13
Informing principles
The facts should speak for themselves (ARRA
should not be a judge or advocate)
(1)
Careful attention to data quality Three sources
(2)
HEIs
?
Government and official agencies
?
Independent third parties (e.g. survey data)
?
14
HEI landscape
Slovak Accreditation Commission (SAC)
?
ARRA
?
HEIs
?
Progress towards system-wide institutional
evaluation (importance of international
involvement)
15
Mutually supportive, but distinct
missions leading to ..
16
The implementation and sustained development of
a transparent system for the reporting of data
which will encourage the development of a robust
quality assurance and improvement for higher
education and research in Slovakia and its
contribution to economic and social development.