Quality assurance in higher education: some international trends Lee Harvey www'uce'ac'ukcrq Higher

1
(No Transcript)
2
Quality assurance in higher education some
international trends Lee Harveywww.uce.ac.uk/c
rq Higher Education ConferenceOslo 22-23
January 2001
3
Overview

• Evaluators

• Delegated accountability

• Purposes

• Internationalisation

• Bureaucracy, legitimacy and burden

• Impact

• Concluding thoughts

4
Internal
External processes are not the primary mechanism
by which quality in higher education is assured.
Day-to-day quality assurance is from internal
academic processes. External processes should
augment internal procedures. Keep the
relationship with internal processes in mind when
analysing external evaluation.
5
Evaluation
Evaluation is an umbrella term for all forms of
quality monitoring, assessment, audit,
enhancement legitimation, endorsement or
accreditation.
6
Evaluators
7
Independent
Whether, in theory, the agency is independent the
reality, in practice, will be mediated by the
organisational ethos, culture, and working
practices. Hence any mapping of independence
status has to take account of the sociology of
the organisation.
8
Delegated accountability
Market
Semi-market
British
Growing state control via EQM
Accountability requirements
Accred-itation
Impro-vement
Accountable autonomy
New autonomy
Newly-devolved
Continental
Centralised
9
Norway

• Delegated accountability and accreditation?
• Improvement internal responsibility?
• How to avoid high cost, intrusiveness and yet be
  potent?
• Role of the agency in increasing institutional
  autonomy?

10
Focus
External monitoring purposes
Rationale
accreditation
accountability
audit
improvement
assessment
standards monitoring
11
Focus
Rationale
improvement
accountability
accreditation
audit
assessment
Standards monitoring
12
Accreditation

• Institution or programme (inputs)
• Public recognition threshold standard, granting
  of licence
• Integrity
• Regional, national or international bodies
  government or agency with delegated power
• Validation, revalidation, (initial)
  accreditation, re-accreditation

13
Audit

• Quality assurance procedures existence or
  effectiveness
• Conformance or compliance
• Documentation and visit
• Non-evaluative
• Drill-down option
• Improvement audit
• Sweden, Finnish polytechnics, NCA in US, AUQA

14
Assessment

• Measure overall or components
• External criteria/fitness for mission
• Complex grading or threshold league tables
• Students receive comparable service
• Research assessment
• Evidence visits, self-ass., observation,
  feedback
• Value added
• Benchmarking

15
Standards monitoring

• Long established external exam/ministries
• Academic/ professional competence
• Specify or check direct, inferred, indirect
  Benchmark standards (UK)
• International comparison
• Credit accumulation and transfer systems

16
Benchmarked subjects

• Accounting
• Archaeology
• Architecture
• Business Management
• Chemistry
• Classics
• Computing
• Environmental
• Economics
• Education
• Engineering

• English
• Geography
• History
• Hospitality etc.
• Law
• Librarianship
• Philosophy
• Politics
• Social Policy etc.
• Sociology
• Theology

17
Benchmarks
Discipline differences have been thrown into
stark relief in the UK by the development of
benchmarks for subject standards.
law
sociology
18
Law benchmarks
...any student graduating in law must show
achievement in all the following seven areas of
performance
application and problem-solving
knowledge
sources and research
general transferable intellectual skills
communication and literacy
autonomy and ability to learn
numeracy, information technology and teamwork.
19
Sociology benchmarks
... all programmes will address the following
areas of sociological knowledge and understanding
social context, processes, diversity and
inequality
key concepts and theories
comparative analysis
individuals, groups and institutions
role of culture in social life
processes underpinning social change
relationship between socio-logical argument
evidence
the distinctive character of sociological
undertstanding
methodology
20
External monitoring purpose
documentation
accreditation
assessment
audit
standards monitoring
visits
21
Reasons for accountability

• Massification

• Accountability for public and private money

• Globalisation of higher education

• Unrestricted market

• Concerns about quality and standards

22
Accountability

• Value for public money
• No erosion of academia credibility
• Delivery and value to students
• Public information transparency accessibility
• information models, league tables, feedback,
  useful.
• Compliance information, political agendas,
  resources smokescreen

23
Improvement

• Expectation of improvement secondary
• Some primary Swedish audits
• Accountability versus improvement
• second stage emphasis,
• continuous improvement accountability
• Catalyst dialogue and trust.
• Performance indicators manipulation to meet
  targets

24
Internationalisation

• Mutual recognition
• International agencies
• CRE UNESCO codes, EQUIS
• Supranational agencies
• quality label of evaluators (IAUP/INQAAHE/UNESCO)
  
• Convergence criteria and methodology
• cultural blindness
• Accreditation obsession US myth/audit
• Trust and unnecessary accreditation

25
Bureaucracy, legitimacy and burden

• Minimum bureaucracy but inevitable
• Legitimacy linked to purpose
• Some more burdensome than others
• UK juggernaut

26
External monitoring
Organisation
External process
Process
HEQC/QAA
Institutional audit
Documentation, self-assessment, visiting panels
Subject (teaching) assessment
HEFCs/QAA
HEFCs/QAA
Research assessment exercise
Documentation
Documentation, visits, exams
PRBs
Professional accreditation
Moderation/ critical friend
External examiners
Institutions
27
Standards infrastructure
Infrastructure element
Status
Subject benchmarking
22 drafts published 21 underway
Qualifications framework
E,W NI 5 levels. Scotland separate
Specify knowledge, key, cognitive and
subject-specific skills
Programme specifications
8 topics 2 drafts
Code of practice
Common-format transcript reflection
Progress files
28
Inflation
Will accreditation systems be inflated so as to
invade and dominate the domain of evaluations and
cause these to refer more to fixed standards? a
case in point is the recent development in
Sweden, where a programme of comprehensive,
cyclical subject reviews is now being introduced,
with accreditation control as one of its explicit
aims. Will more comprehensive systems of
accreditation control really assure and enhance
educational quality, and if so, at what cost?
Jon Haakstad, 2001
29
Impact of external evaluation

• Little research, lots of opinion
• Conservative and inhibiting
• Conformity of process
• Lack of diversity of provision
• academic cleansing
• Vehicle for government policy
• Temporary rapid fade, lack of embedding, event,
  paranoia shift
• Learning other factors

30
Impact
external evaluation
other factors
Processes and performances
learning
31
Impact of external evaluation (continued)

• Biased selection
• Naïve accountability not improvement
• Value for less money,
• Legitimation of deteriorating quality
• Control of academia
• Political opportunism short-termism
• Research into implementation process

32
Conclusion personal model

• Start with shortcomings (and needs) not
  methodology or auspices
• Be explicit about the politics of evaluation
• How will internal and external processes develop
  dynamically and harmoniously
• successful procedures are characterised by
  dialogue, an improvement process, trust and
  transparency
• Resentment of costly, complex and burdensome
  processes

33
Conclusion personal model (continued)

• No peer review
• Professional inspectors enabling inspection
• Patch critical friend
• Dialogue, critical self reflection, ongoing
  improvement linked to internal annual cycle
• Periodic reporting and discussion
• Professional, cheaper, less burdensome, more
  meaningful and far more effective

34
Thank you
http//www.uce.ac.uk/crq