Evaluation practice in the Nordic countries: Different national traditions or a common approach?

1
Evaluation practice in the Nordic countries
Different national traditions or a common
approach?

• Hanne Foss Hansen
• Department of Political Science
• University of Copenhagen

2
Structure

• Educational evaluation Concepts and approaches
• Case 1 Higher education
• -Brief reviews country by country
• -Similarities and differences
• Case 2 Primary and secondary education (P/S)
• The effects of all this evaluation? The future?

3
The concept of evaluation

• Everyday language Measurement, assessment,
  judgement
• Evaluation language A careful assessment of the
  merit and worth of processes, structures, output
  and outcome of interventions and organizations,
  intended to play a role in future, practical
  actions situations

4
The concept of educational evaluation

• Testing, student assessment, programme
  evaluation, personel evaluation, auditing,
  accreditation, benchmarking, curriculum
  evaluation and probably even more.

5
Educational evaluation Focus on many levels

• -Individuals (pupils, students, teachers)
• -Classrooms/courses
• -Curriculum/programmes
• -Organizations (schools, universities)
• -Fields (all schools in a municipality, all
  programmes in a discipline)
• -The national level (national quality development
  and quality assurance systems)
• -The international level (PISA, EQUIS in the
  business school area)

6
Educational evaluation Many purposes, many uses

• -Documenting
• -Controlling
• -Learning/improving
• -Reforming
• -Legitimating
• -Symbolizing

7
Focus today primarely on

• The new forms of evaluation (programme
  evaluation, auditing, accreditation etc.) not on
  the classical questions of testing and student
  assessment
• Meso-evaluation defined as evaluation coupled not
  only to professional practice but also to
  educational policy

8
Higher education I

• Adoption of evaluation in the late 1980s
• 1992-1999 The Danish Center for Evaluation of
  Higher Education
• 1999 The center is reorganized into the Danish
  Evaluation Institute (EVA)

9
Higher education II

• 1990s Programme evaluation
• 2002 Accreditation is introduced
• 2003 A new university law stresses the
  responsibility of the universities themselves to
  conduct evaluations (EVA unclear role)
• 2004 Auditing is introduced
• ----
• 2005 EVA is made responsible for accreditation
  of professional education

10
Higher education I

• Adoption of evaluation in the late 1960s and
  early 1970s, gaining renewed priority in the mid
  1990s
• 1995 The National Agency for Higher Education
  (Högskoleverket) is established

11
Higher education II

• 1999-2002 Auditing is the main task
• 2001-2006 Programme evaluation becomes the main
  task
• Accreditation is also part of the picture

12
Higher education I

• Adoption of evaluation in the mid 1990s
• 1996 The Finnish Higher Education Evaluation
  Council (Finheec) is etablished

13
Higher education II

• As law places responsibility for evaluation with
  the higher educational institutions an important
  purpose of the council is to help institutions to
  develop quality assurance and development systems
• The council also initiates evaluations of
  different types
• Accreditation is important in relation to
  polytechnics and professional courses
• 2004 Auditing

14
Higher education I

• Adoption of evaluation in the late 1990s
• 1998 Norgesnettrådet is established
• 2003 The Norwegian Agency for Quality Assurance
  in Education is established

15
Higher education II

• Auditing of all higher educational institutions
• Accreditation of programmes and institutions
  applying for new programmes and institutional
  status
• The Ministry of Education initiates evaluations
  of higher educational reforms (Høgskolereformen,
  Kvalitetsreformen)

16
Higher education

• Adoption of evaluation in the mid 1990s
• 1999 It becomes mandatory for higher educational
  institutions to develop quality assurance systems
• The Ministry of Education initiates programme
  evaluation ad hoc
• No formalised accreditation system

17
Similarities across countries in talk,
organisation and focus

• Adoption of meso-evaluation in all countries
• Anchoring evaluation in semi-autonomous
  organizations specialized in evaluation (not
  Iceland)
• Educational evaluation is decoupled from
  evaluation of research
• A turn towards auditing (N, DK, FIN)?

18
From national imitation to international
regulative pressures?
strong
Bologna
2005
National pressures
strong
weak
1990
weak
International pressures
19
Factors explaining convergence

• Public-sector reforms New Public Management,
  focus on results and effectiveness
• Internationalization The Bologna proces and the
  aim of establishing a European Higher Education
  Area in 2010
• Networking across agencies at Nordic as well as
  European level

20
Differences in institutional processes

• Time span in adoption (from Sweden in the late
  1960s, to Denmark in the late 1980s and Norway
  in the late 1990s)
• Time span in institutionalization (e.g.
  routinization in Denmark from 1992, in Norway
  from 2003)
• Norway as the late adopter has constructed the
  most radical system

21
Differences in balances between quality
development (QD) and control (C) purposes

• DK QD more than C (except professional
  education)
• S From C more than QD to QD more than C
• FIN QD more than C (except professional
  education)
• N C but also QD
• IS QD more than C

22
Differences in decision contexts

• From Denmark where there is no direct coupling to
  sanctioning and rewarding (except in professional
  education) to Norway where there is a direct
  coupling to sanctioning and rewarding with Sweden
  somewhere in between

23
Differences in evaluation models

• - Self-evaluation is an important element in DK,
  S and FIN but not in N

24
Differences in composition of evaluation panels
DK N S
Peers
Educational research -
Educational leadership - -
Students -
Other users - -
25
Differences in coordination across individual
evaluations
Coordination by Denmark Norway
Procedures Strong Strong
Specified criteria Only used in some evaluations Strong
Panel members Weak Strong
Board decisions - Strong
26
Factors explaining divergence

• Differences in
• political-administrative cultures
• strategies in public-sector reforms
• structures and traditions in educational systems
• timing and content of higher educational reforms

27
P/S education

• Late 1990s the Ministry of Education introduces
  a program Quality development in public
  Schooling (attention and tools )
• 1999 EVA gets responsibility for evalution in
  P/S
• 2002 A law about transparency and openness makes
  it compulsory to educational institutions to
  publish evaluations of the quality of teaching
• 2005 Government proposes to establish a council
  and an agency for quality development

28
P/S education

• 1997 Municipalities have each year to work out
  written quality reports
• 2003 The agency for education is split up in the
  Swedish Agency for Education and an agency for
  school development
• 2004-2009 Inspection programme. Inspection
  reports serve as starting points for improving
  the quality of municpal schooling.

29
P/S education

• 2003 A council for educational evaluation is
  established. The council has to plan and
  implement external evaluations as well as develop
  methods and coordinate local evaluation

30
P/S education

• 2004 The Directorate for Primary and Secondary
  Education is established. The directorate is
  responsible for an internet-based quality
  assessment system ensuring transperency in
  quality information.

31
P/S education

• Schools have to do and publish self-evaluations
• Every 5th year The Ministry of Education assesses
  the evaluation methods used by schools
  (site-visits)

32
P/S education Similarities

• Evaluation adopted in all countries
• International studies have put educational
  quality and evaluation on the agenda (PISA
  TIMMS)
• All countries build national institutional
  capacity to deal with quality and evaluation
  (increasing state control)
• Transparency in monitoring is important
  (strenghtening market forces)

33
P/S education Differences

• Balances between quality development and control
  purposes (S C control but also QD DK, N, FIN
  and IS more soft approaches)
• ? Too early to really conclude on the practice
  of the new agencies

34
Comparing the two cases

• Higher education
• -Time span in adoption (from late 1960s to late
  1990s)
• -Policy-driven development

• P/S
• -Later adoption but no time span
• -Problem-driven development (DK, N)

35
Evaluation practice in the Nordic countries
Different national traditions or a common
approach?

• Conclusions
• Similarities in talk
• Similarities and important differences in actions
• Evaluation is an elastic concept giving room for
  national and local constructions

36
Effects of growth in meso-evaluation I

• Two very different ways of thinking
• Optimism related to the development of learning
  organizations and a knowledge society
• Pessimism related to the development of an audit
  society based om distrust

37
Effects of growth in meso-evaluation II

• Are educational institutions transformed into
• learning organizations or into auditable
  commodities?
• Is professional practice part of or de-coupled
  from evolving evaluation cultures?
• Limited empirical knowledge in the Nordic
  countries

38
The future




 
 
 
 
 

• A turn towards
• -Auditing and accreditation?
• -Evidence-based professional practice?
• -Evidence-based educational policy?