ENQA procedures for external review the Bulgarian experience

1
ENQA procedures for external review the
Bulgarian experience

• Todor.Shopov_at_yahoo.com
• Vilnius,30 May 2009, 11.15-11.45 Hrs.

2
Objectives

• The Agency (3 16)
• The External Review (17 30)
• The External Review Report (31 35)
• The EQAR Application (36)
• The Current State of Affairs (37 40)
•  

3
The Agency (est. 1996)
4
Official Status

• a statutory body for evaluation, accreditation
  and monitoring of the quality in higher education
  institutions and scientific organizations, aiming
  at the enhancement of their teaching and research
  as well as of their development as scientific and
  cultural organizations

5
Activities

• reviews the ability of institutions to provide
  good quality of education and scientific research
• performs project evaluation, institutional and
  programme accreditation, post-accreditation
  monitoring of quality of higher education

6
Mission

• to encourage higher education institutions in
  assuring and enhancing the quality of education
  they offer by sustaining high academic standards
  and good education traditions in Bulgaria

7
Structure

• Decision-making bodies
• Accreditation Council
• Standing Committees on areas of higher education
• Administration
• Expert teams (ad hoc)

8
Standing Committees

• formed by the Accreditation Council which
  appoints their members through selection by
  applications
• comprised of 3 to 7 members, one of whom is
  Chairman of the Committee
• the Chairman of the Accreditation Council
  appoints the members of the Committees for a
  three-year term

9
Standing Committees

• (in alphabetical order)
• AGRARIAN SCIENCES AND VETERINARY MEDICINE
• ECONOMIC SCIENCES AND MANAGEMENT
• EDUCATIONAL SCIENCES AND MUSICAL AND DANCE ART
• HEALTHCARE AND SPORTS
• HUMANITIES AND ARTS
• NATURAL SCIENCES, MATHEMATICS AND COMPUTING
• SOCIAL SCIENCE, LAW AND NATIONAL SECURITY STUDIES
• TECHNICHAL AND MILITARY SCIENCES

10

• Standing Committee on
• POST-ACCREDITATION MONITORING AND CONTROL
• supervise HEIs implementation of the
  recommendations received in the evaluation and
  accreditation process

11
Internal Quality Assurance of NEAA

• IQA system
• Corrector-NEAA

12
IQA system

• Four subsystems
• legal base
• standards, policies, procedures and documents
• management subsystem
• material resources and financing

13
Corrector-NEAA

• Mechanisms for collecting feedback and
  recommendations on improving the activity
• Internal relations for feedback from
  administration, Standing Committees and
  Accreditation Council
• External relations from individuals (academic
  experts, students, professionals) and
  higher-education institutions

14
External Quality Management (Control, Assurance,
Improvement)

• quality terms
• Quality Improvement can be distinguished from
  Quality Control in that Quality Improvement is
  the purposeful change of a process to improve the
  reliability of achieving an outcome.
• Quality Control is the ongoing effort to maintain
  the integrity of a process to maintain the
  reliability of achieving an outcome.
• Quality Assurance is the planned or systematic
  action necessary to provide enough confidence
  that a product or service will satisfy the given
  requirements for quality.

15
The national system of quality management in
higher education

• PRINCIPLES
• quality is not in the hands of TPTB ("the powers
  that be")
• the "chorus of stakeholders" rules

16
The national system of quality management in
higher education
17
The External Review

• the application 7 November 2007
• the language "it's not an English exam
•  the patterns to design the preparation process
  effectively

18
The Patterns

• The patterns (A and B) help us plan and execute
  the external evaluation procedure.
• That provides the Agency with positive
  experiences.

19
Pattern A, identified in the preparation for the
external review

• Pattern A is a complicated scheme. It is known
  as an ill-formed system.
• We could not use it effectively.

20
(No Transcript)
21
Pattern B, identified in the preparation for the
external review

• We needed a well-organised, lean structure.
• So, in June 2007, we developed Pattern B, based
  on classic model of quality management.

22
Pattern B, identified in the preparation for the
external review

• INPUT ? PREPARATION
• PROCESS ? OUTPUT ? SITE
• VISIT ? EFFORTS TO IMPLEMENT
• RECOMMENDATIONS

23
INPUT

• the state of affairs in June 2007

24
PREPARATION PROCESS

• translation of Guidelines Marinov task
  force appointed, issues identified leadership,
  transparency of processes, accountability of
  participants, timelines, etc.
• other issues (we are working on them even now)
  Participation in NEAAs expert teams (ESG 2.4,
  3.7) Reporting (ESG 2.5) Independence of
  recruiting experts (ESG 3.6) Composition of
  decision-making bodies (ESG 3.6, 3.8)

25
OUTPUT

• self-evaluation report (May 2008)

26
The site visit of the Review Panel

• On June 17-18, 2008 an external expert group
  under the aegis of ENQA, coordinated by the
  Spanish Quality Assurance Agency ANECA, visited
  the NEAA to perform an external review of its
  activities. The on-site visit is part of the
  procedures for ENQA membership established in the
  ESG.
• Within the two-day visit the expert group had
  working meetings with representatives of the NEAA
  leadership and its staff as well as with
  representatives of the State Administration,
  Local Communities, Trade-Unions, Rectors
  Council, Higher Education Institutions and
  Students.

27
The TOR

• The TOR were approved by the NEAA
• (14 Feb 2008) and ENQA (3 March
• 2008).

28

• describe the purpose and the structure of the
  external review, i. e. what has to be achieved,
  who will take part in it, how it will be
  achieved, when it will be achieved. The TOR are
  enclosed in the evaluation report. They contain 8
  sections explaining the review process, the
  review panel (6 members), the self-evaluation
  report, the site visit, the evaluation report,
  the final decision by the board of ENQA, the
  publication of the evaluation report, and the
  budget of the review.

29
Official correspondence with ANECA
30
The self-evaluation report

• The task force, appointed by the Chairman
• of NEAA (June 2007), met regularly in the
• period June 2007 June 2008.

31
THE EXTERNAL REVIEW REPORT

• It was produced, as planned, in July 2008.
• Its conclusions are presented in Chapter 5,
• entitled "NEAA's compliance with ESG". The
• first section, "Summary of strengths and areas
• of improvement" describes the positive and
• negative aspects of the activities of NEAA
• (strengths and weaknesses).

32
The second section of Chapter 5, "Final
statement", includes the following positive
evaluation

• "In the light of the documentation and oral
  evidence
• provided during the site visit, the Review Panel
  is satisfied
• that in the performance of its functions NEAA is
  in
• substantial compliance with the ENQA Membership
  Criteria
• and with the Standards and Guidelines for Quality
• Assurance in the EHEA. The Panel therefore
  recommends
• to the Board of ENQA that NEAA should be awarded
  Full
• Membership for a period of five years.

33
July 2008

• comments of NEAA on the factual accuracy of the
  external review report

34
(No Transcript)
35
ISSUES

• Problems solved at present
• Participation in NEAAs expert teams (ESG 2.4,
  3.7)
• Reporting (ESG 2.5)
• Independence of recruiting experts (ESG 3.6)
• Composition of decision-making bodies (ESG 3.6,
  3.8)

36
THE EQAR APPLICATION

• The application was submitted in
• February 2009. Two letters with
• additional explanations were sent to the
• EQAR Chair in March and May 2009.

37
THE CURRENT STATE OF AFFAIRS

• NEAA has a new goal
• to make the process of evaluation and
• accreditation leaner, more user-friendly and
• more effective. For example, the two
• procedures of institutional and programme
• accreditation may be merged into one
• appraisal procedure at institutional level.

38
THE CURRENT STATE OF AFFAIRS

• NEAA has formed a working group to
• examine all issues related to its work and to
• propose fundamental changes in the
• accreditation of higher education institutions
• in line with current developments in quality
• assurance in education.

39
THE CURRENT STATE OF AFFAIRS

• In September 2009, the term of office of all
• Standing Committees will come to an end.
• New members will be elected and in this
• process students and non-academic
• professionals will be included as members
• of the committees. This decision was taken
• by the Accreditation Council in March 2009.

40
THE CURRENT STATE OF AFFAIRS

• In 2010, the NEAA will submit an interim
• report on the progress in implementing the
• changes recommended by ENQA.

41

• THANK YOU!