Quality Assurance in Higher Education

1
Quality Assurance in Higher Education
Recognition proceduresof agencies

• Bruno CURVALE
• Head of international affairs at AÉRES
• Agence dévaluation de la recherche et de
  lenseignement supérieur, France
• ENQA Vice President
• European Association for Quality Assurance in
  Higher Education

2
Outline of the presentation

• 1. Objectives and organisation of the work
• 2. Reminder How to become an ENQA full member?
• 3. The review of the review
• 4. Questions and issues for the working groups

3
1. Objectives of the session

• To raise awareness about ENQA membership
  requirements and mechanisms.
• To think about what makes the quality of an
  agency review and at how to assess this quality.
• To include ENQAs members in the ongoing
  discussion about ENQA membership.
• To help the board to complement and improve
  ENQAs documentation and guidelines.
• To help the board to achieve coherence in the
  membership requirements and processes.

4
1. Organisation of the session

• A presentation aimed at giving a common
  background to the participants.
• Work in parallel workshops. Three main topics
• Meta-evaluation objectives, role, constraints
  and difficulties.
• Reviewing and complementing the Guidelines for
  national review of ENQA member agencies.
• Reviewing and complementing the Briefing pack for
  review panel members of ENQA coordinated reviews.
• A debriefing of the workshop sessions.

5
2. How to become an ENQA full member?

• Coexistence of two mechanisms
• The membership application form and the analysis
  of it
• The scrutiny of the agencys review report
• Since 2005 the ESG are at the core of the
  membership.
• In consequence the two mechanisms are largely
  redundant
• But not completely
• The scrutiny of member reviews is taking a
  pre-eminent part (Cyclical review every 5 years,
  )
• A better linking, or a simplification, could be a
  desirable objective.

6
2. The analysis of the membership application form

• The criteria
• 1. Activities
• 2. Official status
• 3. Resources
• 4. Mission statement
• 5. Independence
• 6. External quality assurance criteria and
  processes
• 6.1 Processes, criteria and procedures
  pre-defined and publicly available
• 6.2 Processes expected to include
  self-assessment, external assessment, publication
  of a report including outcome, follow-up
  procedure
• 7. Accountability procedures
• 8. Miscellaneous
• 8.1 Professionalism and consistency of judgment
• 8.2 Appeal procedure if the matter makes it
  necessary
• 8.3 Willingness of the agency to contribute to
  the aims of ENQA

7
2. What ENQA does?

• The Membership Committee
• consists of 3 members of the Board
• reports to the Board
• The role of the Committee
• To answer the fundamental question
• Does the final report of the agencys review
  provide sufficient, verified evidence that the
  agency meets the ENQA membership criteria and
  thereby the European Standards and Guidelines for
  Quality Assurance in Higher Education (ESG)?
• The evaluation is desk-based according to a
  reading of the application and a consideration of
  what is said and the evidence provided by the
  applicant.
• Today, considering that a review of the agency is
  part of what is needed to become a full member,
  this option should only lead to a candidate
  membership.

8
2. The scrutiny of the member review

• The criteria
• 1. Activities
• 2. Official status
• 3. Resources
• 4. Mission statement
• 5. Independence
• 6. External quality assurance criteria and
  processes
• 7. Accountability procedures
• Integrity of the review process
• Can we trust this review process that says this
  agency complies with the ESG?

9
2. What ENQA does?

• The Review Committee
• It is made of 3 members of the Board.
• It reports to the Board.
• The role of the Committee
• To answer two fundamental questions
• Does the information provided in the final
  report of the agencys review satisfy the board
  that the review was conducted to the required
  level of independence, integrity and robustness?
• Does the final report of the agencys review
  provide sufficient, verified evidence that the
  agency meets the ENQA membership criteria and
  thereby the European Standards and Guidelines for
  Quality Assurance in Higher Education (ESG)?
• The evaluation is desk-based according to the
  reading of the review report and a consideration
  of what is said and the evidence provided by the
  panel.

10
3. The review of the review

• The elements of the judgment What is the Review
  Committee looking at?
• To be kept in mind
• A review is made up of a process and of outcomes.
• The reviews forwarded to ENQA normally conclude
  that agencies comply with the ESG.

11
3. Evaluation AND meta-evaluation
Inclusion in the EQAR
ENQA full membership
Full membership of ENQA normally constitutes
satisfactory evidence for substantial compliance
with the ESG
Evaluation of the evaluation
Evaluation of the evaluation
ENQA
Register Committee
External review
Evaluation against ESG 3
Decided Coordinated For ENQA
membership For other purposes For
application to the Register
Procedures, behaviour
Agency
National regulations
Purposes
Q Standards
Specific objectives
Source Bruno Curvale
12
3. The review of the review

• It is the evaluation of a process of which the
  quality deals mainly with
• The coordination of the review (organised on
  national basis/by ENQA/)
• ENQA provides Guidelines Guidelines for national
  reviews of ENQA member agencies.
• The independence of the reviewers.
• The evidence and facts that support the
  assessment.
• ENQA provides for each panel it coordinates a
  Briefing pack for review panel members of ENQA
  co-ordinated reviews.

• Question What do we need to know and need to do
  in order to be sure about the quality of the
  review process?

13
4. Questions or issues for the working groups

• Is the word meta-evaluation satisfactory? What
  are the difficulties and specific constraints of
  this exercise?
• Opportunity of purposes and quality. To what
  extent do the purposes of the agency have to be
  taken into account when looking at the quality of
  its activities?
• What supplementary documentation would be useful
  when reviewing the review report?
• The notion of substantial compliance. How to deal
  with it?
• The risk of formalism when dealing with a notion
  like independence.
• What could be or should be improved in the
  Guidelines and Briefing pack?

14

• Thank you for your attention

15
Recommendation of the review committee - 1

• The report does not provide sufficient evidence
  that the review was conducted to the required
  level of independence, integrity and robustness.
• No conclusion can be drawn as regard the
  compliance of the agency with the ESG (ENQA
  membership criteria).
• Consequences
• The Board may request further information
• The self-evaluation document, internal agency
  documentation,
• The Board may want to discuss with the panel in
  order to get clarification.
• The application can be considered again if
  clarification are pertinent.
• The agency can appeal against the board decision.

16
Recommendation of the Review Committee - 2

• The report provides sufficient evidence to
  conclude that the agency does not adequately
  comply with ESG (ENQA membership criteria).
• ? The Review Committee contradicts the Review
  Panel.
• The committee recommends that the Board does not
  re-confirm or grant the full membership.
• Consequences
• The Board may give the agency a candidate
  membership meaning that the agency has two years
  to conform to the criteria.
• In case too much criteria are problematic the
  review can be declared unacceptable.
• In case the agency was already at the end of a
  candidate membership, this membership is
  terminated and the agency might apply for
  Associate status.
• The agency can appeal against the board decision.

17
Recommendation of the Review Committee - 3

• The Review Committee finds the review was
  conducted to the required level of independence,
  integrity and robustness.
• The report does not provide sufficient evidence
  to conclude that the agency does adequately
  comply with ESG (ENQA membership criteria).
• The committee recommends that the Board request
  further information.
• Consequences
• The Board request further information
• The self-evaluation document, internal agency
  documentation,
• The Board may want to discuss with the panel in
  order to get clarification.
• The application is considered again after
  clarification.

18
Recommendation of the Review Committee - 4

• The Review Committee finds the review was
  conducted to the required level of independence,
  integrity and robustness.
• The report provides sufficient, verified evidence
  that the agency meets the ESG (ENQA membership
  criteria).
• The Review Committee recommends the Board
  confirms or grants the agency ENQA Full
  Membership.
• Consequences
• Normally, the Board follows the recommendation.