WE HAVE COME THIS FAR QUALITY ASSURANCE IN TRANSITION

1
WE HAVE COME THIS FAR QUALITY ASSURANCE IN
TRANSITION

2
Managing Institutional Performance best
practice for senior management

• Simeon Underwood
• London School of Economics
• 5 July 2004

3
Describing quality assurance I

• We knew that the islands were beautiful
• somewhere round about here where we are
  groping
• a little lower or a little higher,
• the slightest distance.
• from Mythistorema 8, George Seferis

4
Describing quality assurance II

• Still, a light wash of grey is on things,
• A scumbling. It said these places were new
• Just last year
• And still are, in places. Yet there is nowhere
• To hide this year, and the newness
• Keeps coming on. No one explained to it. Is
  encroaching.
• from Fall Pageant, John Ashbery

5
Quality assurance in transition

• The lighter touch ?
• Audit back to basics
• Doing quality assurance themes and players
• What have we done to ourselves ?
• Where do we go from here ?

6
The lighter touch ?

• An end to subject level review (?)
• Significant differences in institutional exposure
  to external inspection
• Reliance on institutional internal systems
• Marginalisation of the QAA
• No return to over-bureaucratic practices

7
The lighter touch ?

• Enhanced Institutional Audit (with Discipline
  Audit Trails)
• plus the quality infrastructure
• plus student input
• plus Teaching Quality Information
• plus the National Student Survey
• plus thematic enhancement
• plus Bologna

8
The lighter touchthe academic infrastructure

• Qualifications framework
• Subject benchmark statements
• Programme specifications
• Codes of practice
• Progress files (?)
• TQI (?)
• Variant version for the Bologna process

9
The lighter touchthe received wisdom

• It was suggested to us that, while QAA
  Institutional Audits are much less burdensome
  than the previous QAA regime, the total effort on
  teaching quality management required to satisfy
  the expectations of external reviewers was not
  less than before, it has merely been
  internalised within the institutions own
  quality management programme. New requirements
  such as TQI reports similarly go beyond the
  information that institutions might otherwise
  collect and publish.
• -- Better Accountability Revisited, PA Consulting
  for HEFCE, 2004

10
The lighter touch debating the process

• One or two questions to start with --
• Are intended learning outcomes an appropriate
  model for QA in higher education?
• Is the priority given to assessment over teaching
  and learning appropriate ?
• If programme specifications didnt exist, would
  we invent them for ourselves ?
• Left to our own devices, what weight would we
  attach to feedback from students ?

11
Audit back to basics I

• the pervasive feature of the new wave of
  audits is that they work not on primary
  activities but rather on other systems of
  control. This gives the audit a more remote
  assurance role that is often understood by the
  publics which they are intended to serve. This
  policing of policing distinguishes the audit
  explosion from an older tradition of
  engineering-based quality control and its
  statistically grounded methods.
• -- Power, The Audit Explosion, p.6

12
Audit back to basics II

• Audits have the remarkable capacity of being
  invulnerable to their own failure. Instead,
  where audit fails, the common response has been
  to call for more of it.
• -- Power, p.7

13
Audit back to basics III

• Far from being passive, audit actively
  constructs the contexts in which it operates.
  The most influential dimension of the audit
  explosion is the process by which environments
  are made auditable, structured to conform to the
  need to be monitored ex-post..
• Audits do as much to construct definitions of
  quality and performance as to monitor them.
• -- Power, p.8 and p.33

14
Audit back to basics IV

• One could argue that, given the right
  information is made available to students, there
  may be no need for institutional quality
  assurance audits. Students would in effect
  provide the quality assurance themselves through
  informed choice Those HEIs providing poor
  quality education would find students preferred
  other places, and would be forced to improve in
  order to attract students. Those already
  providing good quality education would seek
  further improvements to maintain their share of
  the market.
• -- Better Regulation Task Force report, July
  2002, p.29

15
Doing quality assurancesome themes

• Players, processes and possibilities
• Implementation as an act of management
• Implementation as internal relationships
• Implementation in a context of institutional
  individuation
• Implementation in relation to quality enhancement
• Implementation as a linguistic exercise

16
Doing quality assurancea linguistic exercise

• Staff have developed bi-lingualism --
  maintaining both the language and practices of
  audit and their own professionalism side by side
  . Perhaps this is more appropriately termed
  professional schizophrenia as the lecturer not
  only maintains two languages but splits the
  professional self in two. Such duplicity is time
  consuming and exhausting.
• -- Susan Wright, Enhancing the quality of
  teaching in universities, LTSN Generic Centre
  web site

17
Doing quality assurance external and internal
players

• The QAA
• Other agencies (?)
• Institutional leaders
• Academics
• Quality professionals
• Students

18
External playersthe QAA I

• In general, the team was satisfied that the
  University had responded appropriately to the
  Code of practice and had initiated the process of
  taking action to confirm its continuing
  adherence, although further work was required on
  the mechanisms to ensure both local ownership and
  central oversight of local practice.
• -- Institutional audit report, University of
  Cambridge
• (April 2003 Para. 48)

19
External playersthe QAA II

• The University expects its processes to take
  account of the QAAs Code of Practice, but,
  unlike the report, it does expect individual
  staff always to refer explicitly to the Code, and
  it believes that this reflects the spirit of the
  QAAs own guidance.
• Cambridge audit report,
• Response by the University

20
Internal playersinstitutional leaders

• In terms of the leadership of this university,
  it, of course, gives greater power to my arm.
  There is no question whatsoever about that I can
  get people to listen more carefully to the need,
  for example, to have some kind of accountability
  framework the QAA has helped to allow me to be
  heard in an academic institution.
• A female pro-vice-chancellor in an old university
  
• cited in Louise Morley, Quality and Power in
  Higher Education
• (SRHE, 2003), p.51

21
Internal playersinternal relationships

• audit has widened the gulf between teaching
  staff and university management and
  administration, with detrimental impacts on the
  quality and efficiency of teaching
• top-down audit systems have generated a gulf
  between managers and chalk face staff, with a
  lack of communication, mutual understanding and
  trust
• -- Susan Wright, work cited previously

22
Internal playersthe quality professionals I

• A thought police now emerges from the academic
  woodwork to enforce academic management and
  quality audit. Here the Salieri principle
  applies nothing gives greater pleasure to the
  guardians of competence than knowingly to
  suffocate real creativity. Salieri could not
  forgive Mozart his gift. He understood the
  nature of it, for he was himself a musician, but
  by the same token he understood how to destroy
  it.
• -- Richard Roberts, Our Graduate Factories The
  Tablet, 11/10/97

23
Internal playersthe quality professionals II

• We found some evidence that HEIs internal
  quality assurance teams were themselves applying
  the QAA Code over-prescriptively, adding to the
  perception of some academics that the regime was
  burdensome The apparent lack of trust between
  Government and HEIs seems to permeate some HEIs
  internal systems ...
• Cabinet Office Better Regulation Task Force
  report on HE,
• July 2003

24
Internal playersthe students

• The student handbooks seen by the audit team
  were quite basic and did not make use of intended
  learning outcomes a term unfamiliar to the
  students who met the team. The students were,
  nonetheless, satisfied with the information made
  available to them
• Cambridge audit report
• (para. 121, report on a DAT)

25
What have we done to ourselves ?

• The managerialist model
• the model of technocratic space and academic
  complexity

26
What have we done to ourselves ?the
managerialist model I

• audit has widened the gulf between teaching
  staff and university management ad
  administration, with detrimental impacts on the
  quality and efficiency of teaching
• .. top-down audit systems have generated a
  gulf between managers and chalk face staff,
  with a lack of communication, mutual
  understanding and trust
• -- Wright, work previously cited, pp. 2 and 6

27
What have we done to ourselves ?the
managerialist model II

• The audit technologies of quality assessment
  have resulted in a wide gulf between the
  perspectives of managers and lecturers, a lack of
  trust between scrutineers and performers, and a
  feeling among the latter that audit paper trails
  bear little resemblance to reality and are a
  wasteful deflection of energy and resources away
  from their professional interest in improving
  teaching and learning.
• -- Wright, p.10

28
What have we done to ourselves ?the
managerialist model III

• most teaching staff do not understand the
  structure of university decisions making most
  teaching staff do not understand how the
  universitys core functions are organised Often
  we do not know the names and functions of lower
  tiers of administrative staff These university
  managers have arrogated power to themselves
  Academics, meanwhile, find themselves with an
  increased workload but decreasing influence
• -- Wright, pp.4-5

29
QA QE seen through complexity theory
QE WORLD
chaos
zone of complexity on the edge of chaos
QA WORLD
Close to Far from agreement
agreement
rational, technical, political and judgemental
decision making
Close to certainty Far
from certainty
Paul Tosey Teaching on the Edge of Chaos
30
What have we done to ourselves ?academic
complexity I

• the pressure from the wider system (for
  example, the QAA framework the RAE, funding
  systems) are often towards creating certainty and
  risk reduction, but through overloading local
  systems with demands. This seems to drive local
  systems both towards stasis (e.g. we can
  concentrate on producing ever neater and tighter
  controls, QA systems, etc.) and towards chaos
  (e.g. overload of inputs -- demands for QA
  returns, monitoring of research projects and
  funding, on top of local processes of
  restructuring -- may lead the system towards
  breakdown.
• -- Tosey, p.21

31
What have we done to ourselves ?academic
complexity II

• The paradoxes are that systems cannot be
  creative and innovative in an orderly fashion
  nor can they be excellent if their every move is
  monitored. one would expect the local system
  that wishes to survive to filter out these
  demands, reducing, ignoring, or perhaps
  transcending the input overload and refusing to
  become locked into stasis.
• -- Tosey, p.22

32
Where do we go from here ?items for our own
agendas

• Continuing external functions
• Continuing internal functions
• Working with students
• Working on numbers
• Is Quality Enhancement the new Quality Assurance
  ?

33
Where do we go from here ?items on the QAA
agenda I

• Wide range of good practice
• Management and monitoring of collaborative
  provision
• Use of management information
• Engagement of employers in curriculum planning
  and monitoring

34
Where do we go from here ?items on the QAA
agenda II

• Assessment procedures especially robustness in
  sampling and moderation
• Committee structures in teaching and learning
  area
• Student representation (effectiveness and
  attendance)
• Professional training for postgraduate tutors
• The PhD completion rates and processes
• Other postgraduate provision sustainability of
  Masters level activity

35
Where do we go from here -- what are the ways
ahead ?

• Dialogue
• Brokerage
• Radically different models

36
Where do we go from here ?dialogue I

• The parties continually change places between
  being the subject and the object of the
  conversation
• Each partys construction of knowledge and
  interpretation of reality is equally valid
• The powerful have to take on the view from below
• The intended outcome is change
• -- Wright, as earlier slide

37
Where do we go from here ?dialogue II

• There is a danger of excessive optimism about
  the possibility of fruitful dialogue.
  Conversations frequently collapse Dialogue
  tends to become an abstraction which elides the
  constraints of power, the voluntary and imposed
  suppressions which conversation can be said to
  enact. Most human exchanges exist in a middle
  ground of mediation, appropriation, slippage,
  which has the effect of massaging, or indeed,
  occluding otherness. We take away from most
  conversations what we want to take away
• -- Charles Martindale, Redeeming the Text
• Latin poetry and the hermeneutics of reception,
  pp.33 and 106

38
Where do we go from here ?brokerage

• an intentional act in which the broker seeks
  to work in collaborative and creative ways with
  people, ideas , knowledge and resources to
  develop or change something
• Needs infrastructures/processes to support it
• People and organisations in negotiation and
  debate
• -- Norman Jackson, Engaging and changing
• higher education through brokerage, forthcoming

39
Where do we go from here ?radically different
models

• Wright, p.8, after Nelson and Wright
• Participatory development
• Wright, p.8/9, after Engestrom
• Activity Theory
• Wright, p.9, after Senge/Martin
• Learning Organisations
• It is not enough to change strategies,
  structures and systems, unless the thinking that
  produced those strategies, structures and systems
  also changes.

40
Where do we go from here ? measuring management
success

• Assuring standards and quality ?
• Enhancing standards and quality ?
• Avoidance vs engagement
• Cost vs benefit
• Engaging students for real
• Maintaining professional credibility
• Maintaining professional self-respect

41
Describing quality assurance III

• The dancer, by this time, has turned her back.
• He is the more intelligent by far.
• Facing each other rather desperately --
• his eye is like a star --
• we stare and say, Well, we have come this far