Assessment Standards: A Manifesto for Change

1
Assessment StandardsA Manifesto for Change

• Dr Chris Rust, Deputy Director
• ASKe CETL Directorate
• Margaret Price, Jude Carroll, Berry ODonovan and
  Chris Rust

2
Origin

• Weston Manor Group, November 07
• 40 National and International Experts in
  Assessment
• Triggered by the Burgess Report, NSS Results
• Two days of discussions
• Outcome
• Six tenet manifesto for change to assessment
  practice related to standards
• Sent to HEFCE, HEA, QAA, UUK, GuildHE, NUS
• Lead article in THES April 24, 08

3
Why assessment ? Assessment a key driver of
student learning

• Assessment is at the heart of the student
  experience
• (Brown, S Knight, P., 1994)
• From our students point of view, assessment
  always defines the actual curriculum
• (Ramsden, P.,1992)
• Assessment defines what students regard as
  important, how they spend their time and how they
  come to see themselves as students and then as
  graduates.........If you want to change student
  learning then change the methods of assessment
• (Brown, G et al, 1997)

4
But

• QAA subject reviews
• National Student Satisfaction Survey
• the Achilles heel of quality (Knight 2002a, p.
  107)
• Summative assessment practices in disarray
  (Knight 2002b, p. 275
• Broken (Race 2003, p. 5)
• There is considerable scope for professional
  development in the area of assessment (Yorke et
  al, 2000, p7)
• Rising concern about cheating and plagiarism

5
Why change is needed (1)

• The types of assessment we currently use do not
  promote conceptual understanding and do not
  encourage a deep approach to learningOur means
  of assessing them seems to do little to encourage
  them to adopt anything other than a strategic or
  mechanical approach to their studies.
• (Newstead 2002, p3)
• Even when lecturers say that they want students
  to be creative and thoughtful, students often
  recognise that what is really necessary, or at
  least what is sufficient, is to memorise
• (Gibbs, 1992, p. 10)
• Many research findings indicate a declining use
  of deep and contextual approaches to study as
  students progress through their degree
  programmes
• (Watkins Hattie, 1985 Kember et al,
  1997 Richardson, 2000 Zhang Watkins, 2001)

6
Tenet 1

• The debate on standards needs to focus on how
  high standards of learning can be achieved
  through assessment. This requires a greater
  emphasis on assessment for learning rather than
  assessment of learning.

7
Why change is needed (2)

• Our current systems focused on marks and grades
  arent working
• Belief that it is possible to distinguish the
  quality of work to a precision of one percentage
  point (Elander Hardman, 2002)
• Belief that double-marking will ensure fairness
  and reliability (Laming (1990)
• Belief that consistency can be achieved through
  conformity, and simple numerical rules (e.g.
  level 1 essay 3,000 words, level 3 essay 5,000
  or no more than two pieces of assessment per
  module)
• The combination of scores, which obscures the
  different types of learning outcome represented
  by the separate scores
• The distortion of marks by the type of assessment
  (e.g. coursework c.f. examination) and the actual
  subject discipline/s studied (Yorke et al, 2002
  Bridges et al, 2002)
• The distortion of resulting degree
  classifications by the application of
  idiosyncratic institutional rules (e.g. Armstrong
  et al, 1998)
• (Rust, 2007)

8
Why change is needed (2) contd.

• This quest for reliability tends to skew
  assessment towards the assessment of simple and
  unambiguous achievements, and considerations of
  cost add to the skew away from judgements of
  complex learning
• (Knight 2002 p278)
• students become more interested in the mark
  and less interested in the subject over the
  course of their studies. (Newstead 2002, p2)
• summative judgement itself is the problem
• (Burgess, 2007, p. 8)

9
Tenet 2

• When it comes to the assessment of learning, we
  need to move beyond systems focused on marks and
  grades towards the valid assessment of the
  achievement of intended programme outcomes.

10
Why change is needed (3)Some aspects of
quality cannot be communicated through explicit
criteria alone

• Regulative and logical criteria standards can be
  defined in terms of well-defined outcomes
  (Sadler, 1987, p. 70)
• Prescriptive and constitutive criteria refer to
  matters of degree and It would be difficult or
  impossible to guess the educational level at
  which they are applicable (Sadler, 1987,
  p. 70)
• Such types of criteria are often interdependent
  and can only be assessed using holistic/profession
  al judgement (Sadler, 2008)
• Such criteria are socially constructed requiring
  the sharing of tacit knowledge over time
  (ODonovan et al, 2004
  Rust et al, 2005)

11
Limitations of explicit articulation

• Meaningful understanding of standards requires
  both tacit and explicit knowledge
• (ODonovan et al. 2004)
• we can know more than we can tell
  (Polanyi, reprinted 1998, p.136).
• Verbal level descriptors are inevitably fuzzy
  (Sadler 1987)
• There is a cost (in terms of time and resources)
  to codifying knowledge which increases the more
  diverse an audiences experience and language
  (Snowdon, 2002).

12
Tenet 3

• Limits to the extent that standards can be
  articulated explicitly must be recognised since
  ever more detailed specificity and striving for
  reliability, all too frequently, diminish the
  learning experience and threaten its validity.
  There are important benefits of higher education
  which are not amenable either to the precise
  specification of standards or to objective
  assessment.

13
Why change is needed (4)Assessment standards
applied to high-level complex learning can only
be understood through active engagement with
members of a disciplinary community

• making sense of the world is a social and
  collaborative activity (Vygotsky, 1978)
• Tacit knowledge is experience-based and can only
  be revealed through the sharing of experience
  socialisation processes involving observation,
  imitation and practice (Nonaka, 1991)
• An indispensable condition for improvement in
  student learning is that the student comes to
  hold a concept of quality roughly similar to that
  held by the teacher (Sadler, 1989)
• Passive receipt of feedback has little effect on
  future performance (Fritz, et al., 2000)
• Dialogue and participatory relationships are key
  elements of engaging students with assessment
  feedback (ESwAF FDTL, 2007)

14
Why change is needed (4) contd.

• The most significant factor in student academic
  success is student involvement fostered by
  student/staff interactions and student/student
  interactions
• (Astin, 1997)
• The only common factor in a study of departments
  deemed excellent in both research and learning
  and teaching is high levels of student
  involvement
• (Gibbs, 2007)
• participation, as a way of learning, enables the
  student to both absorb, and be absorbed in the
  culture of practice
• (Elwood Klenowski, 2002, p. 246)

15
Tenet 4

• Assessment standards are socially constructed
  so there must be a greater emphasis on assessment
  and feedback processes that actively engage both
  staff and students in dialogue about standards.
  It is when learners share an understanding of
  academic and professional standards in an
  atmosphere of mutual trust that learning works
  best.

16
Why change is needed (5)
Important aspects of complex, high-level learning
outcomes can only be achieved when students are
allowed time to come to know the standards in
use by the community.

• Slowly learnt academic literacies require
  rehearsal and practice throughout a programme
  (Knight Yorke, 2004)
• The achievement of high-level learning requires
  integrated and coherent progression based on
  programme outcomes
• Where there is a greater sense of the holistic
  programme students are likely to achieve higher
  standards than on more fragmented programmes
  (Havnes, p. 2007)
• Students need to engage as interactive partners
  in a learning community, relinquishing the
  passive role of the instructed within processes
  controlled by academic experts (Gibbs et al,
  2004)

17
Tenet 5

• Active engagement with assessment standards
  needs to be an integral and seamless part of
  course design and the learning process in order
  to allow students to develop their own,
  internalised, conceptions of standards and
  monitor and supervise their own learning.

18
Why change is needed (6)

• Changes in higher education (e.g. massification,
  reduced unit of resource, expectations of
  increased productivity in staff) threaten the
  health of disciplinary communities and their
  ability to share and exemplify professional
  judgement.
• There has been slow progress in the
  professionalisation of university teachers
• There has been limited attention paid to
  professional assessment practice
• Reliance on the external examiner system to
  mediate standards within the system is misplaced
  (Newstead and Dennis,1994)
• it cannot be assumed students graduating .
  will have achieved similar standards (QAA, 2007)
• If some aspects of high-level learning can only
  be assessed using professional judgement then we
  need to ensure that judgement is indeed
  professional

19
Tenet 6

• Assessment is largely dependent upon
  professional judgement and confidence in such
  judgement requires the establishment of
  appropriate forums for the development and
  sharing of standards within and between
  disciplinary and professional communities.

20
More.

• For more about the background arguments behind
  the manifesto, go to
• Price, M., ODonovan, B., Rust, C. Carroll, J
  (2008), 'Assessment?standards a manifesto for
  change', Brookes eJournal of Learning and
  Teaching, Vol.2, No. 3. (Online December 2008)
  Available at?
• http//bejlt.brookes.ac.uk/article/assessment_stan
  dards_a_manifesto_for_change
• If you would like to be a personal signatory to
  the manifesto please visit
• http//www.business.brookes.ac.uk/learningandteach
  ing/aske/