Assessing student learning in diverse ways: Portfolios

1
Assessing student learning in diverse ways
Portfolios

• Rosalind Duhs
• Centre for the Advancement of Learning and
  Teaching (CALT)

This document is licensed under the
Attribution-NonCommercial-ShareAlike 2.0 UK
England Wales license, available at
http//creativecommons.org/licenses/by-nc-sa/2.0/u
k/.
2
Introductions

• Name
• Role
• Discipline
• Interest in portfolios for assessment
• Any concerns?

3
Session intended learning outcomesBy the end of
the session, participants will be able to

1. Plan portfolio assessment in line with intended
   learning outcomes and course learning and
   teaching activities to enhance higher order
   learning
2. Evaluate the potential impact of portfolios on
   student learning
3. Outline a plan of action how could your students
   work with portfolios?

4
Portfolios an introduction
5
What are portfolios?

• A portfolio can be viewed as a collection of
  papers and other forms of evidence that learning
  has taken place.
• From
• http//www.medev.ac.uk/resources/resources/feature
  s/AMEE_summaries

6
What might portfolios contain? (Baume 2001)

• A selection of work to evidence the attainment of
  learning outcomes (word-based, images, films,
  sound, webpages)
• Engineering analyses and designs
• Social sciences reports and essays
• Scientists lab reports
• Reflection on group work, work placement,
  clinical practice, projects
• Synoptic portfolios a review of achievement and
  learning throughout a students programme of study

7
Portfolios the essential ingredient?

• An analysis and evaluation of the content in
  relation to the intended learning outcomes of a
  programme, course or module
• - Students explain how the portfolio provides
  evidence of their learning, referring to the
  contents

8
PORTFOLIOS Summative and Formative assessment

• Summative assessment counts towards final results
  in relation to learning outcomes
• Formative assessment does not count towards final
  course grades, but measures progress and provides
  students with valuable feedback
• Summative assessment should also be formative

9
Portfolios can build on a dialogic feedback
system embedding drafting and redrafting
Rosalind Duhs 2010
10
What can portfolios do? (Baume 2001)Summary

• support the development, demonstration and valid
  assessment of a wide range of personal,
  professional and academic capabilities, both
  inside and outside a programme of study
  (Personal Development Planning PDP)
• provide evidence of work done and learning
  achieved
• show reflection on and analysis of evidence and
  learning
• support the integration of learning from
  different parts of the course and beyond.

11
Outcome 1

• Plan portfolio assessment in line with intended
  learning outcomes and course learning and
  teaching activities to enhance higher order
  learning

12
Planning aligned assessment methods
Learning outcomes
Teaching and Learning Activities
Assessment methods
Think and act like a biologist, historian, or
computer scientist, etc.
Act like a biologist, historian or computer
scientist
Learn to act like a biologist, historian or
computer scientist
Learning outcomes, learning activities and
assessment are tightly linked.
13
What is learning/higher order learning (HOL)?
ADOPTIVE LEARNING Knowledge transmitted ADAPTIVE LEARNING Knowledge created HOL
Knowlege and practice of Formation and generation of
Facts, Assertions, Rules and Laws Personal Interpretation and Meaning
Terminology, Language and Protocols Evaluation and Decisions
Techniques and Procedures Arguments, Reasoning and Justification
Organisation and Structure Synthesis and Conceptualisation
Established Principles and Relationships Originality, Creativity and Innovation
From http//www2.warwick.ac.uk/services/cap/resou
rces/pubs/eguides/eskills/guidelines/higher/
14
Aim higher order learning

• Base assessment tasks on using rather than
  replicating knowledge, e.g. assess through
  problem-solving which requires knowledge-based
  analysis and judgement
• Assess in varied ways and include tasks (eg
  Modified Essay Questions - MEQs, projects) which
  require deep engagement and relate to real-life
  roles and competencies

15
Embed feedback and promote student understanding
of how work is assessed

Figure Berry ODonovan Chris Rust ASKe Centre
for Excellence in Teaching and Learning,
(Assessment Standards Knowledge exchange) Oxford
Brookes University
16
Outcome 2

• Evaluate the potential impact of portfolios on
  student learning

17
A systematic review of the use of portfolios (in
Burns 2008)

• Available evidence demonstrates that portfolios
  can support both the learning and assessment of
  general competencies ...
• Success factors included concurrent use in both
  formative promotion of learning and summative
  assessment (mentor)
• Summative assessment of the portfolio was
  important in ensuring portfolio learning
  maintained its status alongside other assessed
  material.

18
E-portfolios flexibility (Woodward 2004)Digital
portfolios Fact or fashion?

• One of the inherent dangers with digital
  portfolios, for example, is that the
  technological novelty of the product could
  overshadow the purpose of the portfolio. The
  danger is that learning to use the technology
  itself could then subsume the learning
  opportunities of portfolio construction.
• There is strong evidence from this research that
  digital portfolios need to be developed within a
  carefully designed framework in the same way the
  paper-based portfolios have been developed

19
E-portfolios flexibility (Woodward 2004)Digital
portfolios Fact or fashion? Cont

• the combination of text, audio, graphic and
  video based representation of information
  collectively termed multimedia ? student
  engagement in learning
• Ownership of author and user hyper-linked
  portfolios offer choice to their audience
  (p.230)
• Hartnell-Young and Morris, however, caution that
  a multimedia portfolio is not expected to be a
  graphic designers dream, the emphasis should be
  on learning (1999, p. 28).

20
E-portfolios comparison three universities
(Wilhelm 2006 p.70)

• Implementing e-portfolios helps to develop a
  culture of evidence (Barrett Wilkerson, 2004)
  for ongoing program improvement. E-portfolios are
  a useful assessment tool in this process.

21
Portfolios reflection (Jones 2010)

• reflection is a process of critically
  examining ones present and past practices as a
  means of building ones knowledge and
  understanding in order to improve practice.
• a more comprehensive understanding of
  reflection was evident and many reported that
  articulating their personal theory (often for the
  first time in their career) impacted positively
  on their ability to reflect on practice.

22
Portfolio assessment and interview

• The effectiveness of the use of a portfolio is
  enhanced by combining portfolio assessment with
  interview.
• a single-examiner portfolio interview focusing
  on standardised questions and a global rating
  scale is a feasible portfolio assessment method
  that can be used to assess clinical reasoning
  skills in an integrated, professionally authentic
  manner.
• Burch and Seggie (2008)

23
Outcome 3

• Outline a plan of action how could your students
  work with portfolios?

24
How to do portfolio assessment practical steps

• Devise assessment tasks which give students the
  opportunity to show that they have achieved
  intended learning outcomes
• Build in choice and drafting and redrafting with
  self- and peer assessment
• Set definite time limits for oral or filmed work,
  number of images and strict word limits for
  written work stress quality above quantity.

25
How to do portfolio assessment criteria

• Write criteria for the assessment of the
  portfolio and give students the opportunity to
  mark each others work applying the criteria
• Ensure that students know how marks will be
  awarded for each section of the portfolio and
  each aspect of the work

26
Portfolio assessment an example from MSc
oncology, UCL

• Presentation of a Portfolio of work developed
  through the year
• The portfolio will consist of 6 written pieces
  covering all six modules
• a. The Holistic Care assignment (details of which
  can be found later in this document) plus either
• b. 4 case studies and a critical review or
• c. 3 case studies, a critical review and an audit
  report.

27
Positive outcomes for the student experience

• The necessary acts of production, selection,
• critical judgement and reflection are, I believe,
• profoundly educational and developmental.
• (Baume 2001 p.11)
• Students value portfolios, as a
• tangible outcome from and demonstration of
• their learning (Baume 2001 p.19).

28
References

• Baume, D. (2001). A briefing on assessment of
  portfolios Electronic Version. Learning and
  Teaching Support Network Generic Series
  Assessment 6 from http//www.bioscience.heacademy
  .ac.uk/ftp/Resources/gc/assess06portfolios.pdf
• Biggs, J. (2003). Teaching for Quality Learning
  at University. 2nd ed. Buckingham The Society
  for Research into Higher Education Open
  University Press.
• Burch VC, Seggie JL. (2008) Use of a structured
  interview to assess portfolio-based learning.
  Medical Education, Vol. 42894-900
• Jones, E. (2010). Personal theory and reflection
  in a professional practice portfolio Electronic
  Version. Assessment Evaluation in Higher
  Education, 35, 699-710 from http//dx.doi.org/10.1
  080/02602930902977731
• Klenowski, V., Askew, S., Carnell, E. (2006).
  Portfolios for learning, assessment and
  professional development in higher education.
  Assessment Evaluation in Higher Education,
  31(3), 267-286.
• Mentowski, M. and Associates (2000). Learning
  that lasts integrating learning development, and
  performance in college and beyond. San
  Francisco Jossey-Bass
• ODonovan, B., Price, M., and Rust, C. (2004)
  Know what I mean? Enhancing student understanding
  of assessment standards and criteria. Teaching in
  Higher Education, Vol. 9, No. 3.
• Rees, C. and Sheard, C. (2002) The reliability of
  assessment criteria for undergraduate medical
  students' communication skills portfolios the
  Nottingham experience. Medical Education, Vol.
  38, No. 2 138-144
• Wilhelm, L., Puckett, K., Beisser, S., Wishart,
  W., Merideth, E., Sivakumaran, T. (2006).
  Lessons Learned from the Implementation of
  Electronic Portfolios at Three Universities
  Electronic Version. Tech Trends, 50 from
  http//www.springerlink.com/content/l3412700x44l47
  52/fulltext.pdf
• Woodward, H., Nanlohy, P. (2004). Digital
  portfolios fact or fashion? Electronic
  Version. Assessment Evaluation in Higher
  Education, 29, 227-238 from http//dx.doi.org/10.1
  080/0260293042000188492

29
Links from LTSS e-portfolioshttp//www.ucl.ac.uk
/ltss-blog/?cat74

• http//www.ascilite.org.au/conferences/auckland09/
  procs/joyes.pdf
• http//www.jiscinfonet.ac.uk/infokits/e-portfolios
  /index_html