Electronic Portfolios for Formative and Summative Assessment

1
Electronic Portfolios for Formative and Summative
Assessment
Simon Cotterill School of Medical Education
Development University of Newcastle upon Tyne, UK
2

• The workshop draws on the experiences of
  developing, implementing and evaluating Web-based
  ePortfolios, at the University of Newcastle, UK.
• Objectives
• To discuss some of the principles of electronic
  portfolios
• To gain familiarity with the potential
  educational and technical
• value-added features of using electronic media.
  
• To consider ePortfolio approaches for assessment
• Proposed Structure
• Presentation Principles of ePortfolios
• Exercise Strengths and weaknesses
• Presentation Case study FDTL4 ePortfolio
  project
• Discussion

3
Introduction What are Portfolios ?

• Unstructured
• Formative
• Sample
• Best work
• Factual / Quantitative
• Learner Owned

Structured Summative All work Representative R
eflective / Interpretive Employer Owned
4
Introduction What are Portfolios?

• A collection of work or evidence on performance
• Processes (the journey)
• Learning Outcomes (the destination)
• Record volume and variety of experiences
• Essays, project work, logs of experiences/achievem
  ents, artwork, records of accomplished work,
  audits etc.
• Portfolios are defined by their PURPOSE(s)
• Employment Builder gets commissions on the basis
  of his record of satisfied customers
• Portfolio for application The researcher
  includes a personal profile in a grant
  application to show experience
• Professional requirements Med/Law/Education
  -evidence of essential experience and competency
  that can be audited

5
What might an Electronic Portfolio be?

• A computer application which stores the users
  information on a local drive.
• A static Web page eg. CV on the Internet
• A portfolio composed using simple tools (e.g.
  Word, Dreamwaever, Powerpoint etc.)
• A sophisticated database-driven system, typically
  accessed over a network.

6
3D Model
Overview (Planning / PDP)
Content (Evidence)
Discussion
Model presented by Jan van Tartwijk (Utrecht
University) Presented at the ALT-SURF Spring
Conference Research Seminar, Edinburgh 2004
7
Principles of Electronic Portfolios
1. Should bring significant advantages over
paper-based alternatives 2. Should be considered
in the context of wider (human) processes 3.
Clarity of purpose(s) 4. One size does not fit
all 5. Should be learner-centric 6. Should be an
integral part of the learning experience 7.
Should support life-long learning 8. Research /
evaluation is essential
.not an exhaustive list !
8
10 value added features of an online approach
to portfolios

• Highly customisable
• Multi-purpose eg. formative summative
  reduced duplication
• Multiple structures / views
• Easier cross-referencing
• Sharable / facilitate interaction
• Transportable
• Searchable
• Reduced admin
• Secure access from a range of locations
• Not left on the bus !

9
Flexibility for the Learner to Add, Link
Cross-Reference ePortfolio Artefacts
Adding child objects to an artefact
Cross-reference An artefact with A skill
10
Potential Weaknesses of ePortfolios

• There may be barriers to effective use of
  electronic portfolios e.g.

• Limited access to computers / Internet
• Varied IT experience Technophobia !
• Reliability of IT systems
• Possible duplication / overlap with paper systems
• Research / Evaluation is essential
• Optimisation will take a number of years

11
Humans are important too !

• Need buy-in of key stakeholders
• Learners
• Clarity of purpose ?
• Sense of Ownership ?
• Tutors Admin staff
• Extra work ?
• Senior Curriculum staff
• Employers / Professional bodies
• Integration with existing workflow and/or change
  in practice and culture ?
• Training and support (all)
• Monitoring and evaluation

12
Clarity of purpose(s)

• Choose / design your portfolio according to
  purpose
• Learners and assessors also need to be clear
  about purpose and requirements (motivation /
  fairness)
• ePortfolios can readily support multiple
  purposes
• Portfolio assessment

13
Summative and Formative Purposes

• Portfolios are often used to evidence the
  achievement of learning outcomes for summative
  assessment.
• There may be potential problems if the portfolio
  serves both formative and summative processes.
• Reflection is less likely to be open and honest
  if the learner knows that the work will be
  assessed (loss of authenticity).
• The production of a portfolio can itself be a
  formative learning process ie. it is as much a
  journey as an end-point for assessment.

14
Summative and Formative Purposes (2)

• The use of electronic portfolios may potentially
  reduce the tension between formative and
  summative processes by supporting both assessed
  and private / non-assessed content.
• Students have the choice to select which content
  is private and which is made available to
  assessors, appraisers, tutors, peers, and others
  involved in their education.

15
Supporting Multiple Purposes
Portfolio for Presentation
Portfolio for Assessment
Portfolio for Application (job / promotion)
Portfolio for Accreditation/ Revalidation
Learners repository
Institutional Data
Portfolio for Appraisal
PDP (shared)
PDP / Reflective (private)
16
Portfolio Assessment

• Where to begin ?

Baume D. A Briefing on the Assessment of
Portfolios. LTSN Generic Centre. Assessment
Series No.6 (2001) Friedman M, Davis MH, Harden
RM, Howie PW, Ker J, Pippard MJ AMEE Medical
Education Guide No. 24 Portfolios as a method of
student assessment. Medical Teacher. 2001
23535-51

• Key issues
• Validity
• Reliability
• Attitudes behaviours
• Time demands

17
ePortfolios and Assessment

• -Potential benefits of ePortfolios
• Formative input during portfolio building (not
  just at end)
• Random sampling drilling down
• Peer Assessment (formative and summative)
• Automatic inclusion of assessment results
  feedback
• Automated summary information (quantitative)
• Reduced monitoring / admin demands
• -Piloting essential before use in high-stakes
  assessment
• -Need for further research in this field !

18
Exercise Strengths and weaknesses
19
ePortfolios should be learner-centric

• Personalisation / customisation
• Sense of ownership / privacy / sharing
• Constructivist theories.
• is eLearning contributing to Paradigm shift ?
• Student Independent Learner
• Teacher Facilitator
• Didactic Interactive Explorative
• 1 location Distance learning
• 1 provider Multiple providers

20
ePortfolios should be an integral part of the
learning experience

• Educational (integration with the curriculum)
• Technical (integration within the MLE)

21

• Should not be perceived as a bolt-on
• Support a holistic view of curriculum / outcomes
  (as well as specific components)

• ePortfolio integrated with the VLE
• Tool properties include module code
• dynamic links to Study Guides resources
• Learning outcomes linked to curriculum
  databases modules and units
• Links to other systems eg. SSC selection

22
Evaluation and Refinement
A good fit happens over time with lots of
use Spandel, 1997 cited by Jan van Tartwijk
(ALT-SURF Conference, 2004)

• Action Research approaches
• Incremental development and refinement
• Qualitative research methodologies
• (some quantitative - not mutually exclusive!)
• Ethical and practical considerations

23
Research and Evaluation

• Does the ePortfolio improve learning ? How so ?
• Some evidence that PDP improves learning (more
  needed)
• A systematic map and synthesis review of the
  effectiveness of personal Development Planning
  for improving student learning. EPI Centre 2003
  http//eppi.ioe.ac.uk
• Can ePortfolios foster a reflective approach to
  learning ?
• How defined ? How measured ?
• How do particular groups of learners approach
  reflective portfolios ?
• Learning Styles
• Gender and Age
• Cultural Background
• John Mole. The geography of thinking. Clin Med
  2002 2343-5
• Does the technology change the learning process ?
• What are learner and staff perceptions /
  attitudes towards ePortfolios ?