The effect of eportfolios on reflective learning and metacogntion

1
The effect of e-portfolios on reflective learning
and metacogntion

• University of Bradford
• Paul Rodway (Centre for Psychology Studies)?
• Astrid Schepman (Centre for Psychology Studies)?
• Peter Hartley (Teaching Quality Enhancement
  Group)?

2
Acknowledgements

• Research funded by HEA/JISC-funded Distributed
  E-Learning II (DeL II)?
• Research Assistance Rebecca Durrans, University
  of Bradford

3
Research context

• Psychology of Learning and Study (First year
  psychology module)
• Academic skills, Personal Development Planning,
  Key concepts from educational psychology.
• Assessment e-portfolio 3500 words
• PebblePad
• 5 sections PDP, CVs, Working with Others,
  Skills, Metacognition

4
Research questions

• Metacognition
• Knowledge and awareness of your own cognitive
  processes and the ability to control and manage
  those processes (Flavell, 1976).
• Two components Knowledge of own cognition and
  the ability to control and execute cognitive
  strategies.
• 1. Can the completion of an e-portfolio ( the
  module) enhance metacognitive knowledge?
• 2. Can reflection enhance the completion of an
  academic task that loads on metacognitive
  strategies?

5
Metacognition

• Introduced to the concept in Psychology of
  Learning and Study
• Week 3 semester 1
• Students told usual finding is overconfidence
• Taught more fully at the end of semester 2.

6
Class experiment An objective measure of
metacognitive knowledge

• Participants Psychology Students and a Control
  Group
• Task Four sets of questions, 20 in total, 75
  sec. per set
• Sets number estimation, number series, word
  definitions, word relations
• Estimation before task How many out of 20 will
  you get correct?
• Actual performance
• Estimation after task Now that you have done the
  test, how many did you get correct?
• Objective measure of students metacognitive
  knowledge

7
Assessed twice during the module

• Phase 1 Week 4 of semester 1
• A week after a lecture introducing metacognition
• Phase 2 and 3 Weeks 7 12 of semester 2
• At data processing Phase 2 and 3 collapsed due
  to small N in Phase 3.

8
Findings

• blue lines Phase 1 pink lines Phase 2
• significantly different from adjacent point
• N 8 control Phase 1 N 9 control Phase 2
• N 35 psych Phase 1 N 19 psych Phase 2

9
Main Preliminary Findings

• Metacognition teaching reverses overconfidence on
  post-test estimate
• May enhance callibration of metacognitive
  knowledge
• However, the effect did not last
• Psychology and control similar in Phase 2

10
Other measures

• Metacognitive Awareness Inventory (Schraw
  Dennison, 1994)?
• Self Efficacy for Learning, Writing and Test
  subscales (Zimmerman et al., 2005)
• Generalised Self-Efficacy (Schwarzer Jerusalem
  1995)
• Portfolio grades (higher more sophisticated the
  reflections?)?
• Currently examining in relation to metacognitive
  performance.

11
Student feedback on E-Portfolio 1-5, 5
strongly agree, N 26

• Students subjective experience of the eportfolio
  was positive
• E-portfolio needs to be used in appropriate
  teaching context

12
Study 2 Reflection Experiment

• E-portfolios encourage reflection on academic
  learning.
• Does reflection enhance performance on a task
  that requires the application of metacognitive
  strategies?
• Do different types of reflection have different
  effects?

13
Reflection experiment Five elements test

• Five elements test based on neuropsychological
  test (Shallice Burgess, 1991) of executive
  functioning.
• Five elements
• Word definitions, Proofreading, Maths, Visual
  Inspection, Memory (list to be recalled
  afterwards)?
• Each allocated different points plus penalty
  points
• Memory weighted at 40 (similar to an exam in a
  module investment of time in future payoff)?
• To do the task well have to plan, schedule, and
  be strategic use metacognitive strategies

14
Reflection Experiment

• 60 participants
• 3 reflection conditions (20 participants per
  condition)
• 1. Control write about holiday, news, or sports.
  
• 2. Unstructured reflection how they thought they
  did and how they might be able to improve.

15

• 3. Structured reflection
• 1. Do you think you performed on the tasks as
  well as you could have?
• 2. What determined the order in which you did the
  tasks?
• 3. How did you decide on the time to allocate to
  each task?
• 4. To what extent did you feel motivated to
  complete the tasks successfully?
• 5. If you had a strategy do you think it was
  optimal?
• 6. Did your emotions affect how you did the
  tasks?
• 7. Can you think of anything else that might
  improve your performance?

16
Effect of reflection on performance

• Better performance second time than first time.
• Non-significant for control
• Significant for both structured and unstructured
  reflection
• Detailed analysis biggest gain from recall task
  (in reflection conditions)?
• Reflection enhanced performance

First time vs. second time
17
Findings

• Reflection enhances performance on a task that
  loads on metacognitive strategies.
• Reflection may improve performance by allowing
  the application of metacognitive strategies.
• Unstructured reflection was as effective as
  structured reflection.
• Has implications for e-portfolio design.
• But may depend on task

18
Summary

• E-portfolio viewed positively.
• Metacognitive knowledge can be influenced by
  teaching and the completion of an e-portfolio.
• Structured and unstructured reflection both
  appear to improve the application of
  metacognitive strategies.
• The Five Element Test is a useful and rapid way
  to examine the effects of reflection on
  metacognitive strategies.