The Effect of Generation on Robust Learning

1
The Effect of Generation on Robust Learning

• Robert G.M. Hausmann
• Kurt VanLehn

2
Placing Study within Robust Learning Framework

• Primary research cluster
• Interactive communication, coordinative
  learning, refinement fluency
• Independent variable(s)
• IC Reflective dialog, scripting collaboration,
  peer tutoring, peer observation of tutoring,
• CL self-explanation vs. paraphrase, complete vs.
  incomplete worked-out examples,
• RF Feature focusing, example comparison,
  cognitive mastery, optimal spacing,
• Dependent variables used
• Normal post-test, long term retention, transfer,
  future learning

3
Outline
Next

• Introduction
• Objectives
• Background
• Research problem
• Study
• Interpretation

4
Background

• Self-explanation is a sense-making process that
  generates inferences to adapt material to
  background knowledge (Chi, 2000)

Passive Active Interactive
Explicit(general) Description Self-explanation Self-explanation with feedback
Implicit(instance) Example Practice Tutored practice
5
Background

• Self-explanation is a sense-making process that
  generates inferences to adapt material to
  background knowledge (Chi, 2000)
• Paraphrasing aids the acquisition of shallow
  knowledge (Hausmann Chi, 2002 VanLehn Jones,
  1995)

Passive Active Interactive
Explicit(general) Description Paraphrasing Self-explanation with feedback
Implicit(instance) Example Practice Tutored practice
6
Research problem

• Problem
• Does self-explanation work in the classroom, even
  when controlling for active learning processes?
• Hypothesis
• Coverage Hypothesis
• If the information is presented, and the student
  actively processes it, then they will exhibit
  robust learning.
• Generation Hypothesis
• If the information is generated by the student,
  then they will exhibit robust learning.

7
Outline

• Introduction
• Objectives
• Study
• Method conditions
• Method procedure
• Results
• Interpretation

Next
8
Treatment Self-explanation by reference

• Design
• Participants
• Physics LearnLab
• USNA Physics II (N104)
• Materials
• Study instructions
• Video solutions to study
• Andes homework system
• Chapter exam (1 question)
• Homework
• Study domain electrodynamics
• AFL domain magnetism

Para-phrase Self-explain
Complete Example n 26 n 27
Incomplete Example n 23 n 28
9
Independent Variable Study Strategy Instructions

• Before students watched the videos, they received
  the following instructions
• We would like you to watch and listen to each
  step and then explain what it means to
  you/paraphrase in your own words. That is

• Self-Explaining Instructions
• What new information does each step provide for
  you?
• How does it relate to what you've already seen?
• Does it give you a new insight into your
  understanding of how to solve the problems?
• Does it raise a question in your mind?

• Paraphrasing Instructions
• How can I put this in my own words?
• How can I say this in a different way?
• In other words _________________.
• What's another way that I could put this?

• Tell us whatever is going through your mind -
  even if it seems unimportant.

10
Whats Wrong Help
Next Step Hint
Error (flag feedback)
Dependent Variable Assistance Score
?(hintserrors)/steps
Bottom-out hint
11
Method Procedure
Warm-upProblem
Problem1
Problem2
Problem3
12
Summary of study design

• Independent variables
• Study Strategy self-explain or paraphrase
• Example Type complete or incomplete
• Control for active learning!
• Dependent variables
• Normal post-test interspersed problem solving
• Retention non-isomorphic chapter exam problem
• Transfer test homework problems
  (electrodynamics)
• Acc. Future Learning (AFL) homework problem
  (magnetism)

13
Results Normal Post-test
Isomorphic problem solving during laboratory
period (electrodynamics)
ME Study Strategy (paraphrase gt self-explain)
14
Results Retention
Non-isomorphic exam problem one week later
(electrodynamics)
15
Results Acc. Future Learning
Andes homework performance six weeks later
(magnetism)
ME Study Strategy (paraphrase gt self-explain)
16
Summary of results

• Normal learning
• Study Strategy SE lt Paraphrase
• Example Type Complete Incomplete
• Robust learning
• Retention CSE CPP
• Transfer items SE lt/ Paraphrase
• Acc. Future Learning SE lt Paraphrase

17
Outline

• Introduction
• Objectives
• Study
• Interpretation
• Micro-level
• Macro-level

Next
18
Macro-level interpretation

• Self-explanation works because
• Self-explaining is a kind of personalized
  monologue, which is a kind of sense-making
  process that yields transfer.
• Self-explanation is fundamentally a
  self-supervised learning pathway because it is
  generated by the individual student.

Original Domain electrodynamics
New Domain magnetism
Self- Explanation
Deep Feature Acquisition
Analogous deep Knowledge components
construct
19
Macro-level explanation
Self-explanation provides a mechanism to
construct infer-ences, resulting in better KCs,
which then improves foundational skills robust
learning
Robust Learning
Outcomes Knowledge, reasoning learning
processes
Foundational Skills
Sense-Making
Learning Processes
Construction, elaboration, discrimination
Refinement of Features
Streng-thening
Co-Training
Instructional Processes(independent variables
or treatments)
Multiple inputs, representations, strategies
Feedback, example variability, authenticity
Tutorial dialogue, peer collaboration
self-explanation
Schedules, part training
20
Which of the 8 main paths probably explain the
results?

• Compared to paraphrasing, self-explanation
  probably increased the number of correct, deep
  KCs
• These raised scores on transfer items and the
  explanation parts of the normal items.

Sense- making outcomes More correct conceptual KCs Deeper KCs facilitate rederivation Deeper KCs facilitate adaptation Deeper KCs ? learning by oneself
Foundational skill outcomes More correct skill KCs Stronger KCs More general KCs Stronger KCs leave cognitive headroom

Measurable outcomes Higher normal test scores Longer retention Further transfer Better future learning
21
Acknowledgements

• Sponsors

• Instructors
• Donald J. Treacy, USNA
• Robert N. Shelby, USNA
• Programming
• Brett van de Sande, LRDC
• Anders Weinstein, LRDC
• Database Assistance
• Min Chi, LRDC
• Michael A. Ringenberg, LRDC