Towards Individualized Instruction With TechnologyEnabled Tools and Methods

1
Towards Individualized Instruction With
Technology-Enabled Tools and Methods
Gregory K. W. K. Chung, Girlie C. Delacruz, Gary
B. Dionne, Eva L. Baker, John Lee, Ellen Osmundson
American Educational Research AssociationAnnual
Meeting Chicago, IL - April 9-13, 2007 Symposium
Rebooting the past Leveraging advances in
assessment, instruction, and technology to
individualize instruction and learning
2
Structure of Talk

• Problem
• Pre-algebra/Algebra
• Classroom constraints
• Research
• Domain analysis, assessment design, instructional
  design, screen shots
• Results
• Did it work?

3
Problem
4
Study Context

• Why pre-algebra?
• Pre-algebra provides students with the
  fundamental skills and knowledge that underlie
  algebra
• Pre-algebra ? Algebra ? STEM
• 2001-02 LAUSD 9th grade cohort
• Only 65 of 9th graders (28,000) progress to
  10th grade (23 retained in 9th grade 10,000
  12 leave district)
• Algebra identified as gatekeeper
• In fall 2006, 38 of CSU first-time freshmen
  needed remediation in mathematics (17,300)
• CSU 5-year graduation rate (STEM) 34

5
6th/7th Grade Standards
6
Research
7
Research Questions

• To what extent can students learn from
  instructional parcelsbrief slices of
  instruction and practice?
• To what extent can automated reasoning (i.e.,
  Bayesian networks) be used for automated
  diagnosis of pre-algebra knowledge gaps?
• What is the architecture for a diagnosis and
  remediation system?

8
General Idea
Automated Reasoning(Bayes net)
Pre-algebra pretest
multiplicative identity
adding fractions
distributiveproperty
Individualized instruction and practice
Individualized posttest
9
Research Study Overview

• Domain analysis and assessment design
• Identify the key concepts that underlie
  pre-algebra and the relations among those
  concepts
• Instruction and practice
• Design based on best-of-breed (worked examples,
  schema-based instruction, multimedia learning,
  effective feedback)

10
Domain Analysis
11
Domain Analysis
12
Domain Analysis
13
Sample Assessment Items
DP distributive propertyCP-ADD commutative
property of additionAP-ADD associative
property of addition
CE common errorOPER operation
14
Instructional Design
15
Design Assumptions

• To maximize the chance of learning with only
  brief exposure to content, instruction should
• Direct the learners attention to important
  content
• Highlight and explain the importance of the
  content
• Use lay language and 1st/2nd person voice
• Use worked examples with visual annotations,
  coordinated and complementary narration
• Provide varied examples with different surface
  features but same underlying concept
• Provide practice on applying the concept
• Provide tailored and explanatory feedback

16
Multiplicative Identity - 4
17
Multiplicative Identity - 5
18
Multiplicative Identity - 6
19
Multiplicative Identity - 7
20
Multiplicative Identity - 8
21
Multiplicative Identity - 9
22
Multiplicative Identity - 10
23
Multiplicative Identity - 11
24
Multiplicative Identity 12/12
25
Multiple Examples
Stage 1 What is the next step in solving the
problem?
Stage 2 What is the result of carrying out the
step?
Stage 3 What is the underlying math concept?
26
(No Transcript)
27
Method

• 2 group pretest, posttest design
• 113 middle school students
• Instruction vs. no instruction (stratified by
  concept and high, medium, low knowledge)
• Procedure
• Pretest (84 items, ? .89)
• Instruction on concepts
• 4-6 concepts per student (out of 10)
• Individualized posttest

28
Method

• Analysis
• Examined 6 scales (pretest ? .61 - .75)
• Adding fractions, distributive property,
  transformations, multiplicative identity
• Multiplying and adding fractions, rational number
  equivalence
• Participants dropped an analysis if
• Diagnosed as high knowledge
• Non-compliant (20-50, depending on scale)

29
Results
max 8 n 9/22 p .50 d --
max 8 n 14/18 p .04 d .76
max 6 n 13/21 p .04 d .77
max 7 n 7/21 p .06 d .91
max 6 n 13/12 p .03 d .91
max 6 n 13/12 p .01 d .50
Adding fractions
Distributive property
Transfor-mations
Multiply, add fractions
Rationale number equivalence
Multiplicative identity
30
Student Perceptions

• 46 reported tool the was very useful, 53
  reported they were very willing to use
• Tool not for everyone
• I really understood the way they took step by
  step to show the problem.
• I thought that the video and the practice
  problems were very easy to understand, but if
  they were a little more exciting it would help
  make the process more fun.
• No, it just made me confused. I like seeing
  everything on a board, not computer.

31
Closing Remarks

• Preliminary results promising
• Instruction appears effective, even if brief
• Technical aspects of individualization of
  instruction and assessment tractable
• DITSy Dumb Intelligent Tutoring System
• Limitations and next steps
• One instructional day between pretest and
  posttest unknown retention effect low ? on some
  scales novel assessment format
• Refine measures, replicate, validate diagnosis,
  examine more complex outcomes, examine
  instructional variables

32
Backup
33
Content

• Ten concepts tested
• Properties (e.g., distributive property)
• Problem solving (e.g., multiplying fractions)
• Time per parcel
• Mean time (mmss) 345
• Range (mmss) 205 to 519, 1058

34
Task Sequence
Instruction(worked examples, narration,visual
annotations)
Practice (stage 1)Identify thenext step
Feedback(tailored andexplanatory)
Practice (stage 2) Identify the result
Feedback(tailored and explanatory)
Practice (stage 3)Identify the math concept
Feedback(tailored and explanatory)
35
Feedback is tailored to the specific option
selected.
Incorrect response feedback (a) confirmation
that the response is incorrect, and (b) a brief
explanation of why the response is wrong
Hint on what to think about to solve problem
36
Correct response feedback (a) confirmation that
the response is correct, and (b) a brief
explanation of why the response is correct
37
Dont know response feedback guidance on what
the student should be considering
38
animation/video feedback goal, why, how, and
common errors
39
Timing
40
Some Observations

• Classroom / group instruction inefficient
• A lot of time spent on non-instructional
  activities
• Teacher telling jokes
• Students settling down (pulling out notebook from
  backpack, opening textbook, getting up to sharpen
  pencil)
• Teacher writing the equation on the whiteboard

41
Pretest-Posttest (Instruction)
max 8 n 22 p .02 d .50
max 8 n 18 p .09 d .54
max 6 n 21 p lt .001 d .61
max 7 n 21 p lt .001 d 1.36
max 6 n 11 p .14 d .37
max 6 n 12 p .10 d .43
Adding fractions
Distributive property
Transfor-mations
Multiply, add fractions
Rational number equivalence
Multiplicative identity
42
Pre-algebra Bayes Net
43
Diagnosis and Remediation

• Model the domain of pre-algebra with a Bayesian
  network
• Treats test items as evidence of understanding
• Computes the probability of a student
  understanding a concept
• Short slices of instruction that are focused on a
  single concept
• NOT intended to replace classroom teaching
• Intended to support homework, review, wrap-around
  activities

44
(No Transcript)
45
Multiplicative Identity - 1
46
Multiplicative Identity - 2
47
Multiplicative Identity - 3