Canadian Language and Literacy Research Network Annual Conference' June 18th, 2006

1
Canadian Language and Literacy Research Network
Annual Conference. June 18th, 2006
ABRACADABRA A web-based learning and research
resource for all Canadians Dr Robert Savage and
colleagues Dr Philip C. Abrami Geoffrey Hipps
and the CSLP team across Canada
2
The central importance of interventions

• The philosophers have only described the world,
  in various ways, the point is to change it.
• (Karl Marx, Theses on Feuerbach, 1886.)
• Philosophers have only described the world in
  various ways, the point is to improve it (me,
  today)
• Theories of reading ultimately rise and fall on
  the basis of whether implementations based on
  asserted principles work in practice
• A pre-requisite to genuinely inclusive education
• The rise of evidence-based practice

3
Evidence-Based Approaches Necessary but not
Sufficient

• No matter how great an intervention sounds,
  no matter how much it costs, no matter how much
  research has been published, and no matter how
  many criteria or belief systems it satisfies, if
  the intervention does not change the childs
  trajectory, then it is not effective for that
  child and a change is indicated.
• Good, Simmons, Smith (1998)

4
Evidence-based practice. One example The
Campbell Collaboration

• Sibling organisation to the Cochrane
  Collaboration - has a similar purpose in
• relation to interventions in health care.
• Named after the American Psychologist and thinker
  Donald Campbell who urged societies to assess
  more rigorously the effects of their social and
  educational experiments (or policies and
  practices, if you prefer).
• International review groups, purpose to prepare,
  maintain and promote the accessibility
  (web-based) of systematic reviews of best
  evidence in areas including education
• Audience - policy makers, professions,
  researchers, the public.
• Supported by research bodies and government
  departments e.g. on computers
• http//www.cmec.ca/stats/SystematicReview2003.en.p
  df

5
Evidence Based PracticeLevels of
Evidence(Reynolds, 2001)

• 1. Systematic Review of RCTs
• 2. High Quality RCT
• 3. Low Quality RCT
• 4. Outcome Evaluations
• 5. Controlled Single Case Studies
• 6. Case Series
• 7. Expert Opinion

• INTERNAL VALIDITY
• (Intervention Efficacy)
• EXTERNAL VALIDITY
• (Effectiveness in Practice)

6
Some evidence from meta-analyses of RCTs

• Cognitively-speaking reading ability (reading
  texts to extract, construct or construe meaning)
  requires distinct component abilities that all
  benefit from being taught
• Word reading
• Reading fluency
• Text comprehension

7
Some evidence from meta-analyses Word reading

• The central role of accurate and efficient word
  reading as a necessary but not sufficient feature
  of reading comprehension
• Phonologically-based decoding (deciphering) of
  alphabetic systems essential is a necessary but
  not sufficient feature of word reading and
  spelling

8
Some evidence from meta-analyses Word reading

• Letter-sound knowledge and phonemic awareness are
  co-requisites
• Teaching a limited numbers of phonic strategies
  (segmenting, blending, with over-learning) is
  most effective
• These activities should be closely tied to
  experience of real reading of texts
• Little and often preventative early approaches
  have largest effect sizes.
• Small group work appears most successful

9
WHAT WORKS FOR COMPREHENSION PROBLEMS? (i.e. What
RCT meta-analyses exist?)

• Comprehension monitoring
• Co-operative learning
• Graphic and semantic organizers
• Question generation (i.e. by child)
• Question answering (teacher use of questions)
• Story structure
• Mental imagery / guided imagery

10
ABRACADABRA

• Literacy
• Beginning readers
• Teachers of beginning readers
• Based on research
• Web-based application
• Modular Activities, Stories, Professional
  Development, Communication, Assessment
• Flexible - no required path, customizable for
  particular needs, single or multiple student use,
  classroom/lab, at-home, etc.

11
CSLP Software Tools How can technology support
learning and performance?
Research Interests
Evidence
Partners / Field testing
Intervention
12
ABRA Research Matrix
13
ABRA Workflow
14

Reciprocal aspects of evidence-based technology
development
15
Some additional factors

• Effectiveness versus efficacy (chimes with my
  school-based work on early reading interventions
  with school staff)
• i.e. what can be done on the basis of research
  and what is possible on a day-to-day basis in
  schools e.g. using school staff, realistic
  budgets etc
• Cost-effectiveness and availability
• Perceived relevance Local feel, ownership,
  collaboration.

16
CSLP Design Principles

• Research
• Design
• Ethics
• Community

17
Research

• All aspects of tool development should be based
  on peer-reviewed empirical research evidence,
  refined on the basis of research evidence, and
  then serve as the basis for collecting new
  evidence about teaching and learning using
  technology.
• Emphasis should be placed on supporting
  meaningful motivational and learning outcomes.
• Tool design should consider what is known about
  the processes and contexts of learning and
  instruction.

18
Design

• We should strive to achieve the highest standards
  of excellence and lowest programming, interface,
  and instructional design error rates in the field
  of application and use.
• Designs should be appealing and easy to learn and
  use.
• Tools should be designed to address the specified
  goals of the target audience.
• To the extent possible, tools should be inclusive
  and conform to Universal Design Principles.
• Technical support should be readily available and
  integral where possible.

19
Design (Cont.)

• Professional Development, with an emphasis on
  pedagogy, should be readily available and
  integral where possible.
• Design should emphasize adoption and use by the
  largest possible audience with a balance between
  the state of the practice and the state of the
  art.
• Assumptions about physical (e.g., input devices)
  and cognitive skills (e.g., memory/attention,
  problem solving, self-regulation, etc.)
  interpersonal (e.g., collaboration), and other
  skills needed to use the tools for learning
  should be specified by design and validated
  (e.g., via transfer to novel tasks) either before
  or during development.
• To the extent possible and appropriate, tools
  should be flexible and re-usable.

20
Ethics

• The intellectual property is owned by the CSLP
  unless otherwise specified by contractual
  agreements with funders.
• All those who made a contribution will receive
  appropriate recognition.
• Tools should reflect the values of universal
  human rights by avoiding any form of media that
  would be seen to unfairly marginalize a segment
  of our society.

21
Community

• We encourage and support design and development
  by collaboration among CSLP members with diverse
  backgrounds and skills.
• Input from the stakeholders should be sought
  throughout design, development, and testing.
• Tools should be scalable and sustainable with
  minimal post-production cost.
• Tools should be distributed not-for-profit with a
  philanthropic purpose in mind.

22
(No Transcript)
23

Reciprocal aspects of evidence-based technology
development
24
V1 ABRACADABRA Overview

• http//grover.concordia.ca/ABRA/school/
• Research-based activities to develop emerging
  reading skills
• Classroom Resource
• Modular Flexibility
• Some activities, digital stories,

25
ABRACADABRA Interface
26
Interface with Explanations
27
Professional Development
28
Stories and Activities
29
ABRACADABRA Preliminary Research

• Observations of childrens response to the system
  in groups (clips)
• Children discuss texts they have shared on a
  computer in small groups.
• Practising skills on the computer was viable and
  popular with children, parents and teachers

30
Pilot Intervention 2004-2005.

• Aims
• to test the effectiveness of ABRACADABRA
• to explore different sub-questions about reading
  instruction.
• Phonological awareness is a necessary but not
  sufficient for reading acquisition (e.g. Ehri et
  al., 2001).
• Beyond this consensus, rhyme and analogy
  strategy models (e.g. Goswami, 1986, 1999) can be
  contrasted with models that emphasize smaller
  grapheme-to-phoneme units in early reading (e.g.
  Ehri, 1992, 1995).

31
Rhyme- and phoneme-based models

• Rhyme-based models (Goswami Bryant, 1990
  Goswami, 1999) emphasize analysis of a syllable
  into the initial consonant(s) and rimes (e.g.
  back might be analyzed into b ack).
• Models based on smaller grapheme-to-phoneme units
  (hereafter GPCs) emphasize analysis of all
  constituent phonemes of a syllable (e.g. back
  segmented into b - a-ck).

32
Direct contrasts of rime- and phoneme-based
instruction

• Some studies have taught grapheme to phoneme
  rules (e.g. Solity, Deavers, Kerfoot, Crane,
  Cannon, 1999, Stuart, 1999 Vellutino et al.,
  1996)
• Other studies have taught rime units (e.g.
  Bradley, Bryant, 1985 Greaney, Tunmer,
  Chapman, 1997 Wise, Olson, Treiman, 1990).
• Interventions are often horse race models,
  contrasting either rime- or phoneme- based
  training with regular classroom teaching (e.g.
  Iversen Tunmer, 1993) or untaught controls
  (e.g. Ball Blachman, 1991)

33
Direct contrast of rime- and phoneme-based
interventions

• Fewer matched studies exist contrasting rime- and
  phoneme-based interventions
• Hatcher, Hulme, Snowling, 2004 Bowey
  Christensen, 2005 Johnston Watson, 2005-
  phonemes advantaged
• Savage, Carless, Stuart, 2003 Savage
  Carless, 2005 dead heat
• Walton et al., 2001 rimes advantaged

34
The need for controlled research studies of
computer-based literacy

• There is a large body of research on
  computer-based intervention
• Much research has not been well-controlled
• Lack of randomized control
• Studies have not isolated the medium of
  instruction
• (Torgersen Zhu, 2003)

35
Abracadabra Research Design

• Groups of 4 students, randomly assigned to
  intervention A or B, removed from their class
  during language arts period
• Each for 20 minutes 4 times per week
• Each group also received identical comprehension,
  fluency tasks but differed in subtle ways on
  aspects of alphabetics.
• Intervention A these students activities had a
  particular focus on phoneme blending and
  segmenting activities.
• Intervention B these students activities
  focused on a progression from sentences to words
  to rhymes.
• Comparison group This group remained in the
  classroom and received classroom language arts
  instruction.

36
Abstract Decoding Skills

• Pre- and post-test decoding skills performance
  for A (Rime) B (Decoding) and C (Control) groups.
  
• All children improved in decoding skills.
• There are signs of moderate additional
  advantages for children
• in Intervention A.

37
Processing Speed

• Pre- and post-test naming speed skills
  performance for A (Rime) B (Decoding) and C
  (Control) groups.
• Children in intervention and control group
  conditions improved in
• processing speed.
• There are signs of greater processing speed
  reductions for
• children in Intervention A over Intervention
  B.

38
Word Reading

• Pre- and post-test reading vocabulary performance
  for A (Rime) B (Decoding) and C (Control) groups.
• Children in all groups improved in word reading.
• Clear signs of moderate additional advantages
  for children in
• Interventions A and B.
• Effect size of 0.33.

39
Reading Comprehension

• Post-test reading comprehension performance for A
  (Rime) B (Decoding) and C (Control) groups.
• This shows that there were modest differences
  in reading
• comprehension at post-test.
• Modest advantages favoured the children taught
  explicit decoding skills.

40

Reciprocal aspects of evidence-based technology
development
41
Current work V2

• Intensive development activity on improving the
  word-analysis activities. Aim for even stronger
  text-level comprehension effects.
• http//grover.concordia.ca/ABRA/version1/abracadab
  ra.html

42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
(No Transcript)
51
(No Transcript)
52
(No Transcript)
53
(No Transcript)
54
Current work

• .

• Further Intervention in grade 1. We are currently
  explore performance in grade 1 (level 1 cycle 1)
  using the revised ABRA (N 150 participants in 6
  schools)
• Kindergarten Intervention. N 43 students used
  the computer center in groups of 4 with the
  guidance of the researcher for a period of 10
  minutes. The intervention groups are as described
  above

55
Grade 1 study Design

• Similar to the pilot study design
• RCT
• Precisely-define program with a total of 13
  hours of intervention per child
• Careful assessment of treatment integrity
• (self-report, repeated treatment observation,
  activity monitoring, ongoing support)

56
Procedural Overview

• 40 Sessions of intervention as pull-out
• 4 times per week
• 20 minutes
• 13 hours total
• Within classroom environment and daily schedule
• All students participated
• Randomly allocated to groups of 4
• 1 rime group and 1 phoneme group and 1 control
  group per class

57
Results of Grade 1 Intervention

• Letter-sound knowledge

58
Results of Grade 1 intervention

• CTOPP Blending standard score

59
Results of Grade 1 Intervention

• CTOPP Phoneme Elision standard score

60
Results of Grade 1 Intervention

• GRADE Listening Comprehension (Stanine)

61
Effects of Grade 1 Intervention

• Woodcock - Johnson Word Attack

62
Results of Grade 1 Intervention

• GRADE Reading Comprehension composite

63
Results of Grade 1 Intervention

• GRADE Total Test (Word reading and passage
  comprehension)

64
Summary

• ABRA was successful in developing
• Letter-sound knowledge
• Phonological awareness
• Listening comprehension
• Reading comprehension
• Discernible effects on word attack

65
Additional questions

• Variability in response to interventions
• Children with attention difficulties might
  ABRA moderate development here?
• Children with EAL or learning in multiple
  language environments (English, French, Hebrew)
• Children from lower SES backgrounds
• . Work in progress!

66
The ABRACADABRA Kindergarten Intervention Study

• Erin Comaskey (M.A. student)
• under the supervision of Dr. Robert Savage

67
Intervention Overview

• Sample 44 Students
• 17 from school 1 control (majority English
  speakers)
• 27 from school 2 Intervention (majority EAL)
• Rime 13
• Phoneme 14
• Control 17

68
Procedural Overview

• 40 Sessions of intervention
• 2-3 times per week
• 15 minutes
• 10 hours total
• Within classroom environment and daily schedule
• All students participated
• Randomly allocated groups of 4
• 2 Rime Groups and 2 Phoneme Groups

69
Assessment Tasks

• PPVT Standard Vocabulary Assessment (pre-only)
• Letter-Sound Knowledge
• WRAT Sight Word Vocabulary
• Blending and Segmenting of Balanced CV and VC
  words (t-ea, ea-t)
• Non-Sense Word Decoding of High and Low Rime
  Words (dat, mip vs tav, pid
• Reading of No Rime Real Words (e.g. soap,
  yes)
• Rime and Coda Phonological Articulation Task
• (pin, win, -gt /in/, pin, men -gt /n/, )

70
Predicted Results Interventions vs. Controls

• Phoneme Intervention
• Better performance in
• blending tasks CV VC
• decoding of all types of words and non-words
• articulation of codas

• Rime Intervention
• Better performance in
• blending of CV words (on-set and rimes)
• decoding of high-rime nonsense words
• articulation of rime units

71
Letter Sound Knowledge Task
Effect Size .163 Means Control 14.952 Phoneme
18.436 Rime 19.449
72
Blending of CV Words
Effect Size .114 Means at Post Test Control
6.357 Phoneme 6.689 Rime 4.253
73
Blending VC words
Effect Size .273 Means (on Post-Test) Control
4.06 Phoneme 7.530 Rime 4.959
74
Analysis of Reading Tasks

• Combination of Reading Tasks (nonsense-no rime
  WRAT sight words

Effect Size .084 Marginal Mean Scores Control
4.54 Interventions 7.975
75
Individual Reading Task Results The WRAT
Effect Size .085 Means at Post Test Control
1.756 Phoneme 2.910 Rime 4.109
76
Decoding of Non-Sense Words
Effect Size .051 Means at Post Test Control
1.292 Phoneme 1.624 Rime 2.715
77
Decoding of Non-Sense Words
Effect Size .051 Means at Post Test Control
1.292 Phoneme 1.624 Rime 2.715
78
No-Rime Word Decoding
Effect Size .083 Means Control 1.392 Phoneme
1.851 Rime 2.879
79
Preliminary conclusions

• ABRA was effective in
• 1. Increasing K students letter-sound knowledge
  (both intervention groups)
• 2. More effective in developing K students
  decoding skills over no ABRA access (both
  intervention groups)
• 3. GPC skills developed in rime-taught group
• 4. Developing K students blending phonemic units
  to form words (phoneme group)

80

Reciprocal aspects of evidence-based technology
development
81
Immediate development goals for ABRACADABRA V3

• Reading with speed / fluency
• Further accuracy / decoding activities
• Story analysis elements
• Sentence parsing
• vocabulary activities

82
(No Transcript)
83
(No Transcript)
84
(No Transcript)
85
(No Transcript)
86
Professional Development to September 2007

• Teacher PD activities just in time videos from
  (and for) professionals
• Teachers discussing techniques they use
• Students using software effectively
• References
• Extension activities
• Information / support for pre-service teachers

87
2006-2007 activities

• Assessment module development
• Link to suggested activities (evaluated by
  research)
• Self-assessment features
• Communication folder links to schools
  reporting?
• Security

88
New formats
89
New formats.
90
New interface developments

• We are exploring 2 new forms of interface
  http//grover.concordia.ca/abra/pitch/

91
(No Transcript)
92
Research 2006-2007

• To date our work has exclusively been small group
  pull-out from the class
• What is the effectiveness of ABRA as part of a
  balanced reading curriculum in whole class
  teaching?
• Need for research funding!!!

93
Canada-wide initiatives

• A multi-site evaluation of ABRACADABRA
• The CSLP has been successful already in
  developing a Pan-Canadian team that stretches
  from the Maritimes to British Columbia, with an
  unparalleled level of world-class expertise.
• Effectiveness of ABRACADABRA for children from
  different language and cultural backgrounds (e.g.
  bi- and multi-lingual backgrounds, children
  arriving in Canada with little or no English
• We plan to have a French version of ABRA)
  language skills, also urban / semi-urban / rural/
  Gender Age/ experience.

94
e-portfolio

• Designed for elementary and early secondary
  schools
• Supports all stages of portfolio process
• Provides both a process and showcase portfolio,
  which allows written work, images, and sound to
  be stored, modified, and viewed
• Developed with PHP, and uses MYSQL db
• Web-based, bilingual
• Freely available from FTP site

95
Student Environment
96
Personal Space
97
Features
Learning Goals
Add New Work
98
Features
Reflections
Conferencing
99
Other Features

• View Work
• Edit Work
• Send to Showcase
• View Showcase
• Showcase (Reasons Why)

100
Digital Reading Portfolio
101
(No Transcript)
102
(No Transcript)
103
e-portfolio Research

• Four-year longitudinal study of the use and
  impact of e-portfolio
• Objectives
• To identify the nature of e-portfolio use
  (quality and quantity), and additional ways to
  promote teacher professional development within
  the tool
• To measure the impact on key processes (e.g
  self-regulation, motivation, collaboration and
  problem-solving skills)
• To measure the impact of digital portfolio use on
  learning outcomes (e.g. literacy skills) and
• To look at at-risk students and students with
  special needs

104
(No Transcript)
105
(No Transcript)
106
ABRACADABRA Links and further information

• ABRACADABRA (V2)
• http//grover.concordia.ca/ABRA/version1/abracadab
  ra.html
• The CSLP website
• http//doe.concordia.ca/cslp/ICT-ABRACADABRA.php
• Me robert.savage_at_mcgill.ca
• My web-page http//www.education.mcgill.ca/pr
  ofs/savage/

107
Rob Savage, McGill University and the CSLP team

• An invite We welcome others to use our tools and
  partner with us in research and development.