A Meta-Analysis of the Definition, Features, and Effects of Secondary Prevention Interventions

1
A Meta-Analysis of the Definition, Features, and
Effects of Secondary Prevention Interventions

• Pei-Yu Chen Carol Ann Davis
• University of Washington

2
Presentation Outline

• Why do we focus on SWPBS secondary prevention
  intervention?
• What is SWPBS secondary prevention intervention?
• Definitions and Key features
• How does the existing evidence reflect the key
  features of SWPBS secondary prevention?
• What do we know about the secondary prevention
  intervention?
• What are questions about secondary prevention
  intervention still left not answered?

3
A brief review of SWPBS model
Source PBIS website http//www.pbis.org/schoolwid
e.htm
4
Why do we focus on SWPBS secondary prevention
intervention?

• More than 50 studies evaluated the effects of
  SWPBS primary intervention (Horner Sugai,
  2007).
• decreased incidents of office discipline referral
  
• improved students perception of school safety
  and academic outcomes
• The effects of secondary and tertiary prevention
  interventions within the SWPBS model remained
  less clear to the researchers (McCurdy et al.,
  2007).
• might be a result of the lack of consensus on
  core elements of secondary intervention as a
  field

5
Reviewing the SWPBS secondary prevention
intervention

• Investigated the definition and key elements of
  SWPBS secondary prevention intervention
• Applied the key elements as evaluating criteria
  to examine existing secondary prevention studies

6
What is SWPBS secondary prevention intervention?

• Researchers have agreed that SWPBS secondary
  prevention intervention is
• for students who are not responding to primary
  prevention interventions
• for students who are at risk for developing
  serious and chronic behavioral problems
• for small groups or individual student

7
Key elements of SWPBS secondary prevention
intervention

• A wide range of core elements of secondary
  prevention are proposed by researchers (Baker,
  2005, Korb, 2006, Hawken ONeill, 2006, PBIS
  website, 2008, Scott et al., 2002, Sugai
  Horner, 2007), including

• rapid access to intervention,
• requiring low teacher efforts,
• being implemented by school staff,
• interventions consistent with school
  expectations,
• applying flexible interventions based on
  assessment,
• conducting Functional Assessment,
• providing students choices to participate in the
  intervention,
• devoting adequate school resources,
• continuously monitoring students progress

• using systematic feedback,
• building connection with key adults at school,
• increasing collaboration among school staff,
  home, and community,
• implementing a reward system,
• making ecological or curricula modifications,
• providing ample practice opportunities to
  increase social and academic competence
• early screening for intervention
• continuously available services

8
Key elements (continued.)
Great variability among researchers about what
defines a secondary intervention

• PBIS secondary intervention key elements
• continuously available services
• rapid access to intervention,
• requiring low teacher efforts,
• being implemented by school staff,
• interventions consistent with school
  expectations,
• applying flexible interventions based on
  assessment,
• conducting Functional Assessment,
• providing students choices to participate in the
  intervention,
• devoting adequate school resources,
• continuously monitoring students progress

• Applied the key elements proposed by PBIS website
  to further examine existing evidence

9

• Method

10

• Combination of descriptors
• secondary intervention
• secondary prevention
• targeted intervention
• school-wide PBS (SWPBS)
• academic intervention
• tutoring
• positive behavior support
• small group intervention
• social skill instruction
• conflict resolution skills
• social skill club
• check and connect
• Behavior Education Program

• Recruiting studies
• Peer-reviewed empirical studies
• Education Resources Information Center (ERIC) and
  PsycINFO database
• Hand-search articles identified through the
  initial searches

11

• Inclusion criteria
• Published between 1995 to 2007
• Included empirical data to verify the efficacy of
  an intervention
• Conducted in school settings
• Included behavioral and/or academic performance
  as outcome measures

12

• 38 studies were identified through the search
  process
• Studies were categorized into 6 types of
  interventions
• self-determination (BEP/CICO/Check and Connect)
  (n11),
• social skills training (n5),
• reading interventions (n7),
• environmental adjustments (n1),
• behavioral interventions (n1),
• multi-component interventions (n13)
• combine social skills training and another type
  of the above-mentioned interventions

13

• The 38 articles were divided into two groups
• within the SWPBS context
• out of the SWPBS context
• Inclusion criteria of the in SWPBS context
  studies
• studies self-identified as secondary prevention
  interventions, and/or
• a description of primary intervention in the
  study settings

14

• Half of the studies (n19) were conducted within
  the context of SWPBS.
• Studies conducted out of the SWPBS context were
  also included in the review to support the
  efficacy of the intervention
• To evaluate whether the intervention is an
  evidence-based practice

15

• Data analysis
• Each type of secondary intervention is analyzed
  by
• The extent to which the key elements is addressed
  across studies
• Number of studies within and out of SWPBS context
• Design of the studies
• Number of replication across subjects,
  researchers, and settings

16

• Results

17
Within SWPBS context Self-Determination
(BEP/CICO/ Check Connect) studies and key
elements
PBIS Key Elements of Secondary Intervention (n6) Addressed by of studies (n?)
continuously available services 3
rapid access to intervention 0
requiring low teacher efforts 2
being implemented by school staff 6
interventions consistent with school expectations 5
applying flexible interventions based on assessment 1
conducting Functional Assessment 1
providing students choices to participate in the intervention 3
devoting adequate school resources 3
continuously monitoring students progress 6
18

• Self-Determination (BEP/CICO/Check and Connect)
  (n11)

of studies Design of replication across subjects of replication across researcher
Out of context (n 5) Case study (n1) Grade K-2 (n4) 4 groups Anderson, Christenson, Sinclair et al. (1998-2005) Fairbanks et al. (2007) Hawken Horner et al. (2003-2007) McCurdy et al. (2007)
Out of context (n 5) Single subject design (n5) Grade K-2 (n4) 4 groups Anderson, Christenson, Sinclair et al. (1998-2005) Fairbanks et al. (2007) Hawken Horner et al. (2003-2007) McCurdy et al. (2007)
Out of context (n 5) Single subject design (n5) Grade 3-5 (n5) 4 groups Anderson, Christenson, Sinclair et al. (1998-2005) Fairbanks et al. (2007) Hawken Horner et al. (2003-2007) McCurdy et al. (2007)
Out of context (n 5) Non-randomized group (n4) Grade 3-5 (n5) 4 groups Anderson, Christenson, Sinclair et al. (1998-2005) Fairbanks et al. (2007) Hawken Horner et al. (2003-2007) McCurdy et al. (2007)
Within context (n6) Non-randomized group (n4) Grade 3-5 (n5) 4 groups Anderson, Christenson, Sinclair et al. (1998-2005) Fairbanks et al. (2007) Hawken Horner et al. (2003-2007) McCurdy et al. (2007)
Within context (n6) Non-randomized group (n4) Grade 6? (n5) 4 groups Anderson, Christenson, Sinclair et al. (1998-2005) Fairbanks et al. (2007) Hawken Horner et al. (2003-2007) McCurdy et al. (2007)
Within context (n6) Randomized groups (n3) Grade 6? (n5) 4 groups Anderson, Christenson, Sinclair et al. (1998-2005) Fairbanks et al. (2007) Hawken Horner et al. (2003-2007) McCurdy et al. (2007)
19
So, what do we learn about Self-determination
(BEP/CICO Check and Connect)?
Christenson, Sinclair et al. (1998-2005)
well-established probably efficacious
at least 2 well-conducted groups-design studies by different investigators, or 2 well-constructed group-design studies conducted by the same investigators, or
more than 9 single-subject design studies to support the positive effects of the intervention more than 3 but less than 9 single-subject design studies
a treatment manual used for the intervention preferred a treatment manual used for the intervention preferred
Sample characteristics clearly specified Sample characteristics clearly specified

• APA guideline of evidence-based practice
  (Lonigan, Elbert, Johnson 1998)

20
Within SWPBS context Social Skills Training
studies and key elements
PBIS Key Elements of Secondary Intervention (n2) Addressed by of studies (n?)
continuously available services 0
rapid access to intervention 0
requiring low teacher efforts 2
being implemented by school staff 0
interventions consistent with school expectations 2
applying flexible interventions based on assessment 0
conducting Functional Assessment 0
providing students choices to participate in the intervention 0
devoting adequate school resources 0
continuously monitoring students progress 2
21
social skills training (n5)
of studies Design of replication across subjects of replication across researcher
Out of context (n 3) Case study (n0) Grade K-2 (n2) 4 groups of researchers Boxer et al. (2005) Lo et al. (2002) Miller, Lane, Wehby et al. (2003 2005) Gresham et al. (2006)
Out of context (n 3) Single subject design (n4) Grade K-2 (n2) 4 groups of researchers Boxer et al. (2005) Lo et al. (2002) Miller, Lane, Wehby et al. (2003 2005) Gresham et al. (2006)
Out of context (n 3) Single subject design (n4) Grade 3-5 (n4) 4 groups of researchers Boxer et al. (2005) Lo et al. (2002) Miller, Lane, Wehby et al. (2003 2005) Gresham et al. (2006)
Out of context (n 3) Non-randomized group (n1) Grade 3-5 (n4) 4 groups of researchers Boxer et al. (2005) Lo et al. (2002) Miller, Lane, Wehby et al. (2003 2005) Gresham et al. (2006)
Within context (n2) Non-randomized group (n1) Grade 3-5 (n4) 4 groups of researchers Boxer et al. (2005) Lo et al. (2002) Miller, Lane, Wehby et al. (2003 2005) Gresham et al. (2006)
Within context (n2) Non-randomized group (n1) Grade 6? (n1) 4 groups of researchers Boxer et al. (2005) Lo et al. (2002) Miller, Lane, Wehby et al. (2003 2005) Gresham et al. (2006)
Within context (n2) Randomized groups (n0) Grade 6? (n1) 4 groups of researchers Boxer et al. (2005) Lo et al. (2002) Miller, Lane, Wehby et al. (2003 2005) Gresham et al. (2006)

• 1/5 study is conducted in a self-contained
  classroom for students with disabilities

22
So, what do we learn about Social Skills Training
studies?
probably efficacious Meet criteria
2 well-constructed group-design studies conducted by the same investigators, or
more than 3 but less than 9 single-subject design studies support the effect of the intervention ?
a treatment manual used for the intervention preferred
Sample characteristics clearly specified ?

• APA guideline of evidence-based practice
  (Lonigan, Elbert, Johnson 1998)

23
Within SWPBS context Reading Intervention
studies and key elements
PBIS Key Elements of Secondary Intervention (n2) Addressed by of studies (n?)
continuously available services 0
rapid access to intervention 0
requiring low teacher efforts 2
being implemented by school staff 0
interventions consistent with school expectations 2
applying flexible interventions based on assessment 0
conducting Functional Assessment 0
providing students choices to participate in the intervention 1
devoting adequate school resources 0
continuously monitoring students progress 2
24
Reading Interventions (n7)
of studies Design of replication across subjects of replication across researcher
Out of context (n 5) Case study (n1) Grade K-2 (n3) 5 groups of researchers Kamps et al. (2003) Locke Fuchs (1995) Scott Shearer-Lingo (2002) Strong, Lane, Wehby et al. (2002, 2004, 2005) Staubitz et al. (2005)
Out of context (n 5) Single subject design (n5) Grade K-2 (n3) 5 groups of researchers Kamps et al. (2003) Locke Fuchs (1995) Scott Shearer-Lingo (2002) Strong, Lane, Wehby et al. (2002, 2004, 2005) Staubitz et al. (2005)
Out of context (n 5) Single subject design (n5) Grade 3-5 (n2) 5 groups of researchers Kamps et al. (2003) Locke Fuchs (1995) Scott Shearer-Lingo (2002) Strong, Lane, Wehby et al. (2002, 2004, 2005) Staubitz et al. (2005)
Out of context (n 5) Non-randomized group (n1) Grade 3-5 (n2) 5 groups of researchers Kamps et al. (2003) Locke Fuchs (1995) Scott Shearer-Lingo (2002) Strong, Lane, Wehby et al. (2002, 2004, 2005) Staubitz et al. (2005)
Within context (n2) Non-randomized group (n1) Grade 3-5 (n2) 5 groups of researchers Kamps et al. (2003) Locke Fuchs (1995) Scott Shearer-Lingo (2002) Strong, Lane, Wehby et al. (2002, 2004, 2005) Staubitz et al. (2005)
Within context (n2) Non-randomized group (n1) Grade 6? (n2) 5 groups of researchers Kamps et al. (2003) Locke Fuchs (1995) Scott Shearer-Lingo (2002) Strong, Lane, Wehby et al. (2002, 2004, 2005) Staubitz et al. (2005)
Within context (n2) Randomized groups (n0) Grade 6? (n2) 5 groups of researchers Kamps et al. (2003) Locke Fuchs (1995) Scott Shearer-Lingo (2002) Strong, Lane, Wehby et al. (2002, 2004, 2005) Staubitz et al. (2005)

• 2/7 studies are conducted in self-contained
  classrooms
• 1/7 study is conducted in a resource room

25
So, what do we learn about Reading Intervention
studies?
probably efficacious Meet criteria
2 well-constructed group-design studies conducted by the same investigators, or
more than 3 but less than 9 single-subject design studies support the effect of the intervention ?
a treatment manual used for the intervention preferred
Sample characteristics clearly specified ?

• APA guideline of evidence-based practice
  (Lonigan, Elbert, Johnson 1998)

26
Behavioral interventions (n1)
of studies Design of replication across subjects of replication across researcher
Out of context (n 1) Case study (n0) Grade K-2 (n0) 1 group of researchers Lohrmann Talerico (2004) (implemented group contingency to reduce problem behaviors)
Out of context (n 1) Single subject design (n1) Grade K-2 (n0) 1 group of researchers Lohrmann Talerico (2004) (implemented group contingency to reduce problem behaviors)
Out of context (n 1) Single subject design (n1) Grade 3-5 (n1) 1 group of researchers Lohrmann Talerico (2004) (implemented group contingency to reduce problem behaviors)
Out of context (n 1) Non-randomized group (n0) Grade 3-5 (n1) 1 group of researchers Lohrmann Talerico (2004) (implemented group contingency to reduce problem behaviors)
Within context (n0) Non-randomized group (n0) Grade 3-5 (n1) 1 group of researchers Lohrmann Talerico (2004) (implemented group contingency to reduce problem behaviors)
Within context (n0) Non-randomized group (n0) Grade 6? (n0) 1 group of researchers Lohrmann Talerico (2004) (implemented group contingency to reduce problem behaviors)
Within context (n0) Randomized groups (n0) Grade 6? (n0) 1 group of researchers Lohrmann Talerico (2004) (implemented group contingency to reduce problem behaviors)
27
So, what do we learn about Behavioral
Intervention studies?
probably efficacious Meet criteria
2 well-constructed group-design studies conducted by the same investigators, or
more than 3 but less than 9 single-subject design studies support the effect of the intervention
a treatment manual used for the intervention preferred
Sample characteristics clearly specified ?

• APA guideline of evidence-based practice
  (Lonigan, Elbert, Johnson 1998)

28
Within SWPBS context Environmental Adjustment
studies and key elements
PBIS Key Elements of Secondary Intervention (n1) Addressed by of studies (n?)
continuously available services 0
rapid access to intervention 0
requiring low teacher efforts 0
being implemented by school staff 0
interventions consistent with school expectations 0
applying flexible interventions based on assessment 0
conducting Functional Assessment 0
providing students choices to participate in the intervention 0
devoting adequate school resources 0
continuously monitoring students progress 1
29
Environmental Arrangement (n1)
of studies Design of replication across subjects of replication across researcher
Out of context (n 0) Case study (n0) Grade K-2 (n0) 1 group of researchers Kern, Bambara, Fogt (2002) (class-wide curricula modification)
Out of context (n 0) Single subject design (n1) Grade K-2 (n0) 1 group of researchers Kern, Bambara, Fogt (2002) (class-wide curricula modification)
Out of context (n 0) Single subject design (n1) Grade 3-5 (n0) 1 group of researchers Kern, Bambara, Fogt (2002) (class-wide curricula modification)
Out of context (n 0) Non-randomized group (n0) Grade 3-5 (n0) 1 group of researchers Kern, Bambara, Fogt (2002) (class-wide curricula modification)
Within context (n1) Non-randomized group (n0) Grade 3-5 (n0) 1 group of researchers Kern, Bambara, Fogt (2002) (class-wide curricula modification)
Within context (n1) Non-randomized group (n0) Grade 6? (n1) 1 group of researchers Kern, Bambara, Fogt (2002) (class-wide curricula modification)
Within context (n1) Randomized groups (n0) Grade 6? (n1) 1 group of researchers Kern, Bambara, Fogt (2002) (class-wide curricula modification)

• The study is conducted in a self-contained EBD
  classroom

30
So, what do we learn about Environmental
Adjustment studies?
probably efficacious Meet criteria
2 well-constructed group-design studies conducted by the same investigators, or
more than 3 but less than 9 single-subject design studies support the effect of the intervention
a treatment manual used for the intervention preferred
Sample characteristics clearly specified ?

• APA guideline of evidence-based practice
  (Lonigan, Elbert, Johnson 1998)

31
Within SWPBS context Multi-component
Intervention studies and key elements
PBIS Key Elements of Secondary Intervention (n8) Addressed by of studies (n?)
continuously available services 2
rapid access to intervention 0
requiring low teacher efforts 6
being implemented by school staff 3
interventions consistent with school expectations 4
applying flexible interventions based on assessment 1
conducting Functional Assessment 1
providing students choices to participate in the intervention 2
devoting adequate school resources 3
continuously monitoring students progress 5
32
Multi-component interventions (n13)
of studies Design of replication across subjects of replication across researcher
Out of context (n 5) Case study (n1) Grade K-2 (n10) 8 groups of researchers Anderson et al. (2006) Cavell Hughes (2000) Ervin et al. (2007) Lane Menzies (1999, 2003) Lewis, Sugai, Colvin (1998) Golly, Walker, Stiller et al. (1998) Kamps, Kravis, et al. (1999, 2000, 2002) Plumer (2005)
Out of context (n 5) Single subject design (n2) Grade K-2 (n10) 8 groups of researchers Anderson et al. (2006) Cavell Hughes (2000) Ervin et al. (2007) Lane Menzies (1999, 2003) Lewis, Sugai, Colvin (1998) Golly, Walker, Stiller et al. (1998) Kamps, Kravis, et al. (1999, 2000, 2002) Plumer (2005)
Out of context (n 5) Single subject design (n2) Grade 3-5 (n8) 8 groups of researchers Anderson et al. (2006) Cavell Hughes (2000) Ervin et al. (2007) Lane Menzies (1999, 2003) Lewis, Sugai, Colvin (1998) Golly, Walker, Stiller et al. (1998) Kamps, Kravis, et al. (1999, 2000, 2002) Plumer (2005)
Out of context (n 5) Non-randomized group (n3) Grade 3-5 (n8) 8 groups of researchers Anderson et al. (2006) Cavell Hughes (2000) Ervin et al. (2007) Lane Menzies (1999, 2003) Lewis, Sugai, Colvin (1998) Golly, Walker, Stiller et al. (1998) Kamps, Kravis, et al. (1999, 2000, 2002) Plumer (2005)
Within context (n8) Non-randomized group (n3) Grade 3-5 (n8) 8 groups of researchers Anderson et al. (2006) Cavell Hughes (2000) Ervin et al. (2007) Lane Menzies (1999, 2003) Lewis, Sugai, Colvin (1998) Golly, Walker, Stiller et al. (1998) Kamps, Kravis, et al. (1999, 2000, 2002) Plumer (2005)
Within context (n8) Non-randomized group (n3) Grade 6? (n2) 8 groups of researchers Anderson et al. (2006) Cavell Hughes (2000) Ervin et al. (2007) Lane Menzies (1999, 2003) Lewis, Sugai, Colvin (1998) Golly, Walker, Stiller et al. (1998) Kamps, Kravis, et al. (1999, 2000, 2002) Plumer (2005)
Within context (n8) Randomized groups (n7) Grade 6? (n2) 8 groups of researchers Anderson et al. (2006) Cavell Hughes (2000) Ervin et al. (2007) Lane Menzies (1999, 2003) Lewis, Sugai, Colvin (1998) Golly, Walker, Stiller et al. (1998) Kamps, Kravis, et al. (1999, 2000, 2002) Plumer (2005)
33
So, what do we learn about Multi-component
interventions?
well-established Meet criteria
at least 2 well-conducted groups-design studies by different investigators, or ?
more than 9 single-subject design studies to support the positive effects of the intervention
a treatment manual used for the intervention preferred
Sample characteristics clearly specified ?

• APA guideline of evidence-based practice
  (Lonigan, Elbert, Johnson 1998)

34
Combination of the interventions
Combination of the interventions Combination of the interventions of studies
Social skills training Academic intervention 4
Social skills training Academic Behavioral intervention 2
Social skills training Behavioral intervention 1
Social skills training Behavioral Self-determination 1
Social skills training Environmental Adjustments 1
Social skills training Family Consultation 3
Social skills training Academic intervention Family Consultation 1

• If we look at each sub-type of intervention
  separately, more studies are needed to support
  the efficacy of each sub-type multi-component
  intervention.

35
Overall, what do we know about the secondary
prevention intervention?

• Small number of studies are conducted within the
  context of SWPBS.
• More studies are needed to support the efficacy
  of each type of SWPBS secondary prevention
  interventions
• Three key elements listed on the PBIS website are
  addressed by most of the within SWPBS context
  intervention, including
• requiring low teacher efforts
• interventions consistent with school expectations
  
• continuously monitoring students progress

36

• Discussion

37
What are the questions about secondary prevention
intervention still left not answered?

• Most of the interventions did not address 7/10 of
  the key elements. Whether schools could apply
  these elements as criteria to select secondary
  intervention is unknown.
• What makes an intervention a secondary
  intervention?

38

• The difference between secondary and tertiary
  prevention intervention remain blurred.
• Functional Assessment
• Small group and individual interventions
• Long-term effects of the interventions are
  uncertain as a result of brief intervention
  phases.

39

• This may be a call to researchers or authors to
  more carefully describe the context under which
  the intervention is being conducted.

40
PBIS Key Elements of Secondary Intervention
continuously available services
rapid access to intervention
requiring low teacher efforts
being implemented by school staff (some not all)
interventions consistent with school expectations
applying flexible interventions based on assessment
conducting Functional Assessment (brief)
providing students choices to participate in the intervention
devoting adequate school resources
continuously monitoring students progress
providing ample practice opportunities to increase social and academic competence
41

• Thank you
• Questions and Feedback
• cadavis1_at_u.washington.edu