Title: Intervention Implementation Integrity Within Conjoint Behavioral Consultation: Strategies to Maximize Outcomes
1Intervention Implementation Integrity Within
Conjoint Behavioral Consultation Strategies to
Maximize Outcomes
- S. Andrew Garbacz, MA
- Susan M. Sheridan, PhD
- Michelle S. Swanger-Gagne, MA
- Amanda L. Witte, MA
- University of Nebraska-Lincoln
- The Nebraska Center for Research on Children,
Youth, Families, and Schools (CYFS
http//cyfs.unl.edu) - The project is funded by the Institute of
Educational Sciences - Grant Award Number R305F050284
CBC in the Early Grades
2School-Based Consultation
- School-based behavioral consultation is an
indirect form of service delivery wherein a
specialist with expertise in social-behavioral
and academic interventions works collaboratively
with a treatment agent (i.e., consultee, such as
a teacher or parent). - School-based consultation may focus on
behavioral, socio-emotional, or academic
concerns, which interfere with learning.
3Conjoint Behavioral Consultation
- Conjoint behavioral consultation (CBC Sheridan,
Kratochwill, Bergan, 1996 Sheridan
Kratochwill, 2008) is an extension of traditional
behavioral consultation services. - CBC is, by definition, an indirect model of
service delivery intended to enhance the skills
of natural treatment agents (i.e., teachers,
parents) who assume responsibility for
implementing intervention plans. - Goals of CBC focus on addressing the specific
needs of the child, while working collaboratively
with the childs teacher and parent to
strengthen family-school partnerships. - - Including enhancing the skills of natural
treatment agents (i.e., consultees).
4Implementation Integrity
- Intervention implementation integrity refers to
the extent to which an intervention plan is
implemented as intended. - An important consideration in CBC is the ability
of individuals to deliver treatment plans
accurately and efficiently. - Thus, the issue of implementation integrity is
central to the utility of CBC in addressing
students behavioral needs.
5Implementation Integrity
- Little systematic attention is afforded to the
specific strategies that consultants can use to
ensure accurate and reliable implementation of
treatment plan elements in natural settings
(Sanetti Kratochwill, 2008). - Strategies to support families in implementing
interventions within a problem-solving framework
have not received adequate attention.
6Socioeconomic Conditions
- A significant correlate with treatment dropout
and poor outcomes following parent training
interventions is socioeconomic hardship
(Henggeler et al., 1992 Kazdin, 1990 McMahon et
al., 1981). - Families living in low socioeconomic conditions
face numerous stressors (e.g., poverty, lack of
social support) that may interfere with their
ability to implement parenting interventions,
thereby compromising treatment effects.
7Purpose
- Describe the degree to which families implement
interventions as designed in a consultation study
to promote positive behavioral outcomes for
children experiencing behavioral difficulties. - Describe and illustrate multiple methods of
assessing integrity. - Discuss strategies used to maximize intervention
implementation integrity of treatment plans by
all families to increase positive outcomes. - Present methods used by consultants to support
families with diverse challenges.
8Methods - Participants
- Child Participants
- 62 children
- 75 Male
- 67 White, non-Hispanic
- Average age 6.72 (range 5 9)
- All students were referred to the study by their
teachers who indicated a need for further
intervention due to behavioral concerns.
9Methods - Participants
- Parent Participants
- 62 Parents
- 83 White, non-Hispanic
- Average age 35
- 88 Female
- 78 received some college or post-secondary
education - 26 parents met the criteria to be included in a
demographically diverse sub-sample - Data used in this investigation represent part of
a larger, four-year study evaluating the efficacy
of CBC as an intervention that addresses concerns
of kindergarten through third grade students
whose disruptive behaviors place them at risk for
academic failure.
10Methods - Setting
- Cases were conducted in 17 kindergarten, first,
second, and third grade classrooms in a
moderately-sized Midwestern city and surrounding
area, including public and parochial schools. - Interventions were implemented across home and
school settings for students who were the focus
of consultation.
11Methods - Selection
- Students were selected via teacher ratings
- Systematic Screening for Behavior Disorders
(SSBD) multiple-gate screening procedure (Walker
et al., 1990). - Screening instrument with ratings for severity of
externalizing behaviors frequency of
externalizing behaviors and degree of need for
additional intervention.
12Methods Data Procedures
- Cases were separated into two groups
- General mainstream group (N36)
- Demographically diverse group (N26)
- Relative to mainstream group, demographically
diverse was defined as low-income status,
racially diverse, linguistically diverse, living
in a single parent home, or less than high school
diploma.
13Consultation Procedures
- Consultants (i.e., trained graduate students) met
with the teacher and two to three parents for
approximately four to five conjoint consultation
sessions over approximately eight weeks via three
phases. - 1) Needs Identification and Needs Analysis
- 2) Intervention Development and Implementation
- 3) Intervention Evaluation
- Behavioral interventions consisted of 3
components. - 1) Communication component
- 2) Motivation component
- 3) Functional component
14Implementation Support
- Strategies to promote high integrity in the home
setting were developed as part of the larger
study - Sources
- 1) Literature review that informed consultant
training. - 2) Narrative information reported by trained
consultants - See Handout 1
15Implementation Support
- General Strategies
- Follow the partnership model during intervention
development phase (Sheridan Kratochwill, 2008). - Provide a rationale for collecting integrity
data. - Script and package intervention implementation
integrity forms (Watson, 2004). - Train and educate the family on intervention
delivery with various techniques. - See handout 1
16Implementation Support
- Additional strategies for families requiring more
support - 1) Consider culturally-sensitive procedures
(Sheridan Kratochwill, 2008). - 2) Focus on strategies that build trust between
families and educators. - 3) Enhance communication between consultant and
families. - 4) Implement family-centered approach throughout
consultant process. - 5) Gather information about the home setting and
family system. - 6) Facilitate regular contacts and provide
collective support. - 7) Adjust integrity data collection as to improve
ease of collection. - See handout 2
17Case Study
- 6 year old female in 1st grade was the focus of
consultation. - Biological parents and 4 siblings moved to the
United States from Sudan 7 years prior as
refugees. - The family currently meets the 2008 HHS criteria
for a family living in poverty.
18Case Study - Preconsultation
- Conducted an initial meeting to introduce
consultation. - Validated parent concerns (e.g., acculturating to
the United States, feedback from classroom
teachers). - Met with a cultural liaison about roles,
structure, and expectations. - Conducted a home visit to discuss roles and
expectations.
19Case Study Needs Identification and Analysis
- Strengths
- Strong verbal skills
- Independent
- Concerns
- Reading skills
- Ability to stay on-task disruptive behaviors
- Functional assessment revealed escape motivated
behavior, and grade level academic skills.
20Case Study Needs Identificationand Analysis
- Intervention development
- School chunking reading work, appropriate
escape, self-monitoring. - Home structured homework routine that included
entire family, home-school note, activity
checklist with visual cues. - Family checklist to increase integrity.
- Additional support (e.g., meeting reminders,
frequent communication).
21Measurement of Treatment Integrity
- Parent Self-Report Form
- Completed daily by parents and used to assess
intervention implementation integrity at home. - Listed all the steps of the behavioral
intervention plan clearly and objectively. - Example includes intervention plan steps that the
parent in this case completed - See handout 3
22(No Transcript)
23Measurement of Treatment Integrity
- Permanent Product Report Form
- Completed by parents daily for the duration of
intervention (i.e., at least 4 weeks of
intervention). - Charts were collected from parents on which
evidence (e.g., stickers, notes, marks, checks)
demonstrating that s/he implemented steps of the
intervention. - Trained research assistants and consultants
reviewed permanent products and completed the
permanent product report forms to reflect
parents delivery of plan components as reported
on permanent products. - See handouts 4 and 5
24Activity Checklist
25(No Transcript)
26Case Study Strategies to Maximize Integrity
- Integrity information was scripted and
standardized - Intervention steps were simple and consistent
- Opportunity to have forms translated to Arabic
use of visual cues. - Home visits for modeling, observation,
performance feedback, and discussion. - The family reported adhering to 100 of
intervention steps over 4 weeks of implementation.
27Case Study Intervention Evaluation
- School
- Increase from an average of 45 accuracy prior to
the intervention to 80 accuracy after
implementation. - Home
- Decrease in the number of problem behaviors
exhibited from an average of 4 to an average of
1. - Parents indicated improved behavior management
strategies, communication with the school, and
praising children for attaining goals.
28Assessment of Implementation
Table 3 Summary Statistics of Intervention
Implementation Integrity across Sample
Parent (self)a report Permanent Products
Mainstream Group
Percent .77 .79
Standard Deviation .26 .24
Sample Size 26 28
Diverse Group
Percent .81 .91
Standard Deviation .17 .18
Sample Size 19 21
a Participants completed self-report report
measure, permanent product measure, or both
measures
29Discussion
- High levels of intervention implementation
integrity were noted in the home setting across
multiple sources (i.e., families via
parent-report and coders of permanent product
data) - Families seem to be responsive to tactics used by
consultants to maximize treatment integrity. - Families also appear to adhere to the steps of
behavioral interventions regardless of the
integrity measure used, with families reporting
slightly higher integrity when recording
integrity on a permanent product.
30Discussion
- Permanent products are a feasible and useful way
of measuring treatment integrity because families
naturally use the products as they implement
interventions (Sanetti Kratochwill, 2008). - High integrity levels reported on permanent
products may suggest that parents are not
over-reporting on self-report measures. - Families at-risk reported implementing
intervention with higher integrity than the
mainstream group of families.
31Implications
- To maximize outcomes, educators may find it
helpful to use a collaborative consultation model
to design and implement behavioral interventions
with families. - In CBC, training, supporting, and partnering with
teachers and families provides a consistent and
continuous approach to service delivery.
32Future Research Directions
- Examine integrity and the variables that may
influence levels of treatment integrity
experimentally. - Specifically, future research could attempt to
measure the impact of disadvantaged social
settings on the degree to which families follow a
behavioral intervention protocol. - Explore resources that are needed and specific
procedures that are critical to intervention
implementation. The cost-effectiveness of such
practices needs to be assessed in future studies.
33Future Research Directions
- Measure and evaluate treatment integrity in other
ways. - For example, the dosage, quality of, and
responsiveness to implementation may be important
in understanding the quality of services students
are receiving (Dane Schneider, 1998). -
- Standardized measures of consultant strategies
for maximizing integrity should be developed and
used in future research.
34Future Research Directions
- Psychometric properties of future measures of
integrity and strategies used by consultants to
improve integrity should be examined (e.g.,
validity, reliability). - Similar issues with regard to measurement and
evaluation should be explored at school. - The link between home and school treatment
integrity and its effects on student performance
requires examination.
35Contact Information
- Susan M. Sheridan ssheridan2_at_unl.edu
- Andy Garbacz agarbacz2_at_unl.edu
- Amanda Witte awitte2_at_unl.edu
- http//cyfs.unl.edu
36References
- Dane, A. V. Schneider, B. H. (1998). Program
integrity in primary and early secondary
prevention Are implementation effects out of
control? Clinical Psychology Review, 18, 23-45. - Henggeler, S. W., Melton, G. B., Smith, L. A.
(1992). Family preservation using Multisystemic
Therapy An effective alternative to
incarcerating serious juvenile offenders. Journal
of Consulting and Clinical Psychology, 60,
953-961. - Kazdin, A. E. (1990). Premature termination from
treatment among children referred for antisocial
behavior. Journal of Child Psychology and
Psychiatry, 31, 415-425. - McMahon, R. J., Forehand, R., Griest, D. L.,
Wells, K. C. (1981). Who drops out of treatment
during parent behavioral training? Behavioral
Counseling Quarterly, 1, 79-85. - Sanetti, L. M. H., Kratochwill, T. R. (2008).
Treatment integrity in behavioral consultation
Measurement, promotion, and outcomes.
International Journal of Behavioral Consultation
and Therapy, 4, 95-113.
37References
- Sheridan, S. M., Kratochwill, T. R. (2008).
Conjoint behavioral consultation Promoting
family-school connections and interventions (2nd
ed.). New York, NY Springer Publishing. - Sheridan, S. M., Kratochwill, T. R., Bergan, J.
R. (1996). Conjoint behavioral consultation A
procedural manual. New York Plenum. - Walker, H.M. Severson, H. H., Todis, B.J.,
Pedego, A.E. (1990). Systematic screening for
behavior disorders (SSBD) Further validation,
replication, and normative data. RASE Remedial
Special Education, 11, 32-46. - Watson, T. S. (2004). Treatment integrity. In T.
S. Watson C. H. Skinner (Eds.), Encyclopedia of
school psychology (pp. 356-358). New York, NY
Springer Publishing.