Sustainability Through the Looking Glass:

1

• Sustainability Through the Looking Glass
• Shifting Contingencies Across Levels of a System

Jack States Randy Keyworth Ronnie Detrich
2
Sustainable Practice

• A practice that has been adopted, implemented,
  and reliably maintained over time and generations
  of practitioners and decision makers.
• average life of an education innovation is 18-48
  months
• (Latham, 1988)

3
Why Initiatives Fail?
Political support
Funding
Competing reforms
Leadership Stability
Insufficient training
Faculty turnover
Model specificity
Faculty commitment
Sustained professional development
Schools past current performance
Positive student outcomes
no one risk is statistically significantcombinat
ions of risk factors
Sustainability Examining the Survival of
Schools Comprehensive School Reform
Efforts American Institute for Research
4
What do we know about the science of
sustainable implementation?

• Maximize the variables that support
  implementation of the intervention
• Minimize the variables that oppose implementation
  of the intervention
• Align the contingencies at all levels of the
  system
• Recognize and manage the different contingencies
  that exist at the different stages of
  implementation

5
What do we know about maximizing and minimizing
the variables?

• Manage Performance
• Implementation is a social / cultural change
  process
• As a social process, success often has very
  little to do with the details or merits of the
  actual innovation
• Which is why poor innovations are adopted more
  frequently than good innovations

6
How do we Manage Performance?

• OBM is Demonstrating That Managing the
  Contingencies for Groups is Much Like Managing
  Contingencies for Individuals
• Contingencies a contingent relationship between
  an individuals behavior and the outcome of that
  behavior that affects the practices subsequent
  probability.
• Example
• Behavior Student A failing reading
• Intervention Individual tutoring after school
• Outcome Student A achieves grade level on
  reading test
• Meta-contingencies a contingent relationship
  between a cultural practice and an outcome of
  that practice that affects the practices
  subsequent probability.
• Example
• Performance High Stakes test results reveal
  students from School A are not meeting the
  minimum standards in reading
• Intervention Align the schools curriculum with
  the test standards
• Outcome Students enrolled in School A improve
  reading performance to minimum state standards

7
Achieving Sustainable Implementation for the
Great Numbers Requires System Change

• Build and manage an effective organization
• Hire and manage people who will be effective in
  doing
• jobs within the organization.

8
CORE COMPONENTS OF IMPLEMENTATION

• Establish OUTCOMES, GOALS, AND MEASURES
• Select the relevant goals (program, practice,
  fiscal, staffing)
• Establish objective and measurable outcomes and
  align levels
• Employ and Align PERFORMANCE MANAGEMENT
  strategies
• Build activities and systems
• Recruitment and hiring
• Performance expectations
• Training
• Consultation and coaching
• Feedback and evaluation
• Manage contingencies
• Conduct frequent and on-going MONITORING
• Outcome and process
• Assure program fidelity (program level) and
  treatment integrity (practice level)
• Be prepared to ADAPT AND INNOVATE
• Evidence-based practices
• Data-based decision making

9
Monitoring

• Monitoring involves observing a behavior for any
  changes that my occur over time, or for effects
  an intervention may have on the observed
  behavior.
• Evaluate success of the program or intervention
  against goals
• Assess program fidelity and treatment integrity
• Monitoring generates information that is
  essential when making data-based decisions.
• Outcome measures
• Process measures
• Monitoring should occur at all levels of
  implementation
• Organizational (implementation)
• Practitioner (intervention)
• Consumer

10
Individual Behavior and Cultural Practices

Individual Behavior Teacher A Has parents
review spelling test results with child
Contingency Alignment
Desired Outcome School exceeds state expectations
for student spelling
11
Levels of Implementation
Contingency Alignment
Student Education Outcomes Achieved
12
Level of RtIImplementationCalifornia
IDIEA permits use of RtI
California has no mandate or requirements for RtI
A committee to study RtI has been formed - no
policy
Non-Alignment
School special ed staff form team to review
evidence based practices
Does not believe that science is best means to
judge effectiveness of practices
Desired Outcomes Unlikely RtI Is implemented
piece meal and sporadically across the state
13
Alignment of Core ImplementationComponents
Across Levels
14
Why Do So Many Programs Fail?

• They do not plan for and address the different
  contingencies required of each stage

15
Stages Of Implementation

Over Time and Over Generations
Desired Outcome Sustainability
16
Stages Of Implementation

• Adoption Exploration
• Assess the fit
• Utilize data based decision-making / consensus
  building
• Achieve cultural integration
• Program Installation (start-up)
• Create new outcome expectations
• Establish and/or redesign the infrastructure
• Construct or redesign reporting systems
  (monitoring)
• Ensure funding streams
• Develop or modify policies
• HR strategic development
• Train and/or retrain personnel
• Initial Implementation (performance change)
• Initiate new practices and performance
• Troubleshoot obstacles
• Performance feedback systems operational
• Adapt systems and apply to novel situations

17
Stages of Implementation

• Full Operation (Integration)
• HR functions operating (staffing and re-staffing)
  
• Training and re-training
• Treatment provided proficiently and with
  integrity
• Managers supports and facilitates the new
  practice
• Policies and procedures adapted to experience
• Performance management systems support outcomes
• Reporting systems functioning (monitoring)
• Long-Term Operation
• Practice refined and treatment integrity of
  practice maintained
• Undesirable drift controlled
• Innovations adopted and incorporated into the
  practice
• Core practices and outcomes monitored and
  effectiveness sustained over time

18
Types of Change

• Planned and Formal
• Adaptation Changes to an intervention that
  effectively address issues unique to the
  operating environment including assuring a
  cultural fit. These changes do not alter the core
  components of the practice and do not modify
  targeted outcome(s). Adoption, Program
  Installation, and/or Initial Implementation.
• Innovation Changes that offer opportunities that
  improve and expand upon an intervention above
  what has been achieved by current practices and
  procedures. Innovations to a practice should not
  be attempted before treatment integrity is first
  attained. Full operation and/or Long-term
  operation
• Unplanned and Informal
• Drift Undesirable changes that are identified as
  threats to the treatment integrity of the
  practice as defined by the core practices and
  outcome(s). Full operation and/or Long-term
  operation

19
Core Components

• CORE COMPONENTS The essential elements
  necessary to achieve the desired outcomes for the
  consumer.
• Not knowing the core components leads to time and
  resources wasted on attempting to implement a
  variety of non-essential elements.
• Knowing the core components are essential to
  answering critical questions required for
  adaptation.

• Examples
• Differential reinforcement
• Phonics instruction
• Reading fluency
• Systematic de-sensitization
• Coaching
• Use of evidence-based practices

20
Practices

• DEFINITION
• PRACTICES Skills, techniques, and strategies,
  that can be used by practitioners for
  interventions that benefit consumers. Such
  practices reliably produce desirable effects and
  can be used individually or in combination to
  form more complex practices and/or programs.

• Examples
• Direct Instruction (DI)
• Positive Behavior Interventions (PBI)
• Good Behavior Game
• RtI
• New Chance, (for young welfare mothers)
• Teaching family model

21
Programs

• DEFINITION
• PROGRAMS Collections of practices for a defined
  consumer base implemented through an
  organizational structure using a specific
  philosophy, value system, service delivery
  structure, particular funding sources, and core
  practice components. Programs represent an
  efficient method to translate practices into
  effective treatment outcomes.

• Boys Town
• Morningside Academy
• Institute for Effective Education
• Spectrum Center
• Oakland Unified School District

22
Programs - Practices - Components
23
Sustainable Implementation Requires
Successful Aligning and Management of the
Contingencies

• Across levels of the system (Fed, State,
  district, school, classroom)
• Across the core components of implementation
  (goals, performance management systems,
  monitoring, and innovations)
• Across the stages of implementation (adoption,
  installation, initial implementation, full
  operation, long-term)
• Across the Intervention (program, practices and
  core components)