Maintaining the integrity of a design in a randomized trial

1
Maintaining the integrity of adesign in a
randomized trial

• John Reid
• Oregon Social Learning Center
• Center for Research to Practice

2
Some General Issues

• Your intervention and analytic models guide the
  way you set up the relationships and contracts
  with all participants in the study.
• In my own work on a multi-component prevention
  trial, the design specified repeated and rather
  extensive assessments involving parents,
  students, parent group leaders, teachers,
  playground supervisors, school administrators,
  juvenile justice systems, and now adult
  corrections, mental health, child welfare.

3
Maintaining participation

• Getting identification and contact information
• Relationship with participants warm and fuzzy yet
  blinded.
• Maintaining participation motivation
• The assessment design is critical
• To ease assessment burden
• To plan for different assessment agents/methods
  as project matures
• To collect initial data to measure effects of non
  random attrition

4
Some practical issues

• Think through your budget. Families under a
  great deal of financial and contextual stress
  tend to move a lot.
• This also has big implications for the design of
  your intervention.
• If school based intervention, many kids wont be
  there at the end of the school year, and many
  kids may move in after intervention but before
  school year is over.

5
An example interventionLinking the Interests of
Families and Teachers (LIFT)

• John Reid, Mark Eddy, Becky Fetrow, Mike
  Stoolmiller, et al
• Oregon Social Learning Center

6
(No Transcript)
7
(No Transcript)
8
(No Transcript)
9
Issues Continued

• Relationships with each set of participants need
  to be spelled out and serviced over time.
  Informed consent, flexibility, understanding,
  warmth, and rigor.
• Each set of relationships must be thought through
  with care.

10
Teachers

• Superintendent -gt Research Review Board
• Principal -gt Teacher -gt
• Teachers on a one to one basis
• Let teachers randomize
• Assessment selection
• Support in Intervention activities

11
Families

• Letter from School sets up contact from
  researcher
• Phone call to set up paid face to face interview
  to get consent or refusal.
• Informed and signed consent.
• Significant payments
• Support for participation in groups

12
Maintaining integrity

• Training groups group leaders responsible for
  attendance, weekly supervision and support
• Phone calls. PDR. Entered/checked daily.flexible
  (pay phones)
• Teacher support for training/ playground and
  LIFT line.

13
(No Transcript)
14
Maintaining sample over time

• State of the art prevention trial designs call
  for repeated measures a lot of them. Analysis
  of growth is a powerful way of examining changes
  in developmental trajectories, and increases
  dramatically with more time points.

15
Families move

• And its not random, in our trial, the families in
  the highest poverty catchment areas moved most,
  and often disconnected phones and no forwarding
  address. From 35 to over 100 turnover rates.
  This can be a huge problem, given that we expect
  variation in effects in universal prevention
  trials, and the families hardest to maintain may
  be most critical to showing outcome effectiveness

16
(No Transcript)
17
(No Transcript)
18
Maintaining families and kids

• Magda will cover this in detail, but
• 1. Initial contact data get enough to evaluate
  effects of attrition
• 2. News letters
• 3. Incentives to provide change address
• 4. Budget travel for assessors and families,
  incentives
• 5. One missed assessment isnt total loss

19
(No Transcript)
20
Assessment requirements change over time
21
(No Transcript)
22
Academic Progress
23
Conduct Disorder (DSM IV Criteria)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
Baltimore analytic model
Decreased Later Substance Abuse
Decreased Aggressive, Disruptive Classrooms
Decreased Individual Aggression
Classroom Behavior Management
Individual
Classrooms
Aggression
Decreased Later Conduct Anti- social
Personality Disorders
Teachers Effective Academic Instruction
Increased Achievement
Poor
Poor
Improved Reading Skills
Reading Skills
Achievement
School Success Decreased Drop-Out
Decreased Depressive Symptoms
Effective Family-Classroom Partnerships
Depressive
Symptoms
Decreased Later Depressive Disorders
Other mediating or moderating variables

• Family structure and poverty

Whole Day First Grade Education and Prevention
Program

• Deviant peers

Whole Day

• School building

• Community economic health,
• resources, drugs, and violence

Kellam Framework
28
Nine parameters in the framework for doing
randomized field trials (RFT)

• RFT Strategies and Phases
• Negotiating community and institutional
  partnerships
• 3)    Epidemiology as context for doing
  randomized field trials
• 4)    Role of theory and previous research
• 5)    Specifying the observable and measurable
  core elements of the intervention
• 6)    Measuring implementation, baseline status,
  mediating and moderating variables, and periodic
  outcomes
• 7)    Design, sampling, and statistical power
• 8)    Analyses of variation in impact
• 9) Economic analysis cost, cost-effectiveness,
  cost-benefit

Kellam Framework