Assessing the Impact of the Experimental Manipulation

1
Assessing the Impact of the Experimental
Manipulation

• Thomas G. Bowers, Ph.D.
• 2003

2
Assessment of the Independent Variable

• Requires both good construct validity and careful
  control of IV
• Check on manipulation is extremely important in
  clinical and educational research

3
What Are the Types of Manipulations?

• Variations
• 1. Types of information varied
• 2. Variations in subject experience
• 3. Treatment or intervention variation

4
Variations in Intervention Conditions

• Subjects are often exposed to different
  treatment, prevention, counseling, or similar
  effort
• The check here is to insure the manipulation was
  carried out successfully, as intended

5
Variations in Intervention Conditions

• Refers to treatment fidelity or treatment
  integrity
• Often not assessed
• Some research suggests there may be considerable
  problems here
• Use of manuals in therapy is now common

6
Variations in Intervention Conditions

• Manual Guides
• Outlines procedures, specific tasks,
  therapist/client characteristics
• Therapists can be specifically trained
• Role-playing, video-taping, specific methods

7
Variations in Intervention Conditions

• Provide continued supervision
• Monitor performance
• Measure the implementation of treatment

8
Variations in Intervention Conditions

• Issues in excluding subjects
• Only include subjects with positive manipulation
  checks?
• Both positive and negative aspects
• Subjects are likely to vary significantly
• It may be useful to know how often the
  manipulation does not work
• An unplanned pos-hoc test may be valuable

9
Variations in Intervention Conditions

• Issues in excluding subjects
• Drop outliners or extreme scores?
• I generally argue against this, unless some
  specific compelling case can be made
• Missing data for subjects
• Drop case
• May systematically bias sample
• Missing subjects may have poorer outcome
• Least squares fit
• Apply group or cell mean

10
Summary

• Manipulation checks on the independent variable
  is very important in most clinical research