Chapter 4

1
Chapter 4 Issues in Single-Subject Research

• Ps534
• Dr. Ken Reeve
• Caldwell College
• Post-Bac Program in ABA

2
Review

• RULES TO FOLLOW AS AN APPLIED BEHAVIOR ANALYST
• 1. Do no harm!
• 2. Do some good!
• 3. Identify WHAT you did that did some good!
  (Focus of this course!)

3
Review

• TARGET BEHAVIOR DEPENDENT VARIABLE
• METHODS TO CHANGE BEHAVIOR (INTERVENTION)
  INDEPENDENT VARIABLE
• FUNCTIONAL RELATIONSHIP degree to which the
  independent variable affects the dependent
  variable (and can you isolate this effect!!)
• Main focus in research is to determine functional
  relationship between intervention X and learning
  outcome Y

4
Features in a Functional Relationship

• When a FUNCTIONAL RELATIONSHIP does really exits
  between IV and DV, then certain other
  characteristics will be present in the research
  findings. These include
• Prediction, replication, verification,
  reliability, and validity (well describe these
  and other related terms)

5
PREDICTION

• When an intervention really DOES have an effect
  on behavior, then there is an expected outcome
  (behavior goes up/down, occurs faster/slower,
  topography changes, etc.)
• When an intervention really has NO effect on
  behavior, then there is an expected NO CHANGE
  as the behavior outcome

6
VERIFICATION

• This refers to CONFIRMING that behavior really
  changed as expected (behavior goes up/down,
  occurs faster/slower, topography changes, etc.)
• VERIFICATION requires that we have a good
  operational definition of the behavior in
  question, a good measurement system, a good way
  to graphically depict the data, and accurate
  observers

7
REPLICATION

• This refers to REPEATING what you did to see if
  your prediction about the functional relationship
  in question holds true more than just once
• Why? Sometimes we can get amazing results JUST BY
  CHANCE and we want to make sure that the results
  werent a fluke

8
REPLICATION S-S vs. GROUP Research

• In group studies, researchers usually replicate
  the observed functional relationship with NEW
  participants
• In single-subject research, researchers usually
  replicate the functional relationship within the
  SAME participant AND then also replicate it with
  new individuals

9
REPLICATION DIRECT vs. SYSTEMATIC

• DIRECT REPLICATION means to do the study exactly
  the same way as before.
• Important but boring.
• Increases INTERNAL validity but not EXTERNAL
  validity (well define these in a minute).
• SYSTEMATIC REPLICATION means to do the study
  with some variation as compared to before (change
  population, setting, stimuli, behavior studied,
  etc.).
• Important and more exciting.
• Increases INTERNAL AND EXTERNAL validity.

10
RELIABILITY

• Refers to how dependable our observations of the
  functional relationship are
• It asks Can we put faith in what we are seeing
  happen before our eyes?
• IF there is a real functional relationship AND we
  are using operational definitions, good
  measurement systems, IOA, THEN we should see the
  same effect repeating to the same degree
• Also means that nothing BREAKS DOWN in the
  research process (think of a reliable car!)
• REPLICATION also shows us RELIABILITY of an
  effect

11
What hurts RELIABILITY?

• POOR IOA observers need to be well trained!
• REACTIVITY of participant when a person changes
  behavior because he or she finds out an
  observation is occurring
• OBSERVER DRIFT over time, the observers change
  their operational definition of the DV

12
VALIDITY

• Recall, yet again, that our goal is to identify a
  functional relationship between IV and DV
• IF we can say that the IV or intervention was the
  ONLY thing that changed the DV, THEN we have high
  INTERNAL VALIDITY
• To be able to do this, we need to rule out
  extraneous variables by not letting them change
  in any way while we are manipulating the IV
  (called controlling for potential confounds)

13
INTERNAL VALIDITY

• Heres an example you might find in the results
  section of a research journal
• As can be seen in Figure 1, when treatment was
  introduced, there was a 40 increase in rate of
  responding as compared to baseline levels. When
  treatment was removed, rate of responding dropped
  back down to baseline measures. When treatment
  was re-introduced, rate of responding again
  increased by 40 as compared to baseline levels.
  Because no other variables were free to change
  during the study, it can be inferred that the
  behavior change was a direct result of the
  introduction and removal of the treatment
  procedure.

14
DREADED THREATS TO INTERNAL VALIDITY

• We have a threat to internal validity when
  other extraneous variables ARE free to change
  during the study
• If a threat is confirmed (or we have good reason
  to think one happened), then we have CONFOUNDED
  the results
• As a result, it CANNOT be inferred that the
  behavior change was a direct result of the
  introduction and removal of the treatment
  procedure.

15
GOOD EXAMPLE OF THREAT TO INTERNAL VALIDITY

• When facilitated communication is used,
  participants are able to answer questions that
  they never could before.
• Sounds like a good functional relationship,
  right?
• If no FC, then no responding from participant
  If YES FC, then YES responding.
• Butthere is a BIG threat to internal validity
  that, in fact, has been confirmed as a CONFOUND
  in pro-FC studies

16
TYPES OF INTERNAL VALIDITYTHREATS

• HISTORY when some event or action by another
  OUTSIDE OF THE LAB affects the DV of the
  participant
• If researcher is made aware of these, then we
  dont have good internal validity!
• EXAMPLES INCLUDE giving a new diet, adding
  another therapy, doing additional training, bad
  week at home, etc.

17
TYPES OF INTERNAL VALIDITYTHREATS

• MATURATION when the DV of the participant
  changes as a result of growth and development
• S-S research controls for this nicely because we
  observe DV of individual multiple times to make
  sure maturation is NOT the cause of any changes
  observed!

18
TYPES OF INTERNAL VALIDITYTHREATS

• Can this change in DV be due to MATURATION?

19
TYPES OF INTERNAL VALIDITYTHREATS

• ATTRITION refers to outcome data that is skewed
  due to certain participants dropping out of a
  study
• Ex. If only the children with the LEAST severe
  autism remain in a study, then it is tough to
  determine if treatment was really good

20
TYPES OF INTERNAL VALIDITYTHREATS

• MULTIPLE TREATMENT INTERFERENCE refers to
  problem of selecting out what part(s) of a
  treatment package were responsible for outcome
  data
• This is a HUGE problem in treatment of autism but
  it is also very pervasive in psychological
  therapies in general
• Problem is that many claims are made about WHAT
  PART is the cause of the effectiveness
• Ex. Floortime, Son-Rise, and TEEACH use many
  ABA techniques. So does Rational Emotive
  Behavior Therapy

21
TYPES OF INTERNAL VALIDITYTHREATS

• TREATMENT DRIFT - refers to problem of therapists
  changing the intervention over time
• Ex.) Therapists may inadvertently increase amount
  of reinforcement, may make instructions more
  salient than before, may prompt more effectively
• If this happens, again we have (take your pick!)
• A problem inferring WHAT caused the change in DV
• A problem identifying a functional relationship
• A threat to internal validity
• A confound in our results

22
TYPES OF INTERNAL VALIDITYTHREATS

• EXPERIMENTER BIAS - refers to MANY possible
  problems of researcher affecting the results
  (inadvertently, we hope)
• Ex.) failing to acknowledge confounds, incomplete
  data collection, selecting participants most
  likely to do well, etc.
• If this happens, again we have (take your pick!)
• A problem inferring WHAT caused the change in DV
• A problem identifying a functional relationship
• A threat to internal validity
• A confound in our results

23
EXTERNAL VALIDITY

• Will the functional relationship identified hold
  true with different participants, settings,
  materials, variations in instruction, etc.
• If so, then we have HIGH external validity.
• Do the findings only work under very narrow
  circumstances?
• If so, then we have LOW external validity
• BIG PROBLEM people often confuse poor external
  validity with poor internal validity in S-S
  research. How?
• We shouldnt use that intervention because it
  has only been shown to work with one person! We
  need more studies with groups of people! (THIS
  IS POOR LOGIC!)

24
EXTERNAL VALIDITYS RELATION TO INTERNAL VALIDITY

• Notice that we need to do SYSTEMATIC REPLICATIONS
  to increase external validity
• But, if each systematic replication continues to
  show the functional relationship between IV and
  DV, then we are ALSO increasing INTERNAL validity
• Successful replications show us that the results
  are likely not due to extraneous variables but
  are caused by the IV manipulated by the
  researcher!
• This relationship is ONE-SIDED, though
  increasing internal validity does NOT increase
  external validity!

25
ETHICS

• What are they?
• A set of standards for behavior that pertains to
  doing what is right
• Behavior analysts must abide by a very stringent
  code of ethics since the behavior change
  techniques used in ABA are extremely powerful
• See the BACBs code for conduct
• (NOTE You should download these from the website
  to study. There will be some general questions
  about the BACB code on our class exam!)

26

• End of chapter 4