Topics in Special Education Research

1
Topics in Special Education Research

• Session 3-Experimental
• Quasi-experimental Research

2
This Afternoons Agenda

• Review Course Requirements Upcoming Assignments
• Ethics in Research
• Review for, Take, Correct Quiz
• Discussion Lecture on Experimental Research
• Activity
• Dismissal

3
Updates/Questions

• Discussion Guides- Please submit to dropbox
  folder and label
• 3 main points for all of the readings for that
  session.
• No need to double space
• Address all of the headings (even the
  difficulties, concerns, questions section).
• Use complete sentences
• These are designed to guide your discussions.
• Please put your questions for ME on the feedback
  guide, so if they arent answered by your
  discussion group, type them on the feedback
  guide!

4
Upcoming Assignments

• Today NIH/CITI training modules
• Annotated Bibliography (individual assignment)
• Each individual reviews 3 research articles
  regarding their topic
• See Example http//rxsped596.pbworks.com/w/file/f
  etch/54804527/Example20Annotated20Bibliography.p
  df
• Conceptual Framework (group assignment)
• Group submits short summary of literature and
  presents a conceptual framework for theories that
  drive their proposal.
• Article Review Assignment (group assignment)
• Written Research Proposal (group assignment)
• Presentation of Research Proposal (group
  assignment)

5
Research Question(s) Guidelines

• Briefly and clearly state how each research
  question will be addressed.
• For example, This research question will be
  answered by comparing the end-of-year state wide
  test scores of students who received the
  intervention and those who did not receive the
  intervention.
• Briefly present the proposed theoretical and
  practical implications of the findings.
• e.g., The results of this study may have
  implications for the use of the evidence-based
  Super-Duper Reading Intervention by elementary
  special education teachers

6
Research Question Tips

• Framed based to operationalize (clearly define so
  that it can be replicated) the objectives of the
  proposed research project.
• Mention the IV and DV and how they will affect
  each other
• Framed based on methodology
• Experimental/Quasi-experimental/Single-subject
  Is there a causal of functional relationships
  between IV and DV
• Correlational Is there a correlational
  relationship between IV and DV
• Descriptive/Qualitative Describe a phenomenon or
  issue better, What is the prevalence of
  intellectual disabilities in African-American
  middle school students?

7
Ethics in Research

• What did you learn from doing the CITI course?
• Regarding
• Research misconduct
• Data Management
• Responsible authorship
• Collaboration in Research
• http//www.youtube.com/watch?vW7sfIA1dIGQ
• http//www.youtube.com/watch?vyr5cjyokVUs
• http//www.youtube.com/watch?vb8McGyYAwcU

8
The Belmont Report (1979), National Commission3
Principles

• 1. Respect for Persons-
• Required to obtain informed consent respect
  privacy of participants
• 2. Beneficence-
• Required to use best possible design to maximize
  benefits minimize harms, show they can perform
  the procedures and handle the risks, prohibit
  research that that is without a favorable
  risk-benefit relationship
• 3. Justice
• Required to select participants equitably
  avoid exploitation of vulnerable populations or
  populations of convenience.

9
PSU Human Subjects Research Review Committee
(HSRC)

• http//www.rsp.pdx.edu/compliance_human.php
• Portland State University (PSU) is responsible
  for the rights and welfare of human subjects
  involved in research sponsored or conducted by
  the university.   In order to meet this
  responsibility, the University established the
  Human Subjects Research Review Committee. 
• Members are charged with reviewing all research
  conducted under the auspices of PSU that involves
  human subjects to ensure adequate protections are
  in place.

10
Review for Quiz
11
In-text Citations Formatting Quotations
When quoting, introduce the quotation with a
signal phrase. Make sure to include the authors
name, the year of publication, the page number,
but keep the citation briefdo not repeat the
information.
Caruth (1996) states that a traumatic response
frequently entails a delayed, uncontrolled
repetitive appearance of hallucinations and
other intrusive phenomena (p.11).
A traumatic response frequently entails a
delayed, uncontrolled repetitive appearance of
hallucinations and other intrusive phenomena
(Caruth, 1996, p.11).
12
In-text Citations A Work with Two Authors
When citing a work with two authors, use and in
between authors name in the signal phrase yet
between their names in parenthesis.
According to feminist researchers Raitt and Tate
(1997), It is no longer true to claim that
women's responses to the war have been
ignored (p. 2). Some feminists
researchers question that women's
responses to the war have been ignored (Raitt
Tate, 1997, p. 2).
13
In-text Citations A Work with Three to Five
authors
When citing a work with three to five authors,
identify all authors in the signal phrase or in
parenthesis. (Harklau, Siegal,
Losey, 1999) In subsequent citations, only use
the first author's last name followed by "et al."
in the signal phrase or in parentheses.
(Harklau et al.,
1999)
14
In-text Citations A work with 6 or more authors

• When citing a work with more than 6 authors, you
  do NOT have to identify all authors in the signal
  phrase or parenthesis.
• You identify the first author and use et al.
  after his/her name with the date of the
  publication
• (Carr et al., 1999) OR
• signal phrase in text e.g.
• Carr et al. (1999) conducted a meta-analysis of
  behavior support practices

15
Steps in the Research/Scientific Process

• 1. Identify socially important issue
• 2. Review current literature
• 3. Define conceptual model
• 4. Define specific hypothesis(es) and research
  question(s)
• 5. Define dependent variable(s)/measure

• 6. Identify independent variable(s)/measures
• 7. Select appropriate research design
• 8. Obtain consents
• 9. Collect data
• 10. Analyze data
• 11. Communicate results
• Written presentation
• Oral presentation

16
Conceptual Models (or theoretical models)

• Theory that drives the research.
• Guides our thinking and provides rules,
  principles that guides the research and
  practice.
• Structure of assumptions, principles, and rules
  that holds together the ideas of a broad concept.
  
• Outlines your research

17
Logical Flaws of FBA use in public schools
(Scott et al., 2005)

• FBA is used mainly as a reactionary approach.
• opportunity is lost to utilize FBA technology to
  develop interventions that address minor
  behaviors that usually precede more serious
  problems.
• FBA is restricted to set of procedures used by
  experts
• The rich supply of information from people with
  whom the student interacts with the most is lost.
• FBA is restricted to rigorous procedures that are
  unrealistic for public school settings.
• Disincentive for using FBA technology.
• Cynicism as to the practicality of FBA .

18
ProactiveParsimoniousPractical FBA in schools

• FBA conceptualized by Scott et al. as a proactive
  pre-referral routine that uses the most
  parsimonious procedures required to create an
  effective behavior support plan.
• Given the time resource constraints in schools,
  we must encourage schools to work smarter to
  develop capacity to implement technology to
  effectively support more students.
• Use Practical FBA procedures to develop capacity
  within a school to utilize FBA technology.

19
Practical FBA Logic Model
Individualized Supports 5 of Students
Behavior Specialist responsible for 25 FBAs in
school of 500
Personnel with flexible roles conduct proactive
Practical FBA to expand the scope of FBA,
prevent intensive problem behaviors, decrease
reliance on specialist.
Secondary Group Supports 10-15 of Students
School-wide Positive Behavioral Supports 80 of
Students
20
Literature Review Guidelines

• A) You should educate readers about the topic and
  provide a clear rationale as to why the study is
  important and necessary based on the previous
  research and writing on the topic.
• B) Within your literature review you will present
  the logic or conceptual framework as to why and
  how your current study (topics, methods, designs)
  is organized the way it is.
• C) Make this section compelling. Concisely
  explain the social importance of what you are
  studying.
• e.g., Start with a powerful statement or
  statistic

21
Quiz
22
Correct Quiz
23
Discussion

• Get together with 2-3 other people and use your
  discussion guides to guide your discussion.
• Spend more time on the chapter reading and the
  Quality Indicator Article as we will be doing an
  activity on the other two articles later!

24
Lecture

• Experimental Quasi Experimental Research
• Research Designs Threats to Validity
• Attempt tonight to apply the principle of
  teaching less more thoroughly.
• https//www.youtube.com/watch?vqtLnBz6lbRQ

25
Experimental Design

• Gold Standard in Research (Moore McCabe,
  1993 Feuer, Towne, Shavelson, 2002 Slavin,
  2002).
• Rooted in postpositivist paradigm.
• Seeks to make causal conclusions.
• Difference between experimental design and
  quasi-experimental design is the use of random
  selection of participants.

26
In both experimental quasi-experimental
designs.

• Emphasis is on operationally defining the
  variables (dependent independent) and the
  context of the research.
• Dependent variable(s)- outcome variables (e.g.,
  reading scores)
• Independent variable(s)- variable that is
  manipulated (the intervention or practice e.g.,
  reading curriculum)
• Context- defined clearly so replication can occur
  (e.g., K-3 school with 200 students, etc.)

27
Experimental Group vs Control Group

• Experimental (or treatment group)- receives the
  intervention
• Control group- business as usual.
• For true experimental research, participants
  are randomly assigned to each group.
• In order for to be considered random, every
  person must have an equal chance of being in
  either group

28
Experimental and quasi-experimental research
involves

• Direct manipulation of an independent variable
  (intervention)

29
Validity

• Refers to whether a study is able to
  scientifically answer the questions it is
  intended to answer.
• Extent to which your test (or study) measures
  what it intends to measure.

30
Internal Validity

• Changes observed in the dependent variable
  (outcome) are due to the effect of the
  independent variable (intervention).. not to
  some other unintended variables (extraneous,
  alternative explanations)
• 12 threats to internal validity (noted by
  Mertens, 2010)
• https//www.youtube.com/watch?v_UPUtlHDM0A

31
12 Threats to Internal Validity (think when
something other than the IV affect the results of
a study)

• History- events other than IV affected results?
• Maturation- changes in participants?
• Testing- participants became test-wise?
• Instrumentation- difference between pre- and
  posttests?
• Statistical regression- extreme groups used?
• Differential selection- groups differed in ways
  other than exposure to IV?

32
Threats to Internal Validity continued

• 7. Experimental mortality- drop outs of study?
• 8. Selection-maturation- was selection a problem
  based on the characteristics of the sample?
  (e.g., participants in one group may have been
  older)
• 9. Experimental treatment diffusion- treatment
  control groups shared information?
• 10. Compensatory rivalry by the control group
• 11. Compensatory equalization of treatments
  extra resources given to control group?
• 12. Resentful demoralization of the control group

33
External Validity (think generalizability)

• External Validity extent to which findings in
  one study can be applied to another situation.
• AKA ecological validity, generalizability
• 10 threats posed as questions (noted by Mertens,
  2010)

34
10 Threats (questions) to External Validity

1. Were the variables, context, and treatment
   described in sufficient detail?
2. Were multiple treatments used? Did they interfere
   with each other?
3. Was the Hawthorne effect (special attention as
   part of study affecting results) operating?
4. Was the treatment influenced by being novel or
   disruptive?
5. What was the influence of the individual
   experimenter?

35
Questions to external validity continued

• 6. Were the participants sensitized by taking a
  pretest?
• 7. Were the participants sensitized by taking a
  posttest?
• 8. What was the influence of the type of
  measurement used for the dependent variable?
• 9. Was there an interaction of history and
  treatment effects?
• 10. What was the influence of the time period
  that elapsed between the treatment and
  administration of the dependent variable?

36
Treatment Fidelityanother threat

• Treatment fidelity- implementer of the
  independent variable follows the exact procedures
  specified for administering the treatment(s).

37
Strength of treatmentanother threat

• May not be reasonable to expect participants to
  improve given the strength of the intervention.
  
• Intervention may not have been tried long enough
  or delivered with adequate intensity.

38
Coding system used for research designs

• R Random assignment of subjects to conditions
• XExperimental treatment (e.g., reading
  curriculum)
• O Observation of the dependent variable (e.g.,
  test or observation measure)

39
Single Group Designs

• One-shot case study
• X O
• Threats
• History, maturation, mortality (drop out)
• Other concerns using this design?
• No control group No pretest to know if it was
  the intervention that affected outcome.
• Very weak design

40

• One-Group Pretest-Posttest Design
• O X O
• Threats
• History, maturation,
• What would help control for these threats?
• Control group- both groups taking the tests at
  same time, but one not receiving the intervention
• But sometimes it is difficult to find a control
  group

41
Time Series Design

• Involves measurement of the dependent variable at
  periodic intervals.
• O O O O O X O O O O O
• If behavior is stable in baseline (before
  intervention), then change can be attributed to
  intervention.
• Controls for several threats
• Maturation, testing, differential selection (same
  persons involved)
• https//www.youtube.com/watch?vGUq_tO2BjaU

42
Experimental Designs

• Pre-test-Post-test Control Group Design
• R O X O
• R O O
• Controls for what threats?
• history, differential selection, mortality
  (pre-test can show differences in drop-outs)
• Posttest-only Control Group Design
• R X O
• R O
• Controls for what threats?
• Same as above, except for mortality (no pretest)

43

• Single-Factor Multiple-Treatment Designs
• X1 intervention 1
• X2 intervention 2
• .
• R O X1 O
• R O X2 O
• R O O
• Controls for threats because participants
  randomly assigned to comparison groups and
  pre-post-tests conducted.

44
Solomon 4-group Design

• If worried about pretesting affecting validity
• R O X O
• R O O
• R X O
• R O
• https//www.youtube.com/watch?v7RRVW4iO7gA

45
Quasi-experimental designs

• Similar designs to experimental designs, except
  for
• Lacking.
• Random assignment
• Cant make a strong causal statement
• https//www.youtube.com/watch?v_WpBxERGNVw

46
How do we know if a research study involves
rigorous, systematic and objective procedures?

• CEC-Division for Research
• Sponsored prominent researchers to author papers
  to propose
• Parameters for establishing that reported
  research has been conducted with high quality
  (quality indicators)
• Criteria for determining whether a practice has
  been studied sufficiently (enough high-quality
  research studies conducted on its effectiveness)
  and shown to improve student outcomes (effects
  are strong enough)

Graham, S. (2005). Criteria for evidence-based
practice in special education special issue.
Exceptional Children, 71.
47
Exceptional Children (2005) volume 71(2)

• Group Experimental and Quasi-Experimental
  Research (Gersten, Fuchs, Comptom, Coyne,
  Greenwood, Innocenti)
• Single-Subject Research (Horner, Carr, Halle,
  McGee, Odom, Wolery)
• Correlational Research (Thompson, Diamond,
  McWilliam, Snyder, Snyder)
• Qualitative Studies (Brantlinger, Jimenez,
  Klingner, Pugach, Richardson)

48
Quality Indicators (QIs) for Experimental (and
Quasi-Experimental) Research

• Describing Participants
• Sufficient information about participants and
  interventionists, selection procedures as well as
  comparability across conditions
• Implementation of Intervention and Description of
  Comparison Conditions
• Clear description of intervention (and comparison
  conditions) with implementation fidelity assessed
• Outcome Measures
• Use of multiple measures at appropriate times
• Data Analysis
• Analysis techniques appropriate to questions and
  unit of analysis with effect size calculated

49
Sampling Strategies

• Probability-Based
• Non-probability Based
• Examples???

50
Random vs Non-random sampling
Random Non-Random Sampling (Purposive)
All have equal chance independent chance Chosen based on criteria
Selects a representative of population No Equal chance
Should be large and random Some have no chance at all
No bias Some types show biases
51
Nonprobability Sampling

• Convenience
• Judgmental
• Quota
• Snowball
• https//www.youtube.com/watch?v-kwdXEXC7yE
• https//www.youtube.com/watch?v1n_6kT42jO8

52
Random Sampling Methods

• Simple Random Sampling
• Stratified Random Sampling
• Cluster Random Sampling
• Two-stage Random Sampling

53
Simple Random Sampling

• Each individual has equal independence chance
  of selection
• The larger the sample the more it represents the
  population
• Any differences is not due to bias
• https//www.youtube.com/watch?vyx5KZi5QArQ

54
Stratified Random Sampling

• Certain strata selected
• Sample in same proportion as they exit in the
  population
• Increases the likelihood of representativeness
• https//www.youtube.com/watch?vsYRUYJYOpG0

55
Cluster Random Sampling

• Ideal to include certain groups/cluster
• However at times it is not possible to select
  individual due to time, effort
• Select individuals based on (not individuals)
• Groups
• Clusters
• Subjects
• Ex All 5th Graders in Selected Schools (cluster)
  in Portland.
• https//www.youtube.com/watch?vQOxXy-I6ogs

56
Two-Stage Random Sampling

• Combination of Cluster Individual Random
  Sampling
• First select clusters randomly
• Then select students randomly from clusters
• https//www.youtube.com/watch?vQFoisfSZs8I
• https//www.youtube.com/watch?vx0XU0ngWpek

57
Statistics, statistics
Descriptive Statistics
Inferential Statistics
Who is in your data?
What your sample says about the population
sample
population
Mean, Median, Mode, Standard Deviation, Variance
Tests of significance (t-, F-Tests)
58
Tests of Significance

• Statistical analyses to determine whether a
  difference is statistically significant
  (probability for result to occur by chance).
• Yes or No answer
• Alpha level (p)
• An established probability level which serves as
  the criterion to determine whether to accept or
  reject the null hypothesis
• Common levels in education
• .01
• .05
• .10

Objectives 4.1 6.1
59
Statistics, statistics
Descriptive Statistics
Inferential Statistics
Who is in your data?
What your sample says about the population
sample
population
Mean, Median, Mode, Standard Deviation, Variance
Tests of significance (t-, F-Tests)
60
Inferential Statistics

• T tests- used when have two groups to compare.
• Independent samples t- if groups are independent
• Different people in each group
• Dependent samples t- if two sets of scores are
  available for the same people (e.g., pre and
  post-tests of same group)
• Matched groups
• ANOVA (analysis of variance)- when you have more
  than 2 groups to compare OR more than one
  independent variable (reports an F-statistic,
  which is basically a t-value squared)
• ANCOVA (analysis of covariance)- ANOVA that
  allows for control of the influence of an IV
  (e.g., characteristics of people) that may vary
  between your groups before treatment is
  introduced.
• Post-hoc method for matching groups on variables
  such as age, prior education, SES, or a measure
  of performance

61
Effect Size

• Way of quantifying the difference between two
  groups.
• Not just was there an effect, but the magnitude
  of the effect.
• Many ways to calculate
• ES Mean of experimental group Mean of
  control group/Standard Deviation
• R-squared, Cohens-D
• Standard deviation is how well the mean
  summarizes the data

62
(No Transcript)
63
(No Transcript)
64
Visible Learningby John Hattie

• Over 800 Meta-analyses
• https//www.youtube.com/watch?vsng4p3Vsu7Y

65
Article Review Use questions as your guide

• Implications for your practice- why you selected
  the article
• Introduction- what was the aim of the article,
  what were the specific research questions
• Methods- Describe
• Participants
• Setting of study
• Target behaviors- dependent variables
• How they measured the dependent variables
• Materials used
• Intervention procedures used- specify
• Were the procedures replicable?

66
Article Review more questions

• Methods used for treatment integrity
• This asks how well does the author measure the
  implementation of the intervention
• What was the design?
• Experimental, quasi-experimental, single-subject
  (specify what type of designmultiple baseline,
  etc.)
• Results- give quantifiable information of how the
  intervention worked
• Discussion- issues discussed, limitations?
• Conclusion- did they answer the research
  question?
• Reflection- commentary, questions

67
In-Class Activity

• Get together with a partner.
• Practice completing an article review for one of
  the articles you read this week.
• Note the design of the study.

68
Proposal Assignment Group Work

• A detailed explanation of the assignment is
  posted on the wiki
• What should you be doing in your groups?
• At this point you should have a topic and start
  coming up with your framework for your research
  project (based on literature).
• Start to draft your conceptual framework,
  research questions identify your dependent and
  independent variables
• You should walk away from your group time with a
  list of tasks to complete.

69

• Socially Important Issue
• 2. Conceptual Model/Hypothesis
• 3. Research Question(s)
• 4. Dependent Variable
• 5. Dependent Variable Measure
• 6. Independent Variable
• 7. Independent Variable Measure
• 8. Research Design

70
Reflection feedback survey