Experimental Research I - PowerPoint PPT Presentation

1 / 34
About This Presentation
Title:

Experimental Research I

Description:

Experimental Research I Day 3 – PowerPoint PPT presentation

Number of Views:166
Avg rating:3.0/5.0
Slides: 35
Provided by: P018
Category:

less

Transcript and Presenter's Notes

Title: Experimental Research I


1
Experimental Research I
  • Day 3

2
Business
  • For Tomorrow
  • One article from Day 2 reading list.
  • Prepare to summarize comment
  • Examine 1 Thesis proposal online
  • List of 12 research articles in APA format
    related you your study (bibliography)
  • Reference List APA example
  • Kotora, E. J. (2005). Assessment practices in the
    choral music classroom A
  • survey of Ohio high school choral music teachers
    and college choral methods professors.
    Contributions to Music Education 32(2), 65-80.
  • Spaces b/w reference elements 1 b/w sentences
    in text 2.

3
Experimental Research
  • Only type of research with an intervention
  • A direct attempt to influence a particular
    variable
  • Only method that can truly begin to untangle
    cause and effect hypotheses
  • Directional Hypothesis Theory statement
    predicting the outcome directional (There will
    be a significant difference). Reflects
    researchers expectations.
  • Bilingual 3rd graders taught with the Kodaly
    method will demonstrate significantly higher
    musical achievement than bilingual 3rd graders
    taught with a traditional eclectic method.

4
Null Hypothesis
  • Null Hypothesis Theory statement predicting the
    outcome stated in the negative non-directional
    (There will be no significant difference) The
    statistical hypothesis that states that there are
    no differences between observed and expected
    data. Does not reflect researchers expectations
    (value free)
  • There will be no significant difference in
    musical achievement of bilingual 3rd graders
    taught with the Kodaly methods and bilingual 3rd
    graders taught with a traditional eclectic
    method.
  • The goal is to REJECT the Null Hypothesis based
    (on 95 Confidence level or above)
  • Cannot prove the null hypothesis (a negative)
  • E.g. not guilty does not innocent
  • Reject/not reject vs. accept

5
Type I and Type II Error
  • Type I Error is erroneously claiming statistical
    significance or rejecting the null hypothesis
    when in fact, its true (claiming success when
    experiment failed to produce results)
  • Possible w. incorrect statistical test
  • Type II Error is when a researcher fails to
    reject the null hypothesis when it is in fact
    false (claiming failure when successful)
  • The smaller the sample size, the more difficult
    it is to detect statistical significance
  • In this case, a researcher could be missing an
    important finding because of study design

6
Group Comparisons
  • Experimental Group
  • Receives a particular treatment specified by the
    researcher
  • Control/Comparison Group
  • Does not receive that particular treatment
  • Sometimes difficult in educational research to
    have a strict no-treatment, control
  • Example Any instruction is likely to be more
    effective than no instruction

7
Randomization
  • Random assignment to groups
  • Every individual has an equal chance of being in
    the experimental or control/comparison group
  • Supposed to help eliminate extraneous sources of
    variance
  • For example if the groups are sufficiently
    large, any differences between groups on
    extraneous variables are likely to be due to
    chance or randomly distributed among the groups
  • Quasi-Experimentalnon-randomized groups
  • Most ed. research
  • Intact classes convenience samples
  • Impacts ability to generalize to whole population

8
Variables
  • Independent variable (IV)
  • The experimental or treatment variable
  • This variable is manipulated by the researcher
  • Examples instructional approach, environmental
    condition, the introduction of a particular
    musical element
  • Participant attribute
  • Dependent variable (DV) Compared b/w groups
  • The criterion or outcome variable
  • Examples student attitudes, student achievement,
    teacher effectiveness as measure by ?
  • Experiments can be expressed as The effect of
    the IV on the DV
  • Extraneous Variables
  • Those that are not specifically included in the
    study but never the less may effect the outcome
  • Object is to control for extraneous variables
  • The researcher may not know them all

9
Manipulating the IV
  • Presence of the variable vs. absence of the
    variable
  • Kodaly instruction (treatment group) vs. no
    Kodaly instruction (control group)
  • One form of the variable vs. another
  • modeling vs. verbal music instruction (vs.
    control group?)
  • Varying degrees of the same variable
  • 100 positive feedback, no negative feedback vs.
    50 positive feedback, no negative feedback

10
Controlling for Extraneous Variables
  • Best case scenario all individuals are as
    similar as possible on all variables other than
    the independent variable
  • Methods to control
  • Randomization large sample
  • Holding variables constant (freeze private
    lessons)
  • Build variable into the design (compare private
    lessons w/ no private lessons)
  • Matching pairs one to control, other to exper.
  • Statistical control analysis of covariance
    (ANCOVA)

11
Design and ExperimentEffect of Colored note
heads on Music Reading
  • State Hypothesis and Null Hypothesis
  • Select sample and assign to group (control and
    treatment). How many in each?
  • Identify independent and dependent variables. Any
    possible extraneous variables?
  • Describe experiment. What will you do w/ each
    group and for how long? How will you know what
    they already know?

12
Discussion of Projects
  • On task
  • Practice explaining your project
  • Background State the problem
  • Purpose statement
  • Research questions
  • Methodology (research design)

13
Experimental Research Designs
14
Nomenclature/Abbreviations
  • When looking at the symbols used to describe
    various experimental design approaches
  • R random assignment
  • O testing (pre- or post-)
  • X treatment
  • C control/comparison
  • M matched

15
Pre-Experimental DesignsPilot Studies
Generally Weak
  • One Shot Case Study (X O)
  • No random assignment, No control/comparison, no
    pre-test
  • One-Group Pre-test, Post-test (O X O)
  • No random assignment, No control/comparison group
  • Static/Intact-group Comparison X O
  • No random assignment O
  • Static/Intact-group Pre-test, Post-test
  • No random assignment, possible pre-test effects O
    X O
  • O

16
True Experimental DesignsStronger not always
possible in educ.
  • Randomized Post-test Only, R X O
  • Control Group R O
  • Still not sure about pre-test levels
  • Randomized Pre-Test, R O X O
  • Post-test, Control Group R O O
  • Good for checking if groups are actually similar
    at the start of the study and possible effects of
    pretest
  • If you do an experiment probably this one
    except w/ non-randomized groups

17
Randomized Solomon Four-Group Design
  • Solomon 4Group controls for possible
    sensitization effects due to testing or
    maturation.
  • 1. R O X O
  • 2. R O O (maturation or pretesting?)
  • 3. R X O (effect of pretest?)
  • 4. R O (control group)
  • In a successful experiment, what would we expect
    for each group?
  • What if the Post Test scores for group 2 were as
    high as the Post Test for group 1?
  • What if the Post Test scores for group 3 were
    lower than group 1?
  • What if the Post Test scores for group 4 were the
    same as groups 1 2?

18
Quasi-Experimental Design
  • So called b/c there is no randomization
  • Matching Only
  • Participants matched in pairs to control for an
    extraneous variable rather than randomly assigned
  • Equivalent Materials Design (next slide)
  • Counterbalanced Design (2 slides down)
  • Multiple groups receive all treatment types in
    different order
  • Average post-test scores across groups are
    compared to determine effectiveness/effect of the
    treatment order
  • Vulnerable to multiple-treatment interference
  • Time-series Design
  • Outcome measured several times before and after
    introduction of the treatment O O O O X O O O O

19
Equiv. Mat. Design TuningTuba vs. Clarinet vs.
oboe vs. drone stimulus
  • Band (N 1) OX(tu)O OX(clar)O OX(ob)O
    OX(drone)O
  • Pre/Post tests indiv. tuning to different
    stimuli. Success mes. w/ a tuner.
  • Research questions Which tuning process leads to
    most growth?
  • Most interested w/ growth w/i group per treatment
  • Other questions Could introducing drone second
    be more effective? How do we know that using just
    one would be just as or more effective? next
    slide

20
Counterbalanced Design(Latin Square) Order effect
  • All groups take a pretest tuning to the piano
    A440 no difference in groups after time 1,
    post test serves as next pretest
  • Each treatment 2 weeks w/ post-test at the end
  • 1 tuba 2 clarinet 3 oboe 4 drone

21
Quasi-Experimental Design
  • Factorial Design (Manipulate 2 or more factors
    IVs at different levels)
  • Allows for examination of attribute (vs.
    manipulated) variables (i.e. gender, age) and
    interaction effects b/w combinations of IVs
  • Example Effect of teaching method on beat
    competency of ELL and non-ELL students
  • Possible outcome showing interaction of two IVs
  • Non-ELL students may do equally well w/ Kod. and
    Gordon methods, while bilingual students may do
    better w/ Kod. vs. Gordon.
  • What if you had not separated these groups out?

22
Factorial Example(Two Way - 2x2)
  • Beat competency improvement using Gordon or
    Kodaly among biling. and non-biling students
  • IVs Language classification (biling. vs.
    non-biling.) method (Kodaly vs. Gordon)
  • DV rhythm pre- post-tests
  • Groups (Six 3rd gr. Sections-3 Kodaly 3 Gordon
    Bilingual Non-Bilingual in all groups.)
  • Bilingual Kodaly
  • Non-Bilingual Kodaly
  • Bilingual Traditional
  • Non-Bilingual Traditional

23
2 Way Factorial Designs (2 independent variables
often one manipulated, one attribute)
24
Internal Validity - Effectiveness of Exp.
DesignControl of Extraneous Variables Time
Bound Factors
  • What happens within the experiment
  • History What happens b/w pretest and posttest
    (private lessons, change in practice routine)
  • Maturation is change result of treatment
    natural result of repetition and improvement over
    time?)
  • Mortality Loss of participants may cause
    imbalance b/w groups

25
Internal Validity Control of Extraneous
Variables Sampling Measurement Factors
  • Testing pretest affect posttest. Ceiling and
    floor effects (eliminate outliers?)
  • Instrumentation changes in measurement or
    observers (judges at contest from one site to the
    next)
  • Statistical regression students who score
    extremely high (ceiling) or low (floor) on
    pretest may regress to the mean on posttest
  • Selection participants do not represent normal
    population (also affects external validity)
  • Interactions influence of a combination of the
    above factors

26
External Validity Generalizability
  • Population Validity
  • Extent sample is representative of the population
    to which the researcher wishes to generalize the
    results.
  • Ecological
  • Study conditions and setting are representative
    of the setting in which the researcher would like
    to apply the findings (e.g. university lab
    school)
  • Replication
  • Results cannot be reproduced (problem w/ Mozart
    effect)
  • Detailed description of the sample needed in
    study
  • Important regardless of sampling method

27
Other Threats to External Validity
  • Effect or interaction of testing (testing will
    not occur in natural setting
  • Reactive effects of sample
  • Hawthorne Effect
  • Effects due simply to subjects knowledge of
    being in a study
  • John Henry Effect
  • Control group performs beyond usual level because
    they perceive they are in competition with the
    experimental group
  • Teacher or Researcher interactions different than
    in population
  • Subconsciously encouraging or discouraging a
    group

28
Review Effect of Intensive Instruction on
Elementary Students Memory for Culturally
Unfamiliar Music (2013)
  • Previous researchers have found that both adults
    and children demonstrate better memory for novel
    music from their own music culture than from an
    unfamiliar music culture. It was the purpose of
    this study to determine whether this
    enculturation effect could be mediated through
    an extended intensive instructional unit in
    another cultures music. Fifth-grade students in
    four intact general music classrooms (two each at
    two elementary schools in a large U.S. city) took
    part in an 8-week curriculum exclusively
    concentrated on Turkish music. Two additional
    fifth-grade classes at the same schools served as
    controls and did not receive the Turkish
    curriculum. Prior to and following the 8-week
    unit, all classes completed a music memory test
    that included Western and Turkish music examples.
    Comparison of pretest and posttest scores
    revealed that all participants (N 110) were
    significantly more successful overall on the
    second test administration. Consistent with
    previous findings, participants were
    significantly less successful remembering items
    from the unfamiliar music culture, a result that
    was consistent across test administrations and
    between instruction and control groups. It
    appears that the effect of enculturation on music
    memory is well established early in life and
    resistant to modification even through extended
    instructional approaches.

29
Identify or State
  • Independent Variable
  • Dependent Variable
  • Treatment Group
  • Control Group
  • Diagram experimental design (O X)
  • Write a hypothesis null hypothesis
  • Paraphrase findings
  • Implications for the classroom? Did the authors
    reject or not reject the null hypothesis?

30
Sampling
31
Samples of individuals/entities
  • Sample vs. population
  • Some vs. All
  • Examples where entire population could be
    sampled?
  • Relationship between sample specificity and
    generalizability
  • Representative sample
  • Captures relevant and essential characteristics
    of the population
  • What about a sample of teachers? What should the
    sample look like?

32
Sampling Methods
  • Systematic
  • Random start and sampling interval
  • i.e., Randomly select pages from IHSA directory
  • choose every ? Name (random number b/w 1-X)
  • Convenience
  • not as valuable but frequent in ed. research
    why?
  • i.e., intact classes, pre-service teachers from
    one institution, conference session attendees
  • Purposive
  • Participants fit a particular profile (female
    band directors in small towns)
  • Exclude those who do not fit profile
  • Often consists of volunteers (problematic)

33
Types of Samples
  • Simple Random
  • Everyone has equal chance of selection
  • Reduce systematic bias error created by
    sampling method
  • Phone book, MENC membership list (But??)
  • Stratified Random
  • Similar proportions between sample and population
  • Gender, race, age, instrument, etc.
  • Cluster Random
  • Groups rather than individuals
  • i.e., classes or ensembles in CPS
  • Then groups can be assigned randomly
  • Two-stage random - groups then individuals
  • i.e., choose classes then assign individual
    students or groups to control or treatment group

34
Sample Size
  • As large as possible given reasonable expenditure
    of time and energy
  • Most likely to get significant results
  • More statistically powerful (more likely to find
    a significant difference b/w groups)
  • Sample size relative to
  • the size of population (50 Cook Co. band
    directors vs. 50 band students throughout US)
  • variability within population (years of teaching,
    gender, etc.)
  • sampling method (need a large enough pool from
    which to draw)
  • study design (qualitative vs. quantitative)
Write a Comment
User Comments (0)
About PowerShow.com