Title: Methodology Matters: Doing Research in the Behavioral and Social Sciences
1Methodology MattersDoing Research in the
Behavioral and Social Sciences
- ICS 205
- Ha Nguyen
- Chad Ata
2The way we do research matters
- Because the meaning of the discovered evidence is
based on the way it is obtained - 3 domains are always involved when doing research
- Substantive ? content of interest
- Conceptual ? ideas that give meaning to the
content - Methodological ? techniques for conducting
research - 3 important keywords
- Relations which are between elements within a
context or embedding system
3The Substantive Conceptual Domains
- Substantive Domain
- Elements Phenomena
- Relations Patterns of phenomena
- Studying the actors behaving towards objects in
context - Conceptual Domain
- Elements Properties of the actors behaviors...
- Relations Interconnections between elements
4The Methodological Domain
- Elements methods Modes of Treatment
- Includes techniques for
- Measuring
- Manipulating
- Giving instruction to participants
- Imposing constraints on the environment
- Selecting materials for use
- Giving feedback about prior performances
- Using experimental confederates
- Controlling
- Distributing impact
5 Methodological Domain
- Relations Comparison Techniques
- Involve 3 features of the system under study
- Dependent variables or features
- These are measured
- Independent variables or features
- These are manipulated
- All of the rest of the systems features
- These are relevant but not the focus of the study
- Comparison performed to assess covariation or
association between dependents and independents
6The Domains
7Research Methods Are Like Tools
- Each method offers opportunities not available by
other means - Each method also has inherent limitations
- Therefore you should use multiple methods in
order to counterbalance the flaws in each
individual method - Make sure to chose methods such that one strength
can compensate for anothers weakness
8The Research Methods
Abstract
Precision Maximum
II
I
III
Unobtrusive
Obtrusive
IV
Realism Maximum
Generalizability Maximum
Concrete
9Quadrant I Field Strategies
Abstract
Precision Maximum
II
III
Unobtrusive
Obtrusive
IV
Realism Maximum
Generalizability Maximum
Concrete
10Quadrant I Field Strategies
- Field Study
- Natural unobtrusive
- Field Experiment
- Manipulating one variable in an otherwise natural
system ? decrease in realism - A bit obtrusive
11Quadrant II Experimental Strategies
Abstract
Precision Maximum
I
III
Unobtrusive
Obtrusive
IV
Realism Maximum
Generalizability Maximum
Concrete
12Quadrant II Experimental Strategies
- Laboratory Experiment
- Concocted system
- Increased precision
- Decreased realism generalizability
- Experimental Simulation
- Similar to lab experiment, but
- strives to increase realism
13Quadrant III Respondent Strategies
Abstract
Precision Maximum
II
I
Unobtrusive
Obtrusive
IV
Realism Maximum
Generalizability Maximum
Concrete
14Quadrant III Respondent Strategies
- Sample Survey
- Sampling of the population ? high
generalizability - Low precision
- Judgment Study
- Here the focus is on the stimulus material,
rather than respondents attributes ? High
precision - actors of convenience ? Low generalizability
over the population - Both often done in neutral settings ? low realism
15Quadrant IV Theoretical Strategies
Abstract
Precision Maximum
II
I
III
Unobtrusive
Obtrusive
Realism Maximum
Generalizability Maximum
Concrete
16Quadrant IV Theoretical Strategies
- Formal Theory
- Formulating general relations between variables
of interest ? high generalizability - No concrete system involved ? low in realism
- No observation ? low precision
- Computer Simulation
- Like experimental simulation, except human
participants are not included - Modeling the real world ? high realism
- Outcomes are predictable ? low precision
- Modeling a particular system ? low
generalizability
17Questions to keep in mind
- Does the presented material coincide with the
strengths weaknesses of the research strategies
that were used? - Is the research evidence well supported by a
single or by multiple research strategies? - Are the strategies weaknesses and strengths
offset such that they counterbalance one another?
18Study Design
- For every empirical study, observations have to
be gathered, combined, and compared - Ex Given a pair of properties, X and Y, can we
tell if X caused Y? - X is independent variable, Y is dependent
variable - What techniques are available for comparison?
- Baserates
- Correlation
- Difference
19Comparison Technique 1
- Baserates
- How often (at what rate, or what proportion of
time) does Y occur? - If I do not know how often Y occurs in the
general case, then I cannot decide whether the
rate of Y in the study is significant
(particularly high or low).
20Comparison Technique 2
- Difference Question
- Is Y present under conditions where X is present?
Is Y absent when X is absent?
21Comparison Technique 3
- Correlational Question
- Do the values of property X covary with the
values of property Y? - If X is high, is it likely that Y will also be
high? - If X is low, is it likely that Y will also be
low? - High positive correlation, high negative
correlation, little or no correlation - This technique tells you whether X and Y go
together but it does not help in determining
causality.
22Randomization
- Randomization using a random assignment
procedure to allocate cases to conditions - Each case must be equally likely to end up in any
given condition - This procedure does not guarantee to rule out all
other factors that could affect the results.
23Validity of Findings
- 4 types of validity
- Internal validity
- How close can you come to asserting that the
presence of X caused the altered level of Y? - Statistical conclusion validity
- Is the given result due to chance?
- Construct validity
- How well defined are the theoretical ideas in the
study? - External validity
- How well will the results hold up for replication
and how generalizable are the results?
24Measure Techniques
- Categorized by two questions
- Who makes the record of the behavior?
- Is the participant aware that his/her behavior is
being recorded for research purposes?
256 Major Types of Measures
26Strengths and Weaknesses of the Measures
- Self-reports
- Strength - Low costs, does not require a lot of
time to create and use - Weakness - Reactive
- Trace measures
- Strength unobtrusive
- Weakness not very versatile, time consuming to
gather
27Strengths and Weaknesses of the Measures
- Observations
- Strength relatively versatile
- Weakness can only be used on overt behavior,
vulnerable to observer errors, can be reactive
for visible observers, possible ethical issues
for hidden observers - Archival records
- Strength good for researching the past, studies
for extensive periods of time - Weakness public records can be reactive,
versatility is limited
28Techniques for Manipulating Variables
- Selection selecting cases with desired values
and assigning them to appropriate conditions - Direct Intervention manipulating the specific
variable directly
29Techniques for Manipulating Variables
- Inductions
- 3 types
- Misleading instructions
- False feedback
- Experimental confederates
- Ethical issues
30Concluding Comments
- Results depend on methods. All methods have
limitations. Therefore, any set of results is
limited. - It is not possible to maximize all desirable
features of different methods in any one study. - Each set of results must be interpreted in
relation to other evidence relating to the same
questions.