Qualitative Research

1
Qualitative Research

• What is it?
• The study aiming to gather an in-depth
  understanding of human behavior and the reasons

2
Qualitative Research

• Overview
• Examination, analysis and interpretation of
  observations
• Smaller but focused samples are more often needed
• Produce information only on the particular cases
• Informative guess (Hypothesis)

3
Qualitative Research

• Data Collection
• Observation
• Interviews
• Case studies
• Grounded theory
• Survey

4
Qualitative Research

• Example(s)
• Semi-structured interview question
• What kind of words that you would like to
  use to describe your playing experience with this
  game ?
• Survey question

5
Qualitative Research

• Data Analysis
• Interpretive techniques
• Coding
• Recursive abstraction
• Mechanical techniques

6
Qualitative -gt Quantitative

• Example(s)
• A 5-Point Likert item

• Not Very Effective (1) or (-2)
• Not Effective (2) or (-1)
• Somewhat Effective (3) or (0)
• Effective (4) or (1)
• Very Effective (5) or (2)

Which Likert Scale method is better?
Social desirability bias
7
Quantitative Research

• What is it?
• A systematic empirical investigation of
  quantitative properties and phenomena and their
  relationships.

8
Quantitative Research

• Overview
• The process of measurement is central to
  quantitative research because it provides the
  fundamental connection between empirical
  observation and mathematical expression of
  quantitative relationships
• Quantitative methods can be used to verify which
  hypotheses are true

9
Quantitative Research

• Overview
• The generation of models, theories and hypotheses
• The development of instruments and methods for
  measurement
• Experimental control and manipulation of
  variables
• Collection of empirical data
• Modeling and analysis of data
• Evaluation of results

10
Quantitative Research

• Data Analysis
• Statistical Analysis !
• Analysis of variance (ANOVA)
• Chi-square test
• Correlation
• Factor analysis
• Regression analysis
• ...

11
Quantitative Research

• Data Analysis
• Example Chi-square test
• Chi-square test of independence allows us to
  determine whether or not two variables are
  associated in some way.

12
Quantitative Research

• Data Analysis
• Example Chi-square test
• Let's say we want to know if the person's
  political affiliation (democratic/republican/indep
  endent) is associated with his or her views on a
  flat income tax (Flat Tax). We've asked a random
  sample of 180 residents their opinion on Flat Tax
  and their political affiliation. First we can
  record the results in
• a contingency table.

13
Quantitative Research

• Data Analysis
• Example Chi-square test

14
Quantitative Research

• Data Analysis
• Example Chi-square test
• Expected frequency of each cell Row total X
  column total / n
• df (r 1) x (c 1)

15
Quantitative Research

• Data Analysis

16
Quantitative Research Experimental Design

• Example(s)
• Case You are contacted by the clinical
  director of a local community mental health
  center. Her staff has been developing what she
  thinks is a promising new therapeutic regimen for
  depression, and she would like you to design a
  study to evaluate its effectiveness. The center
  has approximately 100 to 120 clients with
  diagnoses of major depressive disorder, and in
  anticipation of the implementation of this new
  treatment program, all of them have recently been
  assessed using several measures, including the
  Beck Depression Inventory (a widely-used
  self-report measure of depression) and the
  Hamilton Depression Scale (a measure of symptom
  severity). The clinical director is particularly
  concerned about the ethical implications of
  research of the kind she is asking you to do,
  because it entails deciding that some of the
  center's clients will be chosen to not receive
  what she thinks is likely to be an effective
  therapy.
• Taking all this into account, design a
  research program that would address as many of
  the client's concerns as possible. What specific
  research design would you use? What makes it the
  optimal design among those available? How would
  it be implemented, e.g. assignment of
  participants to conditions, measurements, etc.?

17
Qualitative Research

• Example(s)
• My solution Quasi experiment, and use RD
  (Regression Discontinuity) as the primary method.
  The main reason is that the researchers is not
  entirely sure the clients will be chosen or not
  receive the therapy (treatment) in other words,
  we may lose the ability to do random assignment
  in this case, and it will undermine the validity
  of upcoming results. However, a good thing is
  that those potential participants have recently
  been assessed by some measures such as Beck
  Depression Inventory, because we can treat it as
  an assignment variable, and determine a "cutoff"
  value from those scores. Afterward, we can apply
  pre-test and post-test experimental study.

18
Internal Validity

• What is it?

The approximate truth about inferences regarding
cause-effect or causal relationships
19
Internal Validity

• Other types of validity

• External validity more concerns on
  Generalizability than internal validity
• Construct validity Can live with or without
  internal validity

20
Internal Validity

• An example

In a Post-test only experiment, group A that
receive math tutorial program performs better
than group B that does not.
An key question
whether observed changes can be attributed to
your program or intervention (i.e., the cause)
and not to other possible causes (sometimes
described as "alternative explanations" for the
outcome).
21
Internal Validity

• Single group threat

• History Threat
• Maturation Threat
• Testing Threat
• Instrumentation Threat
• Regression Threat

22
Internal Validity

• Multiple group threat A multiple-group design
  typically involves at least two groups and
  before-after measurement.

• There really is only one multiple group threat to
  internal validity that the groups were not
  comparable before the study - selection bias
• prior group differences?

23
Internal Validity

• Social interaction threat

• Diffusion or Imitation of Treatment
• Compensatory Rivalry
• Resentful Demoralization

24
Thank you

• And questions for us?