Empirical Studies

1
Empirical Studies
2
Outline

• Science
• Types of Studies
• Methods
• Constraints

3
Process of Science
Theory
World
Models
Hypotheses
Research Design
Research Questions
Research Results
4
Types of Studies

• Quantitative
• Qualitative

5
Qualitative Methods

• Methods are considered qualitative because they
  usually involve a judgment, subjective
  assessment, by the observers.

6
Uses for Qualitative Methods

• In-depth understanding of people
• Understanding of culture
• Understanding viewpoint/perspective of those in
  the culture
• Insiders view

7
Quantitative Methods

• These methods assume there are measurable
  properties that are accessible and whose
  manipulation will provide different levels of
  results.

8
Uses for Quantitative Methods

• Most quantitative methods seek to either help
  determine how different measurable features
  influence other factors or to establish the
  degree to which factors influence other factors.

9
Caveat (1)

• Computer science and software engineering are
  newcomers to the games of science. Much research
  needs to be done prior to establishing a firm
  quantitative foundation for computer science and
  software engineering.

10
Caveat (2)

• Yet, some argue that computer science, as a
  science, should rely heavily on the quantitative
  methods. There are things to be learned from
  both qualitative and quantitative approaches to
  research.

11
Not everything that can be counted counts and not
everything that counts can be counted. Albert
Einstein
12
Methods

• Formal Experiment
• Case Study
• Survey

13
Formal Experiment

• Subjects are assigned to treatment groups to
  perform a task using methods of tools.
• Experiments attempt to control the variability in
  the environment so conclusions can be drawn
  regarding the effectiveness of the treatment.

14
Experiments

• In vitro (laboratory)
• In vivo (experimental projects)
• Ecological validity

15
Case Study

• Typically seen on real projects. Data is
  collected regarding various tasks. This can be
  compared to baseline information if it is
  available.

16
Survey

• Typically used to gain information based on
  individuals retrospective reports. Individuals
  who have engaged in certain tasks are asked for
  their perceptions.

17
Constraints

• Cost
• Time
• Population/Sample
• Control