Interpreting Empirical Research Results

1
Interpreting Empirical Research Results

• A Tutorial

2
Agenda

• What is Research?
• What is a Theory?
• Reading Research Papers
• Research Criteria
• Sampling Considerations
• Caveat Emptor

3
What is Research?

• Directed
• Purposive
• Collection of Data
• To shed light on
• A theory

Executed according to a plan
Goal-oriented, with measurable objectives
For empirical research, data are used to see if
theory is true
A sort of stress test for theories if theory
doesnt hold water, then its false.
A set of statements about reality, somewhat
simplified, perhaps saying that X causes Y
4
Hypothetico-Deductive Method
Realm of Ideas and Thoughts
Theory
Reality
Focus on the ideas
Focus on the test
5
Reading Research Papers

• What is the theory?
• What is the model?
• What are the data about?
• Is the research method or procedure trustworthy?
• Is the sample used representative?
• How can results be applied?
• Is the research ethical?

6
Theorizing
Realm of Ideas and Thoughts
Theory
Reality
A Theory is a set of statements that simplify
reality while saying something about its
structure. It is a link between the realm of
ideas and reality. There are many kinds of
theories
7
Kinds of Theories

• Pictorial an image that stands for reality
• Descriptive a set of dimensions that distinguish
  various elements or phenomena
• Associational a set of statements that tell what
  phenomena appear or go together
• Causal a set of statements that say that some
  phenomena cause other phenomena
• Implicational a set of statements that say that
  some phenomena imply other phenomena
• Operational a set of conditions that make things
  happen or cause parts of reality to reappear.

8
Crafting The Hypotheses
Realm of Ideas and Thoughts
Theory
The research hypotheses are logical implications
of the theory. If the theory is true, the
hypotheses should be true. A real-world test of
the hypotheses tries to see if the hypotheses are
false in reality. If they turn out to BE false,
then the theory they are based on must also be
false.
Reality
9
Conducting The Test
Realm of Ideas and Thoughts
The real-world test should stress the
hypotheses strongly. To the extent that the test
tries to prove the hypotheses false, and fails to
do so, then we retain our belief that the
hypotheses (and hence the theory the hypotheses
are based on) are true. If the test is TOO
strong, then ANY hypothesis might seem false and
we dont trust the TEST.
Reality
10
Interpreting the Data
Realm of Ideas and Thoughts
Theory
Data arising from the real-world test are
analyzed and the researcher reviews experiences
in the real world test and revises the ideas in
the theory (or rejects the theory altogether)
Reality
11
Research Criteria

• Validity are the results really about the
  research question? Are the data about pheonomena
  the research question is about?
• Reliability are the results replicable? Are
  they generalizable to a larger set of situations?

12
Sampling Considerations

• Is the sample representative?
• How was the sample obtained?
• What was the sampling frame?
• Was the sample treated ethically?

13
Hypothetico-Deductive Method
Realm of Ideas and Thoughts
Theory
The results are either (1) test failed to show
theory is false or (2) test showed theory to be
false. In the case of (1), there is a question
of whether the test was strong enough. In the
case of (2) the question is whether or not the
test was TOO strong. An unwanted result is (3)
test neither showed nor failed to show test to be
false. NOTE test cannot show theory to be true,
in general.
Reality
14
Caveat Emptor