Modeling Political Phenomena

1
Modeling Political Phenomena

• Using Control Variables and Gauging Validity

2
Face Validity

• Face validity means the measurement of a concept
  is consistent with an agreed definition.
• It does not mean however that this is the best
  measurement to capture the concept.

3
Construct Validity

• construct validity The match between the land of
  theory and the land of observation
• How effective do our operationalized variables
  represent the mental image of a concept into the
  public manifestation of that world?

4
Reliability

• We may want, or need to, test for reliability,
  which is to ask if our variables consistently
  provide the same results.
• To be sure we can measure/test repeatedly or even
  use multiple measures for the same variable. For
  development we can also use GDP per capita
  besides energy consumption per capita.

5
Key question of Internal Validity

• When we test a hypothesis and either accept it or
  reject it, how do we know that we made the right
  decision?
• What about alternative explanations that we did
  not account for?
• What should we do to gain confidence?

6
Internal Validity

• Are there other causes for what I am observing?
  
• If so, a study will lack internal validity if it
  cannot rule out plausible alternative
  explanations.

7
Internal Validity of a Study
Is the relationship causal between...

• What you measured and what you saw?
• Your program and your observations?

Alternative cause
Alternative cause
Program
Observations
Program-outcome Relationship
What you do
What you see
Alternative cause
Alternative cause
Observation
In this study
8
The Purpose of Control Variables

• We use control variables to account for possible
  alternative explanations we can think of.
• For example, when I examined whether democracies
  are generally more peaceful than autocracies I
  included several control variables.

9
Explaining Pacifistic Democracy

• Peace (Y) Democracy (X1) State Power (X2)
  Development (X3) of Bordering States (X4)
• In the model above, I have more confidence that
  Democracy is related to peace considering I
  control for the other variables that may skew my
  test.

10

• We need to take care that our theory is not
  missing other factors that may undermine the
  validity of our theory and tests.
• Our inferences will be flawed if we are actually
  capturing other processes through our variables.
• This means that the validity of our measures
  would be undermined.

11

• Several possible problems arise that are related
  to model misspecification and spurious
  relationships.
• Thus, we need to control for confounding factors
  and alternative explanations!!!

12
Model Misspecification and Spuriousness

• Antecedent variable A variable that indirectly
  affecting the relationship between two other
  variables.
• For example, Ivy league education increases
  income.
• However, parental wealth and legacy admissions
  affect Ivy league education. Thus, income of
  graduates from Ivy League schools may not be
  random.

13

• Here Ivy League Parents is an antecedent variable
• Ivy League Parents Ivy League Kids
  high income kids
• Hence, admission to Ivy schools clearly not
  random or pure merit-based, and thus the income
  earned by these people.

14
Model Misspecification and Spuriousness

• Intervening Variable These may be spuriously
  related to another relationship.
• How can states fight each other if they are not
  contiguous with each other? Only the strongest,
  with large navies, bases, etc., could do so.
• Hence, geographic contiguity or distance is an
  intervening variable. States may or may not be
  more peaceful, but it is hard to avoid conflict
  when it is on your borders.

15
Model Misspecification and Spuriousness

• Alternative Variables We also want to control
  for variables that would bias our results if
  omitted.
• In this case, the X variables in a model would
  produce biased estimates, undermining their
  validity and producing error that leads to
  inaccurate inferences.

16
Here is a spurious relationship from my research

• IGOs conflicts
• Powerful states
• Powerful states both in more IGOs and conflicts,
  but these two variables not directly related but
  a function of state power.

17
Classic Spurious Case
???
Ice Cream Consumption
Crime



Summer Temperatures
Hence we see that despite the fact that ice cream
consumption is correlated with crime, the real
cause is that summer temperatures increase both
ice cream consumption and crime.
18
Assessing your knowledge

• If your scientific study has taken care to make
  sure that your variables are measured correctly,
  used the appropriate control variables, and used
  proper tests, then what is next?

19
Conclusion Validity
Is there a relationship between...

• What you did and what you saw?
• Your program and your observations?

Program
Observations
Program-outcome Relationship
What you do
What you see
Observation
In this study
20
Group Work

• Identify the level on which variables are
  measured.
• Identify problems of construct validity, internal
  validity, and biased samples

21
External Validity

• Now that you are confident of what you found in
  your study, how well does my study or sample
  relate to the general population?In other
  words, how strong is my study able to generalize
  to other cases?

22
Research Designs and Sampling
In most studies what is examined are some cases,
not an entire population. For example, in
Presidential Election polls not every voter is
asked how they will vote but still polls can be
very accurate. How does that happen?
23
Population vs. Sample

• Research in the social sciences typically uses
  sampling methods.
• We draw a sample of subjects from a greater
  population.
• We then draw an inference from the sample about
  the greater population.
• In other words, we are generalizing about a
  population from a subset (the sample).

24
Validity and Bias

• In order to draw an accurate inference from a
  sample, the sample needs to be reflective of the
  population from which it is drawn.
• If a sample is not reflective of the population,
  then it is biased in some manner and the greater
  study will lack validity.

25
Types of Sampling

• Nonrandom snowballing, various improper
  selection techniques or limited data. Measurement
  error is greater.
• Random pure chance of lottery and should reflect
  population the larger the sample. Measurement
  error decreases.
• Quota or stratified Selecting on groups to form
  sample so as to reflect greater population.
  Measurement error is lower.
• Census Includes entire population. No
  measurement error.

26
Mathematical Principle

• The larger the sample size, the more it will
  reflect the population estimates/values.
• Thus, the larger the sample, the less chance of
  measurement error.

27
External Validity of a Study
Theory
What you think
Cause construct
Effect construct
Cause-effect Construct

• Can we generalize to other persons, places, times?

28
External Validity of a Study

• The last graphic is meant to convey the principle
  that external validity is gained by additional
  observations/tests in other studies.
• This is why pundits will compare several election
  polls to see how well they compare. If they do
  not, then somebody is doing something wrong or
  different.