Logistic Regression

1
Logistic Regression

• Richard Rivera
• (aka Rico)

Adapted from Scott Yabikus Lecture for SOC 507
2
Overview

• Purpose of Logistic Regression
• Likelihood
• Probability of an event
• Odds of an event occurs vs not occuring
• Odds - Ratio

3
Why do you need Logistic Regression?

• Predict the likelihood of discrete outcomes
• Group membership
• Binary outcome (disease/no disease)
• Quite Flexible Statistical Assumptions
• No assumptions about the distributions of the
  predictor variables.
• Predictors do not have to be normally distributed
• Does not have to be linearly related.
• Does not have to have equal variance within each
  group.

4
Likelihood of Dichotomous Outcomes

• Binary dependent variables (0, 1) have two
  possible outcomes (e.g., success failure)
• Success (y 1) failure (y 0).
• Goal is to estimate or predict the likelihood of
  success or failure, conditional on a set of
  independent variables.

5
Likelihood of Dichotomous Outcomes

• p
• Odds
• Odds Ratio

6
What is p?

• p probability (or proportion)

7
What is p?

• p probability (or proportion)
• The lower bound is 0, and the upper bound is 1.
• Probability of success Pr(y 1) p
• Probability of failure Pr(y 0) 1 p

8
What is the p of success or failure?
Failure Success Total
1 - p p (1 - p) p 1

9
What is the p of success or failure?
Failure Success Total
250 750 1000

10
What is the p of success or failure?
Failure Success Total
250/1000 750/1000 1000/1000

11
What is the p of success?
Failure Success Total
.25 .75 1

12
What is the p of success?
Failure Success Total
.25 1 - p .75 p 1 (1 - p) p

13
What are odds?

• Odds are related to probabilities
• The odds of an event occuring is the ratio of the
  probability of that event occurring to the
  probability of the event not occuring.
• Odds of success p of success divided by p of
  failure
• omega (?) p/(1-p)

14
What are the odds of success?
Failure Success Total
.25 (1 - p) .75 p 1 (1 - p) p

• omega (?) p/(1-p)
• ? .75/ (1 - .75)
• ? .75/.25 3

15
What is an odds ratio?

• The odds ratio compares the odds of success for
  one group to another group.
• Theta (?) ?groupA pA/(1-pA)
• ?groupB pB/(1-pB)

16
How can we compare the odds (?) of males versus
females
Group Failure Success Total
A (Male) 182 368 550
B (Female) 75 375 450
250 750 1000
17
How can we compare the odds (?) of males versus
females
Group Failure Success Total
A (Male) 182/550 368/550 550/500
B (Female) 75/450 375/450 450/450
250 750 1000
18
How can we compare the odds (?) of males versus
females
Group Failure Success Total
A (Male) .33 .67 1
B (Female) .17 83 1
250 750 1000
19
How can we compare the odds (?) of males versus
females
Group Failure Success Total
A (Male) (1 - pA) .33 pA .67 1
B (Female) (1 - pB) .17 pB .83 1
250 750 1000
20
How can we compare the odds (?) of males versus
females
Group Failure Success Total
A (Male) (1 - pA) .33 pA .67 1
B (Female) (1 - pB) .17 pB .83 1

• ?groupA pA/(1-pA)
• ?groupB pB/(1-pB)

21
How can we compare the odds (?) of males versus
females
Group Failure Success Total
Male .33 .67 1
Female .17 .83 1

• ?male .67/.33
• ?female .83/.17

22
How can we compare the odds (?) of males versus
females
Group Failure Success Total
Male .33 .67 1
Female .17 .83 1

• ?male .67/.33 2.03
• ?female .83/.17 4.88
• Theta (?) ?groupA / ?groupB

23
How can we compare the odds (?) of males versus
females

• Theta (?) ?groupA / ?groupB
• ?male / ?female 2.03 / 4.88
• ?male / ?female .4160
• The odds that males succeeds compared to females
  are only .416 times that of females

24
How can we compare the odds (?) of males versus
females

• How about ? ?groupB / ?groupA
• ?female / ?male 4.88 / 2.03 2.404
• The odds that females succeeds compared to the
  odds that males succeeds are 2.40 times that of
  males (or, 2.40 times greater).
• Or, you could say the odds for females are 218
  greater.
• Take the odds ratio and subtract 1.

25
What is so special about 1

• Take the odds ratio and subtract 1.
• Whats so special about 1? 1.00 is the null
  effectwhen the odds ratio is 1.00, there is no
  difference in the odds for one group relative to
  the other.
• So when we describe odds ratios, we often
  describe them by how much they differ from 1.00

26
Why is it called Logistic regression?

• It uses the logit transformation.
• The logistics transformation can be interpreted
  as the logarithm of the odds of success vs.
  failure.

27
Lets go through an example
28
What are the odds of favoring gun permits? What
are the odds that a male respondent favors gun
permits? What is the odds ratio for a male
favoring gun permits compared to a female? What
is the log odds ratio for a male favoring gun
permits compared to a female?
29
Lets run it in SPSS

• 1st, I recommend that you recode any binary
  variables into new variables with categories 0
  and 1.
• Transform gt Recode gt into a different variable
• Subsequently Analyze gt Regression gt Binary
  Logistic

30
Example Output