SOCI 620: Quantitative methods 2

Agenda

Multinomial regression

  1. Categorical outcomes
  2. Softmax link function
  3. Interpreting coefficients
  4. Hands on:
    Multinomial logistic in R

Slides are licensed under CC BY-NC-SA 4.0

Categorical outcomes

Animation of the wheel from Wheel of Fortune repeated approaching 'Bankrupt'

Income and marital status

Are rich people more likely to be married?

  • Predictor: Annual earnings
  • Outcome: Marital status

The problem

Outcome variable has multiple (>2) categories. Binomial and Poisson models won’t work.

The solution

Use a categorical / multinomial outcome distribution (and a new link function) to account for the data.

A bride walks down an elaborate wedding aisle surrounded by tall live grass. All of the attendees have illuminated butterfly puppets extended.

Categorical distribution

The categorical distribution is analagous to a Bernoulli distribution with k > 2 outcomes:

Multinomial distribution

The Binomial, Bernoulli, and categorical distributions are each special cases of the multinomial distribution.

Binomial distribution

Bin(n, p) = Multinom(n, (1–p, p))

Bernoulli distribution

Bernoulli(p) = Multinom(1, (1–p, p))

Categorical distribution

Cat(p1, p2, … pk) =
Multinom(1, (p1, p2, … pk))

1 trial>1 trial
2 categories BernoulliBinomial
>2 categories CategoricalMultinomial

Categorical outcome

But we still need a link function

The softmax link function

Two comic panels. In the first a hand overs over three red bottons on a control panel. In the second, a man with a concerned look on his face sweats and wipes his brow. (the 'two-buttons' meme but there are three buttons)

Softmax

Softmax is a straightforward generalization of the the inverse logit

Untransformed (μ)

Transformed (p)

Multinomial model

Putting it all together gives us the multinomial logistic regression model (a.k.a. categorical regression model)

Multinomial model

Note that with only two categories, the multinomial logistic regression reduces to the standard logistic regression:

Interpreting coefficients

Multi-panel image of a woman who looks like she is trying to puzzle something out. Equations and geometric diagrams are superimposed in white over the images.

Interpreting coefficients

Estimate Q2.5 Q97.5
αm -5.40 -6.16 -4.67
βm 0.59 0.52 0.67
αd -4.81 -5.80 -3.86
βd 0.41 0.32 0.51
αw -3.43 -4.68 -2.23
βw 0.19 0.07 0.32

Interpreting these cofficients on their own is complex — the results are analagous to a series of logistic regressions against the reference category, conditional on the other outcomes.

E.g. αm and βm determine the probability of a person being married rather than single, assuming that they are neither divorced nor widowed.

Assessing the sign of the α and β estimates is the only straightforward interpretation.

Interpreting coefficients

Image credit

Figures by Peter McMahan (source code)

Animation of the wheel from Wheel of Fortune repeated approaching 'Bankrupt'

Clip from Wheel of Fortune (1983–)

A bride walks down an elaborate wedding aisle surrounded by tall live grass. All of the attendees have illuminated butterfly puppets extended.

Still from Crazy Rich Asians (2018)

Two comic panels. In the first a hand overs over three red bottons on a control panel. In the second, a man with a concerned look on his face sweats and wipes his brow. (the 'two-buttons' meme but there are three buttons)

Adapted from image by Jake Clark

Multi-panel image of a woman who looks like she is trying to puzzle something out. Equations and geometric diagrams are superimposed in white over the images.

Stills adapted from Senhora do Destino (2004)