SOCI 620: Quantitative methods 2

Agenda

Multinomial regression

  1. Categorical outcomes
  2. Softmax link function
  3. Interpreting coefficients
  4. Hands on:
    Multinomial logistic in R

Slides are licensed under CC BY-NC-SA 4.0

Uploading to generative AI tools or use for training machine learning models is prohibited

Categorical outcomes

Animation of the wheel from Wheel of Fortune repeated approaching 'Bankrupt'

Income and marital status

How does income relate to marital status?

  • Predictor: Annual earnings
  • Outcome: Marital status

The problem

Outcome variable has multiple (>2) categories. Binomial and Poisson models won’t work.

The solution

Use a categorical / multinomial outcome distribution (and a new link function) to account for the data.

Movie still (Ready or Not, 2019). A woman in a white wedding dress walks down a slightly creepy, candle-lit hallway in a lavish old house.

Categorical distribution

A pair of hands holds a single playing card face down.

The categorical distribution is analagous to a Bernoulli distribution with k > 2 outcomes:

Multinomial distribution

Photograph of several crudely-carved ivory dice.

The Binomial, Bernoulli, and categorical distributions are each special cases of the multinomial distribution.

Binomial distribution

Bin(n, p) = Multinom(n, (1–p, p))

Bernoulli distribution

Bernoulli(p) = Multinom(1, (1–p, p))

Categorical distribution

Cat(p1, p2, … pk) =
Multinom(1, (p1, p2, … pk))

1 trial>1 trial
2 categories BernoulliBinomial
>2 categories CategoricalMultinomial

Categorical outcome

But we still need a link function

The softmax link function

Two comic panels. In the first a hand overs over three red bottons on a control panel. In the second, a man with a concerned look on his face sweats and wipes his brow. (the 'two-buttons' meme but there are three buttons)

Softmax

Softmax is a straightforward generalization of the the inverse logit

Untransformed (μ)

Transformed (p)

Multinomial logistic model

Putting it all together gives us the multinomial logistic regression model (a.k.a. categorical regression model)

Multinomial logistic model

Note that with only two categories, the multinomial logistic regression reduces to the standard logistic regression:

Interpreting coefficients

Multi-panel image of a woman who looks like she is trying to puzzle something out. Equations and geometric diagrams are superimposed in white over the images.

Interpreting coefficients

Estimate Q2.5 Q97.5
αm -5.40 -6.16 -4.67
βm 0.59 0.52 0.67
αd -4.81 -5.80 -3.86
βd 0.41 0.32 0.51
αw -3.43 -4.68 -2.23
βw 0.19 0.07 0.32

Interpreting these cofficients on their own requires care — the results are analagous to a series of logistic regressions against the reference category, conditional on the other outcomes.

E.g. αm and βm determine the probability of a person being married rather than single, conditional on being neither divorced nor widowed.

Assessing the sign ( + or – ) of the α and β estimates can be useful.

Interpreting coefficients

Image credit

Figures by Peter McMahan (source code)

Animation of the wheel from Wheel of Fortune repeated approaching 'Bankrupt'

Clip from Wheel of Fortune (1983–)

Movie still (Ready or Not, 2019). A woman in a white wedding dress walks down a slightly creepy, candle-lit hallway in a lavish old house.

Still from Ready or Not (2019)

A pair of hands holds a single playing card face down.

Still from Ready or Not (2019)

Photograph of several crudely-carved ivory dice.

Ancient Roman dice, via Yorokobu

Two comic panels. In the first a hand overs over three red bottons on a control panel. In the second, a man with a concerned look on his face sweats and wipes his brow. (the 'two-buttons' meme but there are three buttons)

Adapted from image by Jake Clark

Multi-panel image of a woman who looks like she is trying to puzzle something out. Equations and geometric diagrams are superimposed in white over the images.

Stills adapted from Senhora do Destino (2004)