Categorical › Logistic regression

Logistic regression (binary)

Binary logistic regression models log-odds of a 0/1 outcome as a linear function of predictors. Effects are reported as odds ratios.

What is Logistic regression (binary)?

Logistic regression is the workhorse for binary outcomes. The link function logit(p) = log(p/(1−p)) maps probabilities to the real line, so log-odds can be modelled linearly without constraining predicted probabilities to [0, 1]. Each coefficient β_j is the log of the odds ratio for a one-unit change in x_j; exp(β_j) is the odds ratio itself.

Compared to linear regression of a 0/1 outcome: linear models can produce predictions outside [0, 1] and have heteroscedastic residuals (variance depends on p). Logistic regression sidesteps both issues. For probabilities near 0.5, the two often agree numerically; near 0 or 1 they diverge.

Three close relatives: ordinal_logistic for ordered multi-category outcomes (proportional-odds model), multinomial_logistic for unordered multi-category outcomes (one log-odds equation per non-reference category), and cond_logistic for matched case-control studies (eliminates the within-stratum nuisance intercept).

When should I use Logistic regression (binary)?

  • Any binary outcome with one or more predictors.
  • Effect-size reporting via odds ratios (common in clinical / epi literature).
  • Pair with ROC to assess discrimination of the fitted probability score.

What data does it need?

Binary outcome + numeric / categorical predictors + intercept toggle.

What does it report?

β (log-odds), SE, z, p, OR + 95% CI per predictor; AIC; McFadden pseudo-R²; predicted-probability score for ROC.

What does it assume?

  • Independent observations.
  • Linearity of log-odds in continuous predictors — check by plotting empirical log-odds vs predictor or fitting a spline.
  • No perfect (or near-perfect) separation.
  • Adequate events-per-variable: rule of thumb ≥ 10 events per predictor.

Formula

logit(p_i) = log(p_i / (1 − p_i)) = β₀ + Σ β_j · x_{ij}

How do I interpret the result?

OR > 1 ⇒ predictor increases the odds of the outcome; OR < 1 ⇒ decreases. The point estimate matters far more than statistical significance; an OR of 1.05 with p = 0.001 in a huge sample is rarely clinically meaningful.

OR ≠ risk ratio: with a common outcome (incidence > 10%), the OR overstates the RR. Switch to Poisson with log link or use a log-binomial model when the outcome is common and you want RRs directly.

Pseudo-R²s are not directly comparable to OLS R²s — McFadden values of 0.2 − 0.4 indicate excellent fit, not poor.

See also