Categorical › Classification (supervised)

Regularized regression (lasso / ridge)

Regularized regression fits lasso, ridge or elastic net with a cross-validated λ, detecting a Gaussian, binomial or multinomial family automatically.

What is Regularized regression (lasso / ridge)?

Ridge (α = 0) shrinks all coefficients toward zero; lasso (α = 1) shrinks some to exactly zero, doing variable selection; elastic net mixes both. The penalty strength λ is chosen by 10-fold cross-validation (λ_min reported alongside the more conservative λ_1se).

Regularization trades a little bias for a large variance reduction — the standard cure for many-predictors / collinear-predictors regression. Coefficients are reported at λ_min; treat them as biased-by-design (no SEs / p-values).

When should I use Regularized regression (lasso / ridge)?

  • More predictors than the sample comfortably supports.
  • Collinear predictors where OLS coefficients go wild.
  • Automatic variable selection (lasso).

What data does it need?

Outcome + ≥ 2 numeric predictors + penalty type (lasso / elastic net / ridge).

What does it report?

λ_min / λ_1se, CV RMSE (or CV accuracy for classification), deviance ratio (R² analogue), nonzero-coefficient table at λ_min.

What does it assume?

  • Independent observations.
  • Linearity on the link scale.
  • Predictors standardized internally.

How do I interpret the result?

Lasso zeroing a coefficient means it adds nothing beyond the kept predictors — not that it's marginally unrelated. For inference-grade coefficients, refit the selected variables with ordinary (g)lm.

See also

References

  • Friedman, Hastie & Tibshirani (2010). Regularization paths for GLMs via coordinate descent. JSS 33.