Linear discriminant analysis (LDA)

What is Linear discriminant analysis (LDA)?

Fisher's LDA finds the projection direction that maximises (between-group variance) / (within-group variance). For k groups it generalises to min(k − 1, p) such directions (the discriminant functions), each capturing successively less between-group separation.

Classification via LDA assumes class-conditional distributions are multivariate normal with shared covariance — strong assumption that's rarely strictly met but often robust enough in practice. When the covariances differ, switch to QDA (quadratic discriminant analysis) or a decision tree.

Compared to logistic regression: LDA is more efficient (lower variance) when its normality assumption holds; logistic is more robust when it doesn't. With well-separated classes both work; with class overlap LDA tends to be more decisive (sharper boundary).

When should I use Linear discriminant analysis (LDA)?

• Multi-class classification with normal-ish features.
• Lower-dimensional visualisation of class separation (LD1 vs LD2 scatter).
• Pair with PCA to compare unsupervised vs supervised projections.

What data does it need?

Grouping factor + numeric predictors + LOO CV toggle.

What does it report?

Eigenvalues + % trace + Wilks Λ + Bartlett χ² per function + standardised coefficient matrix + group means + LD1-vs-LD2 scatter + training + LOO confusion matrices.

What does it assume?

• Multivariate normal class distributions.
• Shared covariance across classes.
• Continuous predictors.

How do I interpret the result?

LOO accuracy is the honest estimate; training accuracy is optimistic. Big gap between the two ⇒ overfitting.

Wilks Λ p < 0.05 for a function ⇒ that function adds significant between-group separation.

See also

• Logistic regression (binary)
• Decision tree (CART)
• Multinomial logistic regression

References

• Fisher (1936). The use of multiple measurements in taxonomic problems. Annals of Eugenics 7(2).