Categorical › Classification (supervised)

Naive Bayes

Gaussian naive Bayes classifies by Bayes' rule while assuming predictors are independent within each class — fast, surprisingly robust, and reported with 10-fold CV accuracy.

What is Naive Bayes?

Naive Bayes models each class's predictors as independent Gaussians and classifies by posterior probability. The independence assumption is almost always false, yet the classifier often works because it only needs the argmax to be right.

Laplace smoothing regularizes rare categories (mostly relevant for categorical predictors; harmless here).

When should I use Naive Bayes?

  • Fast baseline, small samples, many predictors.
  • When you want calibrated-ish class posteriors cheaply.

What data does it need?

Categorical outcome + numeric predictors + optional Laplace smoothing.

What does it report?

10-fold CV accuracy, training accuracy, confusion matrix.

What does it assume?

  • Within-class predictor independence (violations degrade gracefully).
  • Gaussian within-class distributions for numeric predictors.

How do I interpret the result?

If naive Bayes matches fancier models' CV accuracy, prefer it — simpler and better calibrated than it has any right to be.

See also

References

  • Hand & Yu (2001). Idiot's Bayes — not so stupid after all? Int Stat Rev 69.