Categorical › Classification (supervised)

Neural network (single hidden layer)

A single-hidden-layer neural network with weight decay learns classification or regression, reported with 5-fold cross-validated performance.

What is Neural network (single hidden layer)?

A classic multilayer perceptron with one hidden layer of logistic units — enough to approximate any continuous function given sufficient units, though modern deep nets are out of scope for a stats tool. Weight decay is L2 regularization on the weights: the single most important knob against overfitting.

Predictors are standardized (networks are scale-sensitive). Training accuracy is reported alongside 5-fold CV so the overfitting gap is visible.

When should I use Neural network (single hidden layer)?

  • Smooth nonlinear decision boundaries or response surfaces.
  • As a nonlinear benchmark next to the linear/logistic model.

What data does it need?

Outcome + numeric predictors + hidden units + weight decay.

What does it report?

5-fold CV accuracy + confusion (classification) or CV RMSE + R² (regression), weight count.

What does it assume?

  • Independent observations.
  • Results vary with random initialization — seed fixed for reproducibility.

How do I interpret the result?

A big train-vs-CV gap = overfitting: raise decay or cut hidden units. If CV accuracy matches logistic regression, prefer the simpler model.

See also

References

  • Venables & Ripley (2002). Modern Applied Statistics with S, ch. 8.