Unless a classifiers should predict a discrete output in the end, classification models usually output scores or probablities first. We will explain why, introduce the concepts of decision regions and decision boundaries and finally differentiate two fundamental approaches to construct classifiers: the generative approach and the discriminant approach.
Linear classifiers are an essential subclass of classification models. This section provides the definition of a linear classifier and depicts differences between linear and non-linear decision boundaries.
Logistic regression is a discrimant approach for constructing a classifier. We will motivate logistic regression via the logistic function, define the log loss for optimization and illustrate the approach in 1D and 2D.
Discriminant analysis is a generative approach for constructing a classifier. We distinguish between linear (LDA) and quadratic (QDA) discriminant analysis where the latter is a more flexible approach.
This section introduces k-nearest neighbors classification. We will explain in which sense this approach is fundamentally different to the previous sections and illustrate the effect of the hyperparameter k.