In this section we make you familiar with essential performance measures for regression. In particular, mean squared error (MSE), mean absolute error (MAE), and a straightforward generalization of R2 are discussed.
In this section we make you familiar with essential performance measures for classification. A classifier predicts either class labels or class probabilities. Hence, its performance can be evaluated based on these two notions. We show you some performance measures for classification, including misclassification error rate (MCE), accuracy (ACC) and Brier score (BS). Additionally you will see the confusion matrix and learn about costs.
When a machine learning model performs well on training data, but doesn't generalize on the test data, we speak of overfitting. We will show you examples of overfitting and how to diagnose overfitting.