2.2. Classification Metrics

Classification Metrics

Once a classification model is built, its effectiveness is evaluated using quantities derived from the confusion matrix, which summarizes prediction outcomes in terms of true/false positives/negatives, such that false positives are negative cases incorrectly predicted as positive, and vice-versa. From these four outcomes, several key metrics are defined:

• Accuracy is proportion of all predictions that are correct:
  $\text{Accuracy}=\frac{TP+TN}{TP+TN+FP+FN}.$
• Precision is the proportion of predicted positives that are actually positive:
  $\text{Precision}=\frac{TP}{TP+FP}.$
• Recall (True Positive Rate) is the proportion of actual positives that were correctly identified:
  $\text{Recall}=\frac{TP}{TP+FN}.$
• False Positive Rate is the proportion of actual negatives misclassified as positives:
  $\text{FPR}=\frac{FP}{FP+TN}.$
• F1 Score is the harmonic mean of precision and recall, balancing both measures:
  $F_1=2\cdot \frac{\text{Precision}\cdot \text{Recall}}{\text{Precision}+\text{Recall}}$

These metrics can be used in tandem to determine a classification threshold for binary classifiers. For example,

• Precision–Recall Tradeoff: Precision ($y$) and recall ($x$) can be plotted for all thresholds $\theta \in [0,1]$ to observe how improving one often reduces the other. The balance point depends on the application’s tolerance for false positives and false negatives.
  ⠀
• Maximum $F_1$ Threshold: The threshold that maximizes $F_1$ represents the best balance between precision and recall when both are equally important.
  ⠀
• ROC Curve and AUC: The ROC curve is constructed by sweeping the classification threshold from 0 to 1 and plotting the TPR/Recall ($y$) against the FPR ($x$) to show how sensitivity and false alarm rates change as the cutoff varies. The Area Under the Curve (AUC) summarizes overall separability: an AUC of 1 indicates perfect classification, while 0.5 corresponds to random guessing.

Choosing the optimal classification threshold depends on the tradeoff between false positives and false negatives. When classes are balanced, the threshold is typically set to maximize accuracy (often near $0.5$), while for imbalanced data or when positive predictions matter more, it is chosen to maximize the $F_1$ score. Accuracy and $F_1$ are threshold-dependent, whereas AUC-based metrics are not.