Confusion Matrix

A confusion matrix is a table that summarizes the performance of a classification model by comparing predicted labels against actual labels. Rows represent true classes, columns represent predicted classes (or vice versa). Diagonal cells show correct predictions; off-diagonal cells show errors. For binary classification, the matrix yields true positives, true negatives, false positives, and false negatives. Operators use it to compute metrics like precision, recall, and F1-score. It matters because accuracy alone can mislead on imbalanced datasets—e.g., a model that always predicts 'not spam' gets 99% accuracy on 99% non-spam data but fails entirely on spam.

Suppose you fine-tune a DistilBERT model on a sentiment dataset (positive/negative) using Hugging Face Transformers. After evaluation on 1000 test samples, the confusion matrix shows: 400 true positives, 350 true negatives, 100 false positives, 150 false negatives. From this, precision = 400/(400+100) = 0.8, recall = 400/(400+150) ≈ 0.727, F1 ≈ 0.762. If you only looked at accuracy (750/1000 = 75%), you'd miss that recall for positive class is low.

In a classification workflow with scikit-learn, after training a model you run from sklearn.metrics import confusion_matrix; cm = confusion_matrix(y_true, y_pred). In Hugging Face Trainer, the compute_metrics function often returns a confusion matrix via evaluate.load('accuracy') or custom code. In LM Studio, you'd export predictions and compute the matrix externally. The matrix helps decide threshold tuning—e.g., lowering the decision threshold increases recall but may raise false positives.