Confusion Matrix Explained: Reading It Correctly

A single accuracy number hides the most important thing about a classifier: which mistakes it makes. The confusion matrix shows them. It is a small table, but people misread it constantly, mostly because different tools put the truth on different axes. This guide reads one cell by cell, using a model with the kind of imbalance real problems have.

The layout in scikit-learn

In scikit-learn, rows are the true classes and columns are the predicted classes: entry i, j counts observations whose true label is i and predicted label is j (scikit-learn: confusion_matrix). For a binary problem with classes 0 and 1:

Other tools and textbooks sometimes transpose this, putting predictions in rows. Before interpreting any matrix, check which axis is which; getting it backwards swaps false positives and false negatives, which can reverse your conclusion.

A realistic example: 5% positives

Here is a fraud-like problem: 5% of cases are positive, generated with scikit-learn's make_classification (scikit-learn: make_classification), and a logistic regression trained on it:

import numpy as np
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import confusion_matrix
from sklearn.model_selection import train_test_split

X, y = make_classification(n_samples=5000, n_features=10, n_informative=4,
                           weights=[0.95, 0.05], flip_y=0.01, random_state=3)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=0, stratify=y)
model = LogisticRegression(max_iter=1000).fit(X_train, y_train)
proba = model.predict_proba(X_test)[:, 1]
print("test set:", np.bincount(y_test))

for threshold in (0.5, 0.2):
    pred = (proba >= threshold).astype(int)
    cm = confusion_matrix(y_test, pred)
    tn, fp, fn, tp = cm.ravel()
    print(threshold, cm.tolist(),
          f"precision {tp / (tp + fp):.2f} recall {tp / (tp + fn):.2f} "
          f"specificity {tn / (tn + fp):.3f} accuracy {(tp + tn) / cm.sum():.3f}")

test set: [1894  106]
0.5 [[1886, 8], [70, 36]] precision 0.82 recall 0.34 specificity 0.996 accuracy 0.961
0.2 [[1834, 60], [49, 57]] precision 0.49 recall 0.54 specificity 0.968 accuracy 0.946

Reading the matrix at the default threshold

At the default threshold of 0.5, the matrix is [[1886, 8], [70, 36]]. In business terms:

• 1,886 true negatives: legitimate transactions correctly passed.
• 8 false positives: legitimate transactions wrongly blocked, which means annoyed customers.
• 70 false negatives: fraud that got through, the expensive mistake.
• 36 true positives: fraud caught.

Accuracy is 96.1%, which sounds excellent, yet the model caught only 36 of 106 fraud cases, a recall of 0.34. A model that labelled everything as legitimate would score 94.7% accuracy while catching nothing. With imbalanced classes, accuracy mostly measures how common the majority class is.

The metrics each cell feeds

Each metric ignores some cells, which is why no single one is enough (scikit-learn: model evaluation). Precision never looks at false negatives; recall never looks at false positives. Evaluation metrics: accuracy, precision, recall and F1 covers how to combine them.

Moving the threshold reshapes the matrix

A classifier outputs a probability; the threshold turns it into a decision. Lowering it from 0.5 to 0.2 flags more cases as fraud. In the output above, true positives rise from 36 to 57, so recall improves from 0.34 to 0.54, and false positives rise from 8 to 60, so precision falls from 0.82 to 0.49. Accuracy barely moves. Which matrix is better is a business question: what does a missed fraud cost, and what does blocking a genuine customer cost? Choose the threshold from those costs, not from the default.

Normalised matrices: rates instead of counts

With imbalanced data, raw counts make the minority row hard to read. Normalising each row by the number of true cases shows rates directly:

print(confusion_matrix(y_test, (proba >= 0.5).astype(int), normalize="true").round(3).tolist())

[[0.996, 0.004], [0.66, 0.34]]

The bottom row says it plainly: 66% of real fraud was missed. normalize="true" divides by the row totals, the true classes; "pred" divides by the column totals, which gives precision-style rates instead.

Presenting it to people who decide

Stakeholders rarely want a grid of four numbers. Translate the matrix into the outcomes they care about, per thousand or per month: "out of every 100 fraudulent transactions, the model stops 34 and lets 66 through; out of every 1,000 genuine ones, it blocks about 4." Then show the same sentence at a second threshold, so the trade-off is a choice they can make with you, not a technical detail they have to trust. Pair the counts with the cost of each kind of mistake, and the conversation moves quickly from "is the model good?" to "which threshold do we want?", which is the right question.

Common mistakes when reading one

• Reading the axes the wrong way round, which swaps false positives and false negatives.
• Judging the model from the diagonal alone. A large number of true negatives looks impressive but says nothing about the positive class.
• Comparing matrices from test sets of different sizes or balance, instead of comparing rates.
• Treating the default threshold as part of the model. It is a separate decision, and often the wrong one for imbalanced problems.
• Evaluating on training data, which makes every matrix look better than reality.

Multiclass matrices

With more classes, the matrix grows to one row and one column per class. The diagonal holds correct predictions, and each off-diagonal cell names a specific confusion: "true 3, predicted 8" tells you exactly which classes the model mixes up. That is often more actionable than any single score: the fix may be more training examples of those two classes, or a feature that separates them.

Stable numbers need honest validation; cross-validation in machine learning shows how to get them, and handling imbalanced datasets covers the training side. Evaluation metrics and model validation form the model evaluation module of the Data Science course.

Sources

• sklearn.metrics.confusion_matrix — scikit-learn documentation
• Model evaluation: confusion matrix — scikit-learn documentation
• sklearn.datasets.make_classification — scikit-learn documentation