Accuracy vs. Precision vs. Recall

Accuracy alone is a misleading metric for most real-world classification tasks — precision and recall expose the specific failure modes that accuracy hides.

Accuracy and Its Limitations

Accuracy = (TP + TN) / Total. It is easy to interpret but breaks down with imbalanced classes — a model that always predicts the majority class can achieve 99% accuracy while being completely useless for detecting the minority class.

The Imbalanced Class Problem

In fraud detection, only 0.1% of transactions are fraudulent. A model predicting "not fraud" for everything achieves 99.9% accuracy yet catches zero frauds. Precision and recall on the minority class reveal that this model is worthless for its intended purpose.

Precision and Recall

Precision = TP / (TP + FP): of all predicted positives, how many are truly positive? Recall = TP / (TP + FN): of all actual positives, how many did we catch?

Computing All Three Metrics

<pre><code class="language-python">from sklearn.metrics import accuracy_score, precision_score, recall_score, classification_report from sklearn.linear_model import LogisticRegression from sklearn.datasets import load_breast_cancer from sklearn.model_selection import train_test_split X, y = load_breast_cancer(return_X_y=True) X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, random_state=42) clf = LogisticRegression(max_iter=5000).fit(X_tr, y_tr) y_pred = clf.predict(X_te) print(f"Accuracy: {accuracy_score(y_te, y_pred):.4f}") print(f"Precision: {precision_score(y_te, y_pred):.4f}") print(f"Recall: {recall_score(y_te, y_pred):.4f}") print(classification_report(y_te, y_pred))</pre>

When to Prioritise Each

Prioritise precision when false positives are costly (e.g., spam filter — you don't want to block legitimate emails). Prioritise recall when false negatives are costly (e.g., cancer screening — you don't want to miss a true case). Use accuracy only when classes are balanced and both error types are equally costly.