Receiver Operating Characteristic (ROC) and AUC

The ROC curve evaluates a classifier's ability to discriminate between classes across all possible thresholds at once, and the AUC summarises that performance in a single threshold-independent number.

The ROC Curve

The ROC curve plots the True Positive Rate (Recall) on the y-axis against the False Positive Rate on the x-axis as the decision threshold sweeps from 1 to 0. A perfect classifier's curve hugs the top-left corner; a random classifier follows the diagonal.

Plotting the ROC Curve

<pre><code class="language-python">from sklearn.metrics import roc_curve, roc_auc_score, RocCurveDisplay from sklearn.linear_model import LogisticRegression from sklearn.datasets import load_breast_cancer from sklearn.model_selection import train_test_split from sklearn.preprocessing import StandardScaler from sklearn.pipeline import make_pipeline X, y = load_breast_cancer(return_X_y=True) X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, random_state=42) clf = make_pipeline(StandardScaler(), LogisticRegression(max_iter=5000)) clf.fit(X_tr, y_tr) y_scores = clf.predict_proba(X_te)[:, 1] auc = roc_auc_score(y_te, y_scores) print(f"AUC: {auc:.4f}") RocCurveDisplay.from_predictions(y_te, y_scores).plot() </pre>

Area Under the Curve (AUC)

AUC ranges from 0.5 (random) to 1.0 (perfect). It is threshold-independent and equals the probability that the model ranks a random positive higher than a random negative.

Interpreting AUC

AUC > 0.9: excellent discrimination. AUC 0.8-0.9: good. AUC 0.7-0.8: fair. AUC 0.5-0.7: poor — barely better than guessing. AUC is particularly valuable when comparing classifiers across different datasets or when the deployment threshold is not yet known.

ROC vs. Precision-Recall Curve

With highly imbalanced datasets the ROC curve can look optimistic because it includes TN in FPR. In those cases, the Precision-Recall (PR) curve is more informative — it focuses entirely on the minority class and does not benefit from a large number of true negatives.

<pre><code class="language-python">from sklearn.metrics import PrecisionRecallDisplay PrecisionRecallDisplay.from_predictions(y_te, y_scores).plot() </pre>