Binary vs. Multi-class Classification

Classification problems fall into two broad categories: binary (two possible classes) and multi-class (three or more), and the choice shapes model selection, loss functions, and evaluation metrics.

Binary Classification

In binary classification the output is one of two classes — typically encoded as 0 and 1. Most classifiers are natively binary: logistic regression, SVMs, and single perceptrons all learn a decision boundary separating two groups.

Common Binary Examples

Email spam detection (spam / not spam), medical diagnosis (disease / healthy), and fraud detection (fraud / legitimate) are prototypical binary problems. The key challenge is often class imbalance — one class is far rarer than the other, requiring special treatment during training and evaluation.

Multi-class Classification

Multi-class problems have three or more target classes. Many algorithms extend naturally (decision trees, random forests, neural networks), while others require a decomposition strategy.

One-vs-Rest and One-vs-One

One-vs-Rest (OvR) trains one binary classifier per class, predicting the most confident class. One-vs-One (OvO) trains one binary classifier for every pair of classes and uses majority voting. scikit-learn handles this automatically for you:

<pre><code class="language-python">from sklearn.linear_model import LogisticRegression from sklearn.datasets import load_iris from sklearn.model_selection import train_test_split X, y = load_iris(return_X_y=True) X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, random_state=0) # multi_class handled automatically clf = LogisticRegression(max_iter=200).fit(X_tr, y_tr) print("Classes:", clf.classes_) print("Accuracy:", clf.score(X_te, y_te))</pre>

Softmax for Multi-class

When using logistic regression with multi_class='multinomial', the model uses the softmax function to produce a probability distribution over all K classes simultaneously, which is generally more accurate than OvR for well-balanced datasets.