The Sigmoid Function and Log-Odds

Understanding the sigmoid function and log-odds reveals why logistic regression is both a probability estimator and a linear classifier in disguise.

Odds and Log-Odds

The odds of an event are P / (1-P) — the ratio of success to failure probability. The log-odds (logit) is log[P/(1-P)], which ranges from \u2212\u221e to +\u221e and is exactly the linear combination \u03b2\u1d40x that logistic regression models.

The Logit Transform

Logistic regression is linear in the log-odds: log[P/(1-P)] = \u03b2\u2080 + \u03b2\u2081x\u2081 + .... Each coefficient \u03b2\u1d62 is the change in log-odds for a one-unit increase in x\u1d62. Exponentiating a coefficient gives the odds ratio — how many times more likely the event becomes per unit increase in that feature.

The Sigmoid Function

Applying the inverse logit (sigmoid) to the linear score maps it back to a probability: \u03c3(z) = 1/(1+e\u207b\u1d63).

Properties of the Sigmoid

<pre><code class="language-python">import numpy as np import matplotlib.pyplot as plt z = np.linspace(-8, 8, 300) sigmoid = 1 / (1 + np.exp(-z)) plt.plot(z, sigmoid) plt.axhline(0.5, color="red", linestyle="--", label="Decision boundary (z=0)") plt.xlabel("z (log-odds)") plt.ylabel("\u03c3(z) = P(y=1)") plt.title("Sigmoid Function") plt.legend() plt.tight_layout() plt.show()</pre>

Decision Boundary

The decision boundary occurs at \u03c3(z) = 0.5, which corresponds to z = 0. This means the model predicts class 1 when the linear combination of features is positive, and class 0 when it is negative — making logistic regression a linear classifier.