Binning and Discretization of Continuous Variables

Binning converts a continuous feature into discrete intervals (bins), allowing linear models to capture non-linear relationships without requiring a more complex model. It also reduces the impact of small measurement errors and outliers.

Equal-Width vs Equal-Frequency Binning

Equal-width binning divides the range into bins of equal size (e.g., 0–20, 20–40, 40–60). Equal-frequency (quantile) binning ensures each bin has roughly the same number of observations, which is better for skewed data.

Binning with Pandas

<pre><code class="language-python">import pandas as pd df = pd.read_csv("data.csv") # Equal-width bins df["age_band"] = pd.cut(df["age"], bins=5, labels=["0","1","2","3","4"]) # Equal-frequency (quantile) bins df["income_quantile"] = pd.qcut(df["income"], q=4, labels=["Q1","Q2","Q3","Q4"]) print(df[["age", "age_band", "income", "income_quantile"]].head())</pre>

Discretization with scikit-learn

KBinsDiscretizer supports multiple strategies and integrates into sklearn pipelines. It can output ordinal integers or one-hot encoded bins, depending on the downstream model.

Using KBinsDiscretizer

<pre><code class="language-python">from sklearn.preprocessing import KBinsDiscretizer disc = KBinsDiscretizer(n_bins=5, encode="ordinal", strategy="quantile") X_train_disc = disc.fit_transform(X_train[["age", "income"]]) X_test_disc = disc.transform(X_test[["age", "income"]]) # encode='onehot' produces one-hot sparse matrix # strategy='kmeans' uses cluster boundaries</pre>

Trade-offs of Binning

Binning introduces information loss by treating all values within a bin as identical. Too few bins over-smooth; too many bins overfit. Use cross-validation to select the number of bins. Binning is most valuable for linear models; tree models discover optimal split points automatically.