Support Vector Regression (SVR)

Support Vector Regression (SVR) finds the flattest function that fits the training data within a specified error margin \u03b5, ignoring errors smaller than that threshold entirely.

The SVR Objective

SVR aims to fit a tube of width \u00b1\u03b5 around the data and penalises only points that fall outside the tube. This makes SVR robust to small noise and outliers within the margin.

Key Hyperparameters

C controls the trade-off between margin width and training error — higher C allows fewer margin violations. epsilon defines the width of the insensitive tube. kernel maps data into a higher-dimensional space; 'rbf' is the common default.

Fitting SVR in scikit-learn

<pre><code class="language-python">from sklearn.svm import SVR from sklearn.preprocessing import StandardScaler from sklearn.pipeline import make_pipeline from sklearn.datasets import fetch_california_housing from sklearn.model_selection import train_test_split from sklearn.metrics import r2_score data = fetch_california_housing() X_tr, X_te, y_tr, y_te = train_test_split( data.data, data.target, test_size=0.2, random_state=42 ) svr = make_pipeline(StandardScaler(), SVR(C=1.0, epsilon=0.1, kernel="rbf")) svr.fit(X_tr, y_tr) print(f"SVR Test R\u00b2: {r2_score(y_te, svr.predict(X_te)):.3f}")</pre>

Strengths and Limitations

SVR excels on small-to-medium datasets with complex patterns but does not scale well to very large datasets due to its quadratic training complexity.

When to Use SVR

SVR is a strong choice when: your dataset has fewer than ~10,000 samples, features have been scaled, and you suspect a non-linear relationship. For large datasets, gradient-boosted trees or deep learning are usually faster and more scalable alternatives.