Parameters vs. Hyperparameters

In machine learning, parameters are learned automatically from data during training, while hyperparameters are configuration values you must set before training starts — they control how learning happens.

Model Parameters

Parameters are the internal values a model adjusts to minimize its loss function — they are the model itself, not something you configure.

Examples of Parameters

Common parameters include the weights and biases of a neural network, the coefficients in linear regression, and the support vectors in an SVM. These are never set manually; they emerge from the training process.

Accessing Parameters in Scikit-Learn

After fitting a model you can inspect its learned parameters directly:

<pre><code class="language-python">from sklearn.linear_model import LinearRegression import numpy as np X = np.array([[1], [2], [3], [4]]) y = np.array([2, 4, 6, 8]) model = LinearRegression() model.fit(X, y) print("Coefficient (weight):", model.coef_) # learned parameter print("Intercept (bias):", model.intercept_) # learned parameter</pre>

Hyperparameters

Hyperparameters sit outside the training loop; you choose them ahead of time to shape the model's capacity, regularisation strength, or optimisation behaviour.

Common Hyperparameters by Model

  • Decision Tree: max_depth, min_samples_split
  • Random Forest: n_estimators, max_features
  • SVM: C, kernel, gamma
  • Ridge Regression: alpha (regularisation strength)

These are passed to the constructor and do not change during fit().

Viewing Hyperparameters with get_params()

<pre><code class="language-python">from sklearn.ensemble import RandomForestClassifier rf = RandomForestClassifier(n_estimators=100, max_depth=5, random_state=42) print(rf.get_params()) # {'bootstrap': True, 'max_depth': 5, 'n_estimators': 100, ...}</pre>

Why the Distinction Matters

Confusing the two leads to common mistakes: manually setting parameters (overfitting by design) or forgetting to tune hyperparameters (leaving performance on the table).

The Hyperparameter Tuning Workflow

The standard approach is: 1) choose a model family, 2) define a hyperparameter search space, 3) use cross-validation to evaluate each candidate, and 4) retrain the best configuration on all available training data. Tools like GridSearchCV and RandomizedSearchCV automate steps 2–4.