54.54. Ridge Regression (L2)

Implement **Ridge Regression** (Linear Regression with L2 regularization) using gradient descent.

Loss Function:

Loss = (1/n) * sum((y_pred - y)^2) + alpha * sum(w^2)

Gradient Descent Update Rules:

dw = (2/n) * (X^T @ (y_pred - y)) + 2 * alpha * w

db = (2/n) * sum(y_pred - y)

w = w - learning_rate * dw

b = b - learning_rate * db

Note: The L2 penalty **only applies to weights**, not the bias term.

Your class should have:

Example:

X = [[1, 2], [2, 3], [3, 4], [4, 5]]

y = [5, 8, 11, 14]

alpha = 0.1

model = RidgeRegression(alpha=0.1, learning_rate=0.01)

model.fit(X, y, n_iterations=1000)

# Weights will be slightly shrunk toward 0 compared to unregularized regression

**Explanation:** L2 regularization adds a penalty proportional to the square of weights, discouraging large weight values and reducing overfitting.

Constraints: