多项式回归度数递增误差

Question

我正在尝试预测波士顿房价。当我选择多项式回归度 1 或 2 时，R2 得分是可以的。但是 3rd degree 会降低 R2 分数。

# Importing the libraries
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

# Importing the dataset
from sklearn.datasets import load_boston
boston_dataset = load_boston()
dataset = pd.DataFrame(boston_dataset.data, columns = boston_dataset.feature_names)
dataset['MEDV'] = boston_dataset.target

X = dataset.iloc[:, 0:13].values
y = dataset.iloc[:, 13].values.reshape(-1,1)

# Splitting the dataset into the Training set and Test set
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0)

# Fitting Linear Regression to the dataset
from sklearn.linear_model import LinearRegression

# Fitting Polynomial Regression to the dataset
from sklearn.preprocessing import PolynomialFeatures
poly_reg = PolynomialFeatures(degree = 2)   # <-- Tuning to 3
X_poly = poly_reg.fit_transform(X_train)
poly_reg.fit(X_poly, y_train)
lin_reg_2 = LinearRegression()
lin_reg_2.fit(X_poly, y_train)

y_pred = lin_reg_2.predict(poly_reg.fit_transform(X_test))

from sklearn.metrics import r2_score
print('Prediction Score is: ', r2_score(y_test, y_pred))

输出（度数=2）：

Prediction Score is:  0.6903318065831567

输出（度=3）：

Prediction Score is:  -12898.308114085281

Answer 1

这被称为过度拟合模型。您所做的是在训练集上完美拟合模型，这将导致高方差。当您在训练集上很好地拟合您的假设时，它将在测试集上失败。您可以使用r2_score(X_train,y_train) 检查您的训练集的 r2_score。会很高。您需要平衡偏差和方差之间的权衡。

您可以尝试其他回归模型，例如 lasso 和 ridge，并且可以使用它们的 alpha 值，以防您正在寻找高 r2_score。为了更好地理解，我将展示假设线如何影响增加多项式次数的图像。