Sklearn 线性回归损失函数与手动代码不匹配

Question

我一直在尝试使用手动代码按照 Sklearn 线性回归库复制成本的结果。两者之间存在巨大差异，我无法弄清楚为什么。这是 Sklearn 的代码：

SkLearn 实施：

X_train, X_test, Y_train, Y_test = model_selection.train_test_split(X, Y, test_size=0.30)
classifier = sklearn.linear_model.LinearRegression()
classifier.fit(X_train,Y_train)
cost = np.sqrt(np.sum((np.dot(X_train,classifier.coef_.reshape(9,1)) + classifier.intercept_ - Y_train.reshape(478,1))**2))
print(cost)

cost = 4.236441942240197

我复制结果的尝试：

a = X_train_rev.shape
assert(X_train_rev.shape == (478,10)) # assert shape of the X_train_rev
Y_train = Y_train.reshape(478,1)

alpha = 0.0005 # Learning_Rate
coefficient = np.random.randn(1,10) # Initialisation of coefficients including intercept

# Loop through iterations
for i in range(100000):
    cost = np.sqrt(np.sum((np.dot(X_train_rev,coefficient.T) - Y_train)**2)) # cost result
    if i % 10000 == 0: print(cost)
    grad = np.dot((np.dot(X_train_rev,coefficient.T) - Y_train).T, X_train_rev) # Compute Gradients
    coefficient = coefficient - (alpha * grad) # adjust coefficients including intercept



Cost after Iterations:
45.23042864973579
10.428401916963285
10.428401916963285
10.428401916963285
10.428401916963285
10.428401916963285
10.428401916963285
10.428401916963285
10.428401916963285
10.428401916963285

根据我的手动代码，成本并没有进一步降低，而且与 Sklearn 的成本相差甚远。我尝试使用 alpha 变量，但 alpha 的任何增加都会导致成本趋于正无穷大。

请注意，我的手动代码中使用的 X_train_rev 数据在 Sklearn 训练集中有 10 个列/特征而不是 9 个特征，因为我在训练集中添加了一列“ones”来表示截距。同样，系数向量也包含截距。

Answer 1

我试图复制你的问题

from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error

X, Y = make_regression(n_samples=500, n_features=9, bias=0, random_state=1)

X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.30, random_state=1)
classifier = LinearRegression(fit_intercept=False)
classifier.fit(X_train,Y_train)
cost = np.sqrt(np.sum((np.dot(X_train,classifier.coef_.reshape(9,1)) + classifier.intercept_ - Y_train.reshape(-1,1))**2))

print(cost)
print('Manual regression')
Y_train = Y_train.reshape(-1,1)

alpha = 0.0005 # Learning_Rate
coefficient = np.random.randn(1,9) # Initialisation of coefficients including intercept

# Loop through iterations
for i in range(100000):
    cost = np.sqrt(np.sum((np.dot(X_train,coefficient.T) - Y_train)**2)) # cost result
    if i % 10000 == 0: print(cost)
    grad = np.dot((np.dot(X_train,coefficient.T) - Y_train).T, X_train) # Compute Gradients
    coefficient = coefficient - (alpha * grad) # adjust coefficients including intercept

小调整以使代码完全可重现。我没有遇到任何问题。 MSE有一点点差异，但是两个分数都小于1e-11，所以是数值问题。