Scikit-learn 线性模型，coef_ 为特征返回高值

Question

问题陈述：在给定客户订购某些物品（例如：靴子、运动鞋等）的情况下，预测快递包裹的重量

所以我拥有的数据框由历史数据组成，其中 product_item_categories（例如：靴子、运动鞋等）构成特征，而重量是我要预测的“y”变量。数据框的每一行都包含客户订购了多少 product_item_categories 的计数。

示例：客户订购了 1 双靴子、1 双运动鞋。该行看起来像：

x1  x2  x3  x4  x5  x6  x7  x8  x9  x10 x11 x12 x13 x14 x15 x16 x17 x18 x19 x20 x21 x22 x23 x24 x25 x26 x27 x28 x29 x30 x31 x32 x33 x34 x35 x36 x37 x38 x39 x40 x41 x42 x43 x44 x45 x46 x47 y
1   0   0   0   0   0   0   0   0   0   0   0   1   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   2   2.94

其中一个功能是 items_total，这里是 x47（客户总共订购了多少商品）。

我使用

创建了一个线性模型

regr_model = linear_model.LinearRegression()

将数据框拆分为训练集和测试集后，我使用regr_model.fit(x_train, y_train) 运行模型

当我查看系数时，我得到以下输出（格式化后更有意义）

1   feature x1  6494532107.689080 (this is the items_total feature)
2   feature x2  (-6494532105.548431)
3   feature x3  (-6494532105.956598)
4   feature x4  (-6494532105.987348)
5   feature x5  (-6494532106.081478)
6   feature x6  (-6494532106.139558)
7   feature x7  (-6494532106.163167)
8   feature x8  (-6494532106.326231)
9   feature x9  (-6494532106.360985)
10  feature x10 (-6494532106.507434)
11  feature x11 (-6494532106.678183)
12  feature x12 (-6494532106.711108)
13  feature x13 (-6494532106.906321)
14  feature x14 (-6494532106.916800)
15  feature x15 (-6494532106.941691)
16  feature x16 (-6494532107.049221)
17  feature x17 (-6494532107.071664)
18  feature x18 (-6494532107.076819)
19  feature x19 (-6494532107.095350)
20  feature x20 (-6494532107.124458)
21  feature x21 (-6494532107.208526)
22  feature x22 (-6494532107.291896)
23  feature x23 (-6494532107.315606)
24  feature x24 (-6494532107.319578)
25  feature x25 (-6494532107.322818)
26  feature x26 (-6494532107.337678)
27  feature x27 (-6494532107.345344)
28  feature x28 (-6494532107.347136)
29  feature x29 (-6494532107.374278)
30  feature x30 (-6494532107.403748)
31  feature x31 (-6494532107.405770)
32  feature x32 (-6494532107.411852)
33  feature x33 (-6494532107.469144)
34  feature x34 (-6494532107.470899)
35  feature x35 (-6494532107.471970)
36  feature x36 (-6494532107.489899)
37  feature x37 (-6494532107.495930)
38  feature x38 (-6494532107.504712)
39  feature x39 (-6494532107.522346)
40  feature x40 (-6494532107.557917)
41  feature x41 (-6494532107.561793)
42  feature x42 (-6494532107.562286)
43  feature x43 (-6494532107.601017)
44  feature x44 (-6494532107.603461)
45  feature x45 (-6494532107.686674)
46  feature x46 (-6494532107.843128)
47  feature x47 (-6494532107.910987)

截距为：0.555702083558 模型得分为：0.79

当我删除 items_total 时。我得到了更有意义的系数：

1   feature x2  2.140582
2   feature x3  1.732328
3   feature x4  1.701661
4   feature x5  1.607465
5   feature x6  1.549196
6   feature x7  1.526227
7   feature x8  1.363067
8   feature x9  1.329225
9   feature x10 1.18109
10  feature x11 1.010639
11  feature x12 0.978123
12  feature x13 0.782569
13  feature x14 0.773164
14  feature x15 0.747479
15  feature x16 0.638743
16  feature x17 0.617082
17  feature x18 0.61257
18  feature x19 0.593665
19  feature x20 0.565309
20  feature x21 0.480105
21  feature x22 0.396592
22  feature x23 0.373675
23  feature x24 0.369643
24  feature x25 0.365989
25  feature x26 0.350971
26  feature x27 0.343381
27  feature x28 0.34158
28  feature x29 0.314405
29  feature x30 0.285344
30  feature x31 0.282827
31  feature x32 0.277007
32  feature x33 0.219727
33  feature x34 0.217814
34  feature x35 0.217466
35  feature x36 0.198526
36  feature x37 0.193277
37  feature x38 0.184332
38  feature x39 0.166745
39  feature x40 0.130655
40  feature x41 0.127573
41  feature x42 0.126665
42  feature x43 0.087371
43  feature x44 0.085545
44  feature x45 0.003045
45  feature x46 (-0.153778)
46  feature x47 (-0.221548)

模型的截距和分数相同。当我删除 items_total 列时，有人可以帮我理解系数有何不同吗？

Answer 1

我认为这主要是理论上的问题。 最好在https://stats.stackexchange.com/ 或https://datascience.stackexchange.com/ 中问这个问题

它叫Multicollinearity。

我将提供更好的示例来演示该问题，此示例可在俄语版的维基百科页面中找到： 假设您具有以下功能：x1、x2、x3，其中x1 = x2+x3 所以我们有一个看起来像 的模型。

让我们在b1 中添加一些任意的a，并从b2 和b3 中减去a：

所以我们在随机修改系数后得到了相同的模型，这就是问题所在。因此，您应该避免特征之间存在如此强的相关性（您的最后一个特征与所有其他特征相关）。