在嵌套回归器上使用 Pipeline 和 TransformedTargetRegressor 处理数据和目标缩放

Question

我想同时使用 Pipeline 和 TransformedTargetRegressor 来处理 BaggingRegressor 及其所有估算器上的所有缩放（数据和目标）。

我的第一次尝试效果很好（不使用 Pipeline 和 TransformedTargetRegressor）

$ cat test1.py
#!/usr/bin/python
# -*- coding: UTF-8 -*-

import numpy as np
import matplotlib.pyplot as plt
from sklearn.ensemble import BaggingRegressor
from sklearn.svm import SVR

def f(x):
    return x*np.cos(x) + np.random.normal(size=500)*2

def main():
    # Generate random data.
    x = np.linspace(0, 10, 500)
    rng = np.random.RandomState(0)
    rng.shuffle(x)
    x = np.sort(x[:])
    y = f(x)

    # Plot random data.
    fig, axis = plt.subplots(1, 1, figsize=(20, 10))
    axis.plot(x, y, 'o', color='black', markersize=2, label='random data')

    # Create bagging models.
    model = BaggingRegressor(n_estimators=5, base_estimator=SVR())
    x_augmented = np.array([x, x**2, x**3, x**4, x**5]).T
    model.fit(x_augmented, y)

    # Plot intermediate regression estimations.
    axis.plot(x, model.predict(x_augmented), '-', color='red', label=model.__class__.__name__)
    for i, tree in enumerate(model.estimators_):
        y_pred = tree.predict(x_augmented)
        axis.plot(x, y_pred, '--', label='tree '+str(i))

    axis.axis('off')
    axis.legend()
    plt.show()

if __name__ == '__main__':
    main()

这没关系：bagging regressor 被叠加到所有估计器上

现在我想使用 Pipeline 和 TransformedTargetRegressor 来处理数据和目标的所有缩放，但它不起作用，因为 bagging 缩放与估算器缩放不同：

$ cat test2.py
#!/usr/bin/python
# -*- coding: UTF-8 -*-

import numpy as np
import matplotlib.pyplot as plt
from sklearn.ensemble import BaggingRegressor
from sklearn.svm import SVR
from sklearn import preprocessing
from sklearn.pipeline import Pipeline
from sklearn.compose import TransformedTargetRegressor

def f(x):
    return x*np.cos(x) + np.random.normal(size=500)*2

def main():
    # Generate random data.
    x = np.linspace(0, 10, 500)
    rng = np.random.RandomState(0)
    rng.shuffle(x)
    x = np.sort(x[:])
    y = f(x)

    # Plot random data.
    fig, axis = plt.subplots(1, 1, figsize=(20, 10))
    axis.plot(x, y, 'o', color='black', markersize=2, label='random data')

    # Create bagging models.
    model = BaggingRegressor(n_estimators=5, base_estimator=SVR())
    x_augmented = np.array([x, x**2, x**3, x**4, x**5]).T
    pipe = Pipeline([('scale', preprocessing.StandardScaler()), ('model', model)])
    treg = TransformedTargetRegressor(regressor=pipe, transformer=preprocessing.MinMaxScaler())
    treg.fit(x_augmented, y)

    # Plot intermediate regression estimations.
    axis.plot(x, treg.predict(x_augmented), '-', color='red', label=model.__class__.__name__)
    for i, tree in enumerate(treg.regressor_['model'].estimators_):
        y_hat = tree.predict(x_augmented)
        y_transformer = preprocessing.MinMaxScaler().fit(y.reshape(-1, 1))
        y_pred = y_transformer.inverse_transform(y_hat.reshape(-1, 1))
        axis.plot(x, y_pred, '--', label='tree '+str(i))
    axis.axis('off')
    axis.legend()
    plt.show()

if __name__ == '__main__':
    main()

如何正确处理 bagging 回归器及其所有嵌套估计器的缩放？

2 个测试之间的区别在于使用 Pipeline 和 TransformedTargetRegressor

$ diff test1.py  test2.py
7a8,10
> from sklearn import preprocessing
> from sklearn.pipeline import Pipeline
> from sklearn.compose import TransformedTargetRegressor
27c30,32
<     model.fit(x_augmented, y)
---
>     pipe = Pipeline([('scale', preprocessing.StandardScaler()), ('model', model)])
>     treg = TransformedTargetRegressor(regressor=pipe, transformer=preprocessing.MinMaxScaler())
>     treg.fit(x_augmented, y)
30,32c35,39
<     axis.plot(x, model.predict(x_augmented), '-', color='red', label=model.__class__.__name__)
<     for i, tree in enumerate(model.estimators_):
<         y_pred = tree.predict(x_augmented)
---
>     axis.plot(x, treg.predict(x_augmented), '-', color='red', label=model.__class__.__name__)
>     for i, tree in enumerate(treg.regressor_['model'].estimators_):
>         y_hat = tree.predict(x_augmented)
>         y_transformer = preprocessing.MinMaxScaler().fit(y.reshape(-1, 1))
>         y_pred = y_transformer.inverse_transform(y_hat.reshape(-1, 1))

编辑

尝试使用 TransformedTargetRegressor 实例的 tranformer_ 成员：test3（与 test2 相同，直至以下差异）也失败了！...

$ diff test2.py test3.py
38c38
<         y_transformer = preprocessing.MinMaxScaler().fit(y.reshape(-1, 1))
---
>         y_transformer = treg.transformer_

Answer 1

我不认为你的代码有问题，而是绘图部分有问题。

# Plot intermediate regression estimations.
    axis.plot(x, treg.predict(x_augmented), '-', color='red', label=model.__class__.__name__)
    for i, tree in enumerate(treg.regressor_['model'].estimators_):
        y_hat = tree.predict(x_augmented)
        y_transformer = preprocessing.MinMaxScaler().fit(y.reshape(-1, 1))
        y_pred = y_transformer.inverse_transform(y_hat.reshape(-1, 1))
        axis.plot(x, y_pred, '--', label='tree '+str(i))

tree 这里将是一个SVR()，而您预测的是x_augmented，而在上一部分中，x_augmented 是用StandardScaler 缩放的。因此，预测与您的预期不符。

所以用下面的sn-p改一下代码就好了：

# Plot intermediate regression estimations.
axis.plot(x, treg.predict(x_augmented), '-', color='red', label=model.__class__.__name__)
for i, tree in enumerate(treg.regressor_['model'].estimators_):
    x_augmented_scaled = treg.regressor_.named_steps['scale'].transform(x_augmented)
    y_hat = tree.predict(x_augmented_scaled)
    y_transformer = preprocessing.MinMaxScaler().fit(y.reshape(-1, 1))
    y_pred = y_transformer.inverse_transform(y_hat.reshape(-1, 1))
    axis.plot(x, y_pred, '--', label='tree '+str(i))
axis.axis('off')
axis.legend()
plt.show()