如何将正则化参数传递给模型选择（sklearn）？

Question

我有一个简单的岭回归模型，如下所示：

import numpy as np
import sklearn.linear_model
import sklearn.model_selection
import matplotlib.pyplot as plt

n_samples_train, n_samples_test, n_features = 75, 150, 500
np.random.seed(0)
coef = np.random.randn(n_features)
coef[50:] = 0.0  # only the top 10 features are impacting the model
X = np.random.randn(n_samples_train + n_samples_test, n_features)
y = np.dot(X, coef)

ridge = linear_model.Ridge(alpha=0.1, fit_intercept=False)

fit_params = {'alpha': 0.1, 'alpha': 1, 'alpha': 10}
ms = model_selection.cross_validate(ridge, X, y, cv=10, verbose=3, scoring='neg_mean_squared_error', n_jobs=-1, return_train_score=True)

f = plt.figure(figsize=(10,7))

ax = f.add_subplot(111)
_ = ax.scatter(np.arange(len(ms['train_score'])), ms['train_score'])
_ = ax.scatter(np.arange(len(ms['train_score'])), ms['test_score'])

ax.set_xlabel('Regularization Parameter')
ax.set_ylabel('Negative '+r'$MSE$')

这会生成一些虚拟数据。它将拟合 alpha=0.1 的 Ridge 回归，然后绘制训练和测试误差。 我的问题是如何在模型选择中加入 fit_params？ 我尝试使用该字典 fit_params，但它给了我一个错误。文档没有说明键应该是什么（我怀疑它实际上是样本权重......），在这种情况下，如何使用不同的 alpha 对选择进行建模？ （RidgeCV 不起作用，因为它不会绘制训练/测试错误图）

为什么不使用 RidgeCV？：

1) 如果您想存储 cv_errors，RidgeCV 可以与 leave-one-out 一起使用，但不能与 10 倍验证（或除 LOO 之外的任何交叉验证）一起使用。

错误信息：

---------------------------------------------------------------------------
RemoteTraceback                           Traceback (most recent call last)
RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/nazariy/anaconda/envs/logging/lib/python3.6/site-packages/sklearn/externals/joblib/_parallel_backends.py", line 350, in __call__
    return self.func(*args, **kwargs)
  File "/home/nazariy/anaconda/envs/logging/lib/python3.6/site-packages/sklearn/externals/joblib/parallel.py", line 131, in __call__
    return [func(*args, **kwargs) for func, args, kwargs in self.items]
  File "/home/nazariy/anaconda/envs/logging/lib/python3.6/site-packages/sklearn/externals/joblib/parallel.py", line 131, in <listcomp>
    return [func(*args, **kwargs) for func, args, kwargs in self.items]
  File "/home/nazariy/anaconda/envs/logging/lib/python3.6/site-packages/sklearn/model_selection/_validation.py", line 458, in _fit_and_score
    estimator.fit(X_train, y_train, **fit_params)
TypeError: fit() got an unexpected keyword argument 'alpha'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/nazariy/anaconda/envs/logging/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/home/nazariy/anaconda/envs/logging/lib/python3.6/site-packages/sklearn/externals/joblib/_parallel_backends.py", line 359, in __call__
    raise TransportableException(text, e_type)
sklearn.externals.joblib.my_exceptions.TransportableException: TransportableException
___________________________________________________________________________
TypeError                                          Thu Jun  7 19:32:53 2018
PID: 1372      Python 3.6.4: /home/nazariy/anaconda/envs/logging/bin/python
...........................................................................
/home/nazariy/anaconda/envs/logging/lib/python3.6/site-packages/sklearn/externals/joblib/parallel.py in __call__(self=<sklearn.externals.joblib.parallel.BatchedCalls object>)
    126     def __init__(self, iterator_slice):
    127         self.items = list(iterator_slice)
    128         self._size = len(self.items)
    129 
    130     def __call__(self):
--> 131         return [func(*args, **kwargs) for func, args, kwargs in self.items]
        self.items = [(<function _fit_and_score>, (Ridge(alpha=0.1, copy_X=True, fit_intercept=Fals...lse, random_state=None, solver='auto', tol=0.001), array([[ 1],
       [ 2],
       [ 3],
       [ ...    [ 7],
       [ 8],
       [ 9],
       [10]]), array([ 1. ,  3.5,  4. ,  4.9,  6.1,  7.2,  8.1,  8.9, 10. , 11.1]), {'score': make_scorer(mean_squared_error, greater_is_better=False)}, array([1, 2, 3, 4, 5, 6, 7, 8, 9]), array([0]), 3, None, {'alpha': 10}), {'return_times': True, 'return_train_score': True})]
    132 
    133     def __len__(self):
    134         return self._size
    135 

...........................................................................
/home/nazariy/anaconda/envs/logging/lib/python3.6/site-packages/sklearn/externals/joblib/parallel.py in <listcomp>(.0=<list_iterator object>)
    126     def __init__(self, iterator_slice):
    127         self.items = list(iterator_slice)
    128         self._size = len(self.items)
    129 
    130     def __call__(self):
--> 131         return [func(*args, **kwargs) for func, args, kwargs in self.items]
        func = <function _fit_and_score>
        args = (Ridge(alpha=0.1, copy_X=True, fit_intercept=Fals...lse, random_state=None, solver='auto', tol=0.001), array([[ 1],
       [ 2],
       [ 3],
       [ ...    [ 7],
       [ 8],
       [ 9],
       [10]]), array([ 1. ,  3.5,  4. ,  4.9,  6.1,  7.2,  8.1,  8.9, 10. , 11.1]), {'score': make_scorer(mean_squared_error, greater_is_better=False)}, array([1, 2, 3, 4, 5, 6, 7, 8, 9]), array([0]), 3, None, {'alpha': 10})
        kwargs = {'return_times': True, 'return_train_score': True}
    132 
    133     def __len__(self):
    134         return self._size
    135 

...........................................................................
/home/nazariy/anaconda/envs/logging/lib/python3.6/site-packages/sklearn/model_selection/_validation.py in _fit_and_score(estimator=Ridge(alpha=0.1, copy_X=True, fit_intercept=Fals...lse, random_state=None, solver='auto', tol=0.001), X=array([[ 1],
       [ 2],
       [ 3],
       [ ...    [ 7],
       [ 8],
       [ 9],
       [10]]), y=array([ 1. ,  3.5,  4. ,  4.9,  6.1,  7.2,  8.1,  8.9, 10. , 11.1]), scorer={'score': make_scorer(mean_squared_error, greater_is_better=False)}, train=array([1, 2, 3, 4, 5, 6, 7, 8, 9]), test=array([0]), verbose=3, parameters=None, fit_params={'alpha': 10}, return_train_score=True, return_parameters=False, return_n_test_samples=False, return_times=True, error_score='raise')
    453 
    454     try:
    455         if y_train is None:
    456             estimator.fit(X_train, **fit_params)
    457         else:
--> 458             estimator.fit(X_train, y_train, **fit_params)
        estimator.fit = <bound method Ridge.fit of Ridge(alpha=0.1, copy...se, random_state=None, solver='auto', tol=0.001)>
        X_train = array([[ 2],
       [ 3],
       [ 4],
       [ ...    [ 7],
       [ 8],
       [ 9],
       [10]])
        y_train = array([ 3.5,  4. ,  4.9,  6.1,  7.2,  8.1,  8.9, 10. , 11.1])
        fit_params = {'alpha': 10}
    459 
    460     except Exception as e:
    461         # Note fit time as time until error
    462         fit_time = time.time() - start_time

TypeError: fit() got an unexpected keyword argument 'alpha'
___________________________________________________________________________
"""

The above exception was the direct cause of the following exception:

TransportableException                    Traceback (most recent call last)
~/anaconda/envs/logging/lib/python3.6/site-packages/sklearn/externals/joblib/parallel.py in retrieve(self)
    698                 if getattr(self._backend, 'supports_timeout', False):
--> 699                     self._output.extend(job.get(timeout=self.timeout))
    700                 else:

~/anaconda/envs/logging/lib/python3.6/multiprocessing/pool.py in get(self, timeout)
    643         else:
--> 644             raise self._value
    645 

TransportableException: TransportableException
___________________________________________________________________________
TypeError                                          Thu Jun  7 19:32:53 2018
PID: 1372      Python 3.6.4: /home/nazariy/anaconda/envs/logging/bin/python
...........................................................................
/home/nazariy/anaconda/envs/logging/lib/python3.6/site-packages/sklearn/externals/joblib/parallel.py in __call__(self=<sklearn.externals.joblib.parallel.BatchedCalls object>)
    126     def __init__(self, iterator_slice):
    127         self.items = list(iterator_slice)
    128         self._size = len(self.items)
    129 
    130     def __call__(self):
--> 131         return [func(*args, **kwargs) for func, args, kwargs in self.items]
        self.items = [(<function _fit_and_score>, (Ridge(alpha=0.1, copy_X=True, fit_intercept=Fals...lse, random_state=None, solver='auto', tol=0.001), array([[ 1],
       [ 2],
       [ 3],
       [ ...    [ 7],
       [ 8],
       [ 9],
       [10]]), array([ 1. ,  3.5,  4. ,  4.9,  6.1,  7.2,  8.1,  8.9, 10. , 11.1]), {'score': make_scorer(mean_squared_error, greater_is_better=False)}, array([1, 2, 3, 4, 5, 6, 7, 8, 9]), array([0]), 3, None, {'alpha': 10}), {'return_times': True, 'return_train_score': True})]
    132 
    133     def __len__(self):
    134         return self._size
    135 

...........................................................................
/home/nazariy/anaconda/envs/logging/lib/python3.6/site-packages/sklearn/externals/joblib/parallel.py in <listcomp>(.0=<list_iterator object>)
    126     def __init__(self, iterator_slice):
    127         self.items = list(iterator_slice)
    128         self._size = len(self.items)
    129 
    130     def __call__(self):
--> 131         return [func(*args, **kwargs) for func, args, kwargs in self.items]
        func = <function _fit_and_score>
        args = (Ridge(alpha=0.1, copy_X=True, fit_intercept=Fals...lse, random_state=None, solver='auto', tol=0.001), array([[ 1],
       [ 2],
       [ 3],
       [ ...    [ 7],
       [ 8],
       [ 9],
       [10]]), array([ 1. ,  3.5,  4. ,  4.9,  6.1,  7.2,  8.1,  8.9, 10. , 11.1]), {'score': make_scorer(mean_squared_error, greater_is_better=False)}, array([1, 2, 3, 4, 5, 6, 7, 8, 9]), array([0]), 3, None, {'alpha': 10})
        kwargs = {'return_times': True, 'return_train_score': True}
    132 
    133     def __len__(self):
    134         return self._size
    135 

...........................................................................
/home/nazariy/anaconda/envs/logging/lib/python3.6/site-packages/sklearn/model_selection/_validation.py in _fit_and_score(estimator=Ridge(alpha=0.1, copy_X=True, fit_intercept=Fals...lse, random_state=None, solver='auto', tol=0.001), X=array([[ 1],
       [ 2],
       [ 3],
       [ ...    [ 7],
       [ 8],
       [ 9],
       [10]]), y=array([ 1. ,  3.5,  4. ,  4.9,  6.1,  7.2,  8.1,  8.9, 10. , 11.1]), scorer={'score': make_scorer(mean_squared_error, greater_is_better=False)}, train=array([1, 2, 3, 4, 5, 6, 7, 8, 9]), test=array([0]), verbose=3, parameters=None, fit_params={'alpha': 10}, return_train_score=True, return_parameters=False, return_n_test_samples=False, return_times=True, error_score='raise')
    453 
    454     try:
    455         if y_train is None:
    456             estimator.fit(X_train, **fit_params)
    457         else:
--> 458             estimator.fit(X_train, y_train, **fit_params)
        estimator.fit = <bound method Ridge.fit of Ridge(alpha=0.1, copy...se, random_state=None, solver='auto', tol=0.001)>
        X_train = array([[ 2],
       [ 3],
       [ 4],
       [ ...    [ 7],
       [ 8],
       [ 9],
       [10]])
        y_train = array([ 3.5,  4. ,  4.9,  6.1,  7.2,  8.1,  8.9, 10. , 11.1])
        fit_params = {'alpha': 10}
    459 
    460     except Exception as e:
    461         # Note fit time as time until error
    462         fit_time = time.time() - start_time

TypeError: fit() got an unexpected keyword argument 'alpha'
___________________________________________________________________________

During handling of the above exception, another exception occurred:

Answer 1

我建议您使用GridSearchCV。这将处理您的目标。

import numpy as np
import sklearn.linear_model as linear_model
import sklearn.model_selection as model_selection
import matplotlib.pyplot as plt

n_samples_train, n_samples_test, n_features = 75, 150, 500
np.random.seed(0)
coef = np.random.randn(n_features)
coef[50:] = 0.0  # only the top 10 features are impacting the model
X = np.random.randn(n_samples_train + n_samples_test, n_features)
y = np.dot(X, coef)

# instantiate your ridge model first
ridge = linear_model.Ridge(fit_intercept=False)

# now create your grid search object
grid = model_selection.GridSearchCV(ridge, param_grid={"alpha": [0.1, 1, 10]},
                                    cv=10, scoring='neg_mean_squared_error',
                                    n_jobs=-1, return_train_score=True)

# fit
grid.fit(X, y)

由于您希望能够检索 CV 分数，您可以使用网格搜索对象的 cv_results_ 属性访问它们。该属性将保存每个折叠的测试分数（以及训练分数，如果您设置了return_train_score=True）以及所有折叠的平均分数：

# show cv results
grid.cv_results_

简历结果：

{'mean_fit_time': array([ 0.03192089,  0.00980701,  0.00800555]),
 'mean_score_time': array([ 0.00030019,  0.00010006,  0.00020015]),
 'mean_test_score': array([-39.76136733, -39.7700976 , -39.90061844]),
 'mean_train_score': array([ -4.55700050e-06,  -4.51109497e-04,  -4.10175706e-02]),
 'param_alpha': masked_array(data = [0.1 1 10],
              mask = [False False False],
        fill_value = ?),
 'params': [{'alpha': 0.1}, {'alpha': 1}, {'alpha': 10}],
 'rank_test_score': array([1, 2, 3]),
 'split0_test_score': array([-46.32878735, -46.33132325, -46.42467545]),
 'split0_train_score': array([ -4.33377675e-06,  -4.29368239e-04,  -3.93263016e-02]),
 'split1_test_score': array([-23.65685521, -23.70826719, -24.23222395]),
 'split1_train_score': array([ -4.71698023e-06,  -4.66957024e-04,  -4.24618411e-02]),
 'split2_test_score': array([-25.10691203, -25.11680407, -25.25664803]),
 'split2_train_score': array([ -5.07409398e-06,  -5.02011049e-04,  -4.53910939e-02]),
 'split3_test_score': array([-49.02718917, -48.98855648, -48.69939824]),
 'split3_train_score': array([ -4.48268791e-06,  -4.43818654e-04,  -4.04080484e-02]),
 'split4_test_score': array([-58.25312869, -58.30869711, -58.89565988]),
 'split4_train_score': array([ -4.39368907e-06,  -4.35091383e-04,  -3.96619155e-02]),
 'split5_test_score': array([-34.55649537, -34.61271569, -35.15148114]),
 'split5_train_score': array([ -4.79768741e-06,  -4.74334047e-04,  -4.26642818e-02]),
 'split6_test_score': array([-48.89509143, -48.92121206, -49.21661278]),
 'split6_train_score': array([ -4.27579707e-06,  -4.23581125e-04,  -3.87674266e-02]),
 'split7_test_score': array([-37.843457  , -37.74098694, -36.80684638]),
 'split7_train_score': array([ -4.18314427e-06,  -4.14549050e-04,  -3.80652817e-02]),
 'split8_test_score': array([-49.12264863, -49.14574319, -49.42603306]),
 'split8_train_score': array([ -4.42193101e-06,  -4.37800204e-04,  -3.98496419e-02]),
 'split9_test_score': array([-24.66101592, -24.66289001, -24.7145367 ]),
 'split9_train_score': array([ -4.89021729e-06,  -4.83584192e-04,  -4.35798731e-02]),
 'std_fit_time': array([ 0.00705221,  0.01158253,  0.00279475]),
 'std_score_time': array([ 0.00045855,  0.00030019,  0.0004003 ]),
 'std_test_score': array([ 11.77428115,  11.77462622,  11.79882886]),
 'std_train_score': array([  2.79473118e-07,   2.73681039e-05,   2.25174600e-03])}

分数数组中的每个值都对应于作为参数传递给param_grid 的alpha 列表中的位置值。因此，如果我们查看split0_test_score，值-46.32878735对应alpha=0.1，-46.33132325对应alpha=1，-46.42467545对应alpha=10。