如何在从模型中选择特征时执行随机网格搜索？

Question

我希望根据随机森林、梯度提升和极端梯度提升的特征重要性来选择特征。我正在尝试使用随机化网格搜索来拟合我的模型以获得最佳模型的特征重要性，但这给了我一个我不明白的错误，这是我的代码：

gbr = GradientBoostingRegressor(random_state=seed)
gbr_params = {
    "learning_rate": [0.001, 0.01, 0.1],
    "min_samples_split": [50, 100],
    "min_samples_leaf": [50, 100],
    "max_depth":[5, 10, 20]}

xgbr = xgboost.XGBRegressor(random_state=seed) 
xgbr_params = {  
    "learning_rate": [0.001, 0.01, 0.1],
    "min_samples_leaf": [50, 100],
    "max_depth":[5, 10, 20],
    'reg_alpha': [1.1, 1.2, 1.3],
    'reg_lambda': [1.1, 1.2, 1.3]}

rfr = RandomForestRegressor(random_state=seed)
rfr_params={'n_estimators':[100, 500, 1000], 
             'max_features':[10,14,18],
             'min_samples_split': [50, 100],
             'min_samples_leaf': [50, 100],} 

fs_xgbr = dcv.RandomizedSearchCV(xgbr, xgbr_params, cv=5, iid=False, n_jobs=-1)
fs_gbr = dcv.RandomizedSearchCV(gbr, gbr_params, cv=5,iid=False, n_jobs=-1)
fs_rfr = dcv.RandomizedSearchCV(rfr, rfr_params, cv=5,iid=False, n_jobs=-1)

fs_rfr.fit(X, Y)
model = SelectFromModel(fs_rfr, prefit=True)
X_rfr = model.transform(X)
print('rfr', X_rfr.shape)

在X_rfr = model.transform(X) 的行它给出了这个错误：

ValueError: The underlying estimator RandomizedSearchCV has no `coef_` or `feature_importances_` attribute. Either pass a fitted estimator to SelectFromModel or call fit before calling transform.

我不是程序员，也没有在其他地方找到任何解决方案来解决这个问题，难道不能用随机搜索决定的最佳参数来获取模型的 feature_importances_ 吗？

Answer 1

不要传递给SelectFromModelfs_rfr，它是RandomizedSearchCV类型的对象，而是传递最佳估计器，例如fs_rfr.best_estimator_

证明

import xgboost
from sklearn.ensemble import GradientBoostingRegressor, RandomForestRegressor
from sklearn.datasets import make_regression
from sklearn.model_selection import RandomizedSearchCV
from sklearn.feature_selection import SelectFromModel

seed=42

gbr = GradientBoostingRegressor(random_state=seed)
gbr_params = {
    "learning_rate": [0.001, 0.01, 0.1],
    "min_samples_split": [50, 100],
    "min_samples_leaf": [50, 100],
    "max_depth":[5, 10, 20]}

xgbr = xgboost.XGBRegressor(random_state=seed) 
xgbr_params = {  
    "learning_rate": [0.001, 0.01, 0.1],
    "min_samples_leaf": [50, 100],
    "max_depth":[5, 10, 20],
    'reg_alpha': [1.1, 1.2, 1.3],
    'reg_lambda': [1.1, 1.2, 1.3]}

rfr = RandomForestRegressor(random_state=seed)
rfr_params={'n_estimators':[100, 500, 1000], 
             'max_features':[10,14,18],
             'min_samples_split': [50, 100],
             'min_samples_leaf': [50, 100],} 

fs_xgbr = RandomizedSearchCV(xgbr, xgbr_params, cv=5, iid=False, n_jobs=-1)
fs_gbr = RandomizedSearchCV(gbr, gbr_params, cv=5,iid=False, n_jobs=-1)
fs_rfr = RandomizedSearchCV(rfr, rfr_params, cv=5,iid=False, n_jobs=-1)

X, y = make_regression(1000,10)

fs_xgbr.fit(X, y)
fs_gbr.fit(X, y)
fs_rfr.fit(X, y)

model = SelectFromModel(fs_rfr.best_estimator_, prefit=True)
X_rfr = model.transform(X)
print('rfr', X_rfr.shape)

model = SelectFromModel(fs_xgbr.best_estimator_, prefit=True)
X_xgbr = model.transform(X)
print('xgbr', X_xgbr.shape)

model = SelectFromModel(fs_gbr.best_estimator_, prefit=True)
X_gbr = model.transform(X)
print('gbr', X_gbr.shape)

rfr (1000, 3)
xgbr (1000, 3)
gbr (1000, 4)