基于 .coef_ 值的百分位使用 SVC 进行 Scikit-Learn 特征选择

Question

我正在尝试编写一个 Python 类，以便使用 .coef_ 属性值来选择 scikit-learn 0.17.1 中的功能。我只想选择 .coef_ 值在第 10 个百分位及以上的特征（第 10、第 11、第 12、第 13、第 14、第 15、第 16、第 ....、第 94、第 95、第 96、第 97、第 98、第 99 , 100)。

我无法使用SelectFromModels() 执行此操作，因此我尝试为此功能选择编写一个名为ChooseCoefPercentile() 的自定义类。我正在尝试使用以下函数根据.coef_ 的百分位数选择特征：

from sklearn.datasets import load_iris
from sklearn.cross_validation import train_test_split
X_train, X_test, y_train, y_test = train_test_split(load_iris().data,
                                   load_iris().target, test_size=0.33, random_state=42)

def percentile_sep(coefs,p):
    from numpy import percentile as pc
    gt_p = coefs[coefs>pc(coefs,p)].argsort()
    return list(gt_p)

from sklearn.base import BaseEstimator, TransformerMixin
class ChooseCoefPercentile(BaseEstimator, TransformerMixin):
    def __init__(self, est_, perc=50):
        self.perc = perc
        self.est_ = est_
    def fit(self, *args, **kwargs):
        self.est_.fit(*args, **kwargs)
        return self
    def transform(self, X):
        perc_i = percentile_sep(self.est_.coef_,self.perc)
        i_ = self.est_.coef_.argsort()[::-1][perc_i[:]]
        X_tr = X[:,i_]
        self.coef_ = self.est_.coef_[i_]
        return X_tr

# Import modules
from sklearn import svm,ensemble,pipeline,grid_search

# Instantiate feature selection estimator and classifier
f_sel = ChooseCoefPercentile(svm.SVC(kernel='linear'),perc=10)
clf = ensemble.RandomForestClassifier(random_state=42,oob_score=False)

CustPipe = pipeline.Pipeline([("feat_s",f_sel),("Clf",clf)])
bf_est = grid_search.GridSearchCV(CustPipe,cv=2,param_grid={'Clf__n_estimators':[100,200]})
bf_est.fit(X_train, y_train)

我收到以下错误：

Traceback (most recent call last):
  File "C:\Python27\test.py", line 35, in <module>
    bf_est.fit(X_train, y_train)
  File "C:\Python27\lib\site-packages\sklearn\grid_search.py", line 804, in fit
    return self._fit(X, y, ParameterGrid(self.param_grid))
  File "C:\Python27\lib\site-packages\sklearn\grid_search.py", line 553, in _fit
    for parameters in parameter_iterable
  File "C:\Python27\lib\site-packages\sklearn\externals\joblib\parallel.py", line 800, in __call__
    while self.dispatch_one_batch(iterator):
  File "C:\Python27\lib\site-packages\sklearn\externals\joblib\parallel.py", line 658, in dispatch_one_batch
    self._dispatch(tasks)
  File "C:\Python27\lib\site-packages\sklearn\externals\joblib\parallel.py", line 566, in _dispatch
    job = ImmediateComputeBatch(batch)
  File "C:\Python27\lib\site-packages\sklearn\externals\joblib\parallel.py", line 180, in __init__
    self.results = batch()
  File "C:\Python27\lib\site-packages\sklearn\externals\joblib\parallel.py", line 72, in __call__
    return [func(*args, **kwargs) for func, args, kwargs in self.items]
  File "C:\Python27\lib\site-packages\sklearn\cross_validation.py", line 1531, in _fit_and_score
    estimator.fit(X_train, y_train, **fit_params)
  File "C:\Python27\lib\site-packages\sklearn\pipeline.py", line 164, in fit
    Xt, fit_params = self._pre_transform(X, y, **fit_params)
  File "C:\Python27\lib\site-packages\sklearn\pipeline.py", line 145, in _pre_transform
    Xt = transform.fit_transform(Xt, y, **fit_params_steps[name])
  File "C:\Python27\lib\site-packages\sklearn\base.py", line 458, in fit_transform
    return self.fit(X, y, **fit_params).transform(X)
  File "C:\Python27\test.py", line 21, in transform
    i_ = self.est_.coef_.argsort()[::-1][perc_i[:]]
IndexError: index 6 is out of bounds for axis 0 with size 3

以下行中的 .coef_ 值的 NumPy 数组似乎存在问题：

i_ = self.est_.coef_.argsort()[::-1][perc_i[:]]

在这一行中，我试图根据它们的索引只选择那些位于第 10 个百分位以上的 .coef_ 值。索引存储在列表perc_i 中。我似乎无法使用此列表正确索引 .coef_ 数组。

出现这个错误是因为数组需要分成行吗？还是应该使用其他方法根据百分位数提取 .coef_ 值？

Answer 1

我建议根据行数使用模算术计算系数矩阵的相关列：

def transform(self, X):
    perc_i = percentile_sep(self.est_.coef_,self.perc)
    nclass=self.est_.coef_.shape[0]
    i_ = list(set(map(lambda x:x%nclass,perc_i)))
    X_tr = X[:,i_]
    self.coef_ = self.est_.coef_[i_]
    return X_tr