scikit-learn 中详尽的特征选择？

Question

在 scikit-learn 中是否有任何内置的蛮力特征选择方法？ IE。详尽地评估输入特征的所有可能组合，然后找到最佳子集。我熟悉“递归特征消除”类，但我特别感兴趣的是一个接一个地评估输入特征的所有可能组合。

Answer 1

您可能想看看MLxtend's Exhaustive Feature Selector。它显然没有内置在 scikit-learn 中（还没有？），但确实支持它的分类器和回归器对象。

Answer 2

结合 Fred Foo 的答案以及 nopper、ihadanny 和 jimijazz 的 cmets，以下代码得到的结果与 Lab 1 (6.5.5) 中第一个示例的 R 函数 regsubsets()（leaps 库的一部分）相同。 1 Best Subset Selection）在“An Introduction to Statistical Learning with Applications in R”一书中。

from itertools import combinations
from sklearn.cross_validation import cross_val_score

def best_subset(estimator, X, y, max_size=8, cv=5):
'''Calculates the best model of up to max_size features of X.
   estimator must have a fit and score functions.
   X must be a DataFrame.'''

    n_features = X.shape[1]
    subsets = (combinations(range(n_features), k + 1) 
               for k in range(min(n_features, max_size)))

    best_size_subset = []
    for subsets_k in subsets:  # for each list of subsets of the same size
        best_score = -np.inf
        best_subset = None
        for subset in subsets_k: # for each subset
            estimator.fit(X.iloc[:, list(subset)], y)
            # get the subset with the best score among subsets of the same size
            score = estimator.score(X.iloc[:, list(subset)], y)
            if score > best_score:
                best_score, best_subset = score, subset
        # to compare subsets of different sizes we must use CV
        # first store the best subset of each size
        best_size_subset.append(best_subset)

    # compare best subsets of each size
    best_score = -np.inf
    best_subset = None
    list_scores = []
    for subset in best_size_subset:
        score = cross_val_score(estimator, X.iloc[:, list(subset)], y, cv=cv).mean()
        list_scores.append(score)
        if score > best_score:
            best_score, best_subset = score, subset

    return best_subset, best_score, best_size_subset, list_scores

查看笔记本http://nbviewer.jupyter.org/github/pedvide/ISLR_Python/blob/master/Chapter6_Linear_Model_Selection_and_Regularization.ipynb#6.5.1-Best-Subset-Selection

Answer 3

否，未实施最佳子集选择。最简单的方法是自己编写。这应该可以帮助您开始：

from itertools import chain, combinations
from sklearn.cross_validation import cross_val_score

def best_subset_cv(estimator, X, y, cv=3):
    n_features = X.shape[1]
    subsets = chain.from_iterable(combinations(xrange(k), k + 1)
                                  for k in xrange(n_features))

    best_score = -np.inf
    best_subset = None
    for subset in subsets:
        score = cross_val_score(estimator, X[:, subset], y, cv=cv).mean()
        if score > best_score:
            best_score, best_subset = score, subset

    return best_subset, best_score

这会在循环内执行 k-fold 交叉验证，因此在使用 提供数据时它将适合 k 2 个 ᵖ 估计>p 特征。