随机森林：寻找相关特征

Question

我正在尝试在 sklearn 中训练一个 RF 模型进行分类。对于一组指定的特征向量，我得到的测试准确度非常低。我假设我选择的特征向量误导了模型。所以我尝试了 RFE、RFECV 等来找到一组相关的特征向量 - 无助于提高准确性。我想出了一个简单的特征选择过程如下>

ml_feats = #initial set of feature vector

while True
    feats_to_del=[]
    prev_score=0
    for feat_len in range(2,len(ml_feats)):
        classifier = RandomForestClassifier(**init_params)
        classifier.fit(X[ml_feats[:feat_len]],Y)
        score = classifier.score(Xt[ml_feats[:feat_len]],Yt)
        if score<prev_score:
             #feature that caused the score to decrease
             print ml_feats[feat_len]
             feat_to_del.append(ml_feats[feat_len])
        prev_score=score
    if len(feats_to_del)==0:
        break
    #delete irrelevant features
    ml_feats=list(set(ml_feats)-set(feats_to_del))

print ml_feats #print all relevant features

上面的代码是否有助于找出正确的功能集？ 谢谢

Answer 1

你正在做的是一个贪婪的特征选择。如果你想使用 RandomForestClassifier 来选择特征，你可以这样做：

from sklearn.ensemble import RandomForestClassifier
from sklearn.feature_selection import SelectFromModel
# xtrain : training data
# ytrain : training labels

clf = RandomForestClassifier()
sfm = SelectFromModel(estimator=clf, threshold='mean') # threshold of selection is mean of feature importances by random forest classifier
sfm.fit(xtrain, ytrain)
selected_xtrain = sfm.transform(xtrain)