如何有效地比较所有模型的准确性

Question

我已经拆分了训练数据并初始化了 11 个我现在想要比较的分类器模型。

我在 Ubuntu 18.04 上运行 VS Code。

我试过了：

# Prepare lists
models = [ran, knn, log, xgb, gbc, svc, ext, ada, gnb, gpc, bag]         
scores = []

# Sequentially fit and cross validate all models
for mod in models:
    mod.fit(X_train, y_train)
    acc = cross_val_score(mod, X_train, y_train, scoring = 
    "accuracy", cv = 10)
scores.append(acc.mean())

# Creating a table of results, ranked highest to lowest
results = pd.DataFrame({
    'Model': ['Random Forest', 'K Nearest Neighbour', 'Logistic 
     Regression', 'XGBoost', 'Gradient Boosting', 'SVC', 'Extra 
     Trees', 'AdaBoost', 'Gaussian Naive Bayes', 'Gaussian Process', 
     'Bagging Classifier'],
     'Score': scores})

在最后一部分返回：

ValueError: 数组的长度必须相同

我数了 2 倍，确实有 11 个模型。

我错过了什么？

Answer 1

您的代码中似乎存在缩进错误，请参阅下面的编辑代码。在您的代码中，如果您执行len(scores)，您将获得1，因为只有最后一个值被添加，因为在循环外调用了追加。

# Prepare lists
models = [ran, knn, log, xgb, gbc, svc, ext, ada, gnb, gpc, bag]         
scores = []

# Sequentially fit and cross validate all models
for mod in models:
    mod.fit(X_train, y_train)
    acc = cross_val_score(mod, X_train, y_train, scoring = 
    "accuracy", cv = 10)
    scores.append(acc.mean())

Answer 2

已经对上一个答案投了赞成票，我继续证明该错误确实是由于您的score.append() 在您的for 循环之外：

我们不需要实际拟合任何模型；我们可以通过对您的代码进行以下修改来模拟这种情况，这不会改变问题的本质：

import numpy as np
import pandas as pd

models = ['ran', 'knn', 'log', 'xgb', 'gbc', 'svc', 'ext', 'ada', 'gnb', 'gpc', 'bag']         
scores = []
cv=10

# Sequentially fit and cross validate all models
for mod in models:
    acc = np.array([np.random.rand() for i in range(cv)]) # simulate your accuracy here
scores.append(acc.mean()) # as in your code, i.e outside the for loop

# Create a dataframe of results
results = pd.DataFrame({
    'Model': ['Random Forest', 'K Nearest Neighbour', 'Logistic Regression', 'XGBoost', 'Gradient Boosting',  
    'SVC', 'Extra Trees', 'AdaBoost', 'Gaussian Naive Bayes', 'Gaussian Process', 'Bagging Classifier'],
    'Score': scores})

不出所料，这基本上复制了您的错误：

ValueError: arrays must all be same length

因为，正如在另一个答案中已经讨论过的，您的 scores 列表只有一个元素，即仅来自循环的 last 迭代的 acc.mean()：

len(scores)
# 1
scores
# [0.47317491043203785]

因此 pandas 抱怨，因为它无法填充 11 行数据框...

在for 循环内移动scores.append()，正如另一个答案中已经建议的那样，解决了这个问题：

for mod in models:
    acc = np.array([np.random.rand() for i in range(cv)])
    scores.append(acc.mean()) # moved inside the loop

# Create a dataframe of results
results = pd.DataFrame({
    'Model': ['Random Forest', 'K Nearest Neighbour', 'Logistic Regression', 'XGBoost', 'Gradient Boosting',  
    'SVC', 'Extra Trees', 'AdaBoost', 'Gaussian Naive Bayes', 'Gaussian Process', 'Bagging Classifier'],
    'Score': scores})
print(results)
# output:
                   Model     Score
0          Random Forest  0.492364
1    K Nearest Neighbour  0.624068
2    Logistic Regression  0.613653
3                XGBoost  0.536488
4      Gradient Boosting  0.484195
5                    SVC  0.381556
6            Extra Trees  0.274922
7               AdaBoost  0.509297
8   Gaussian Naive Bayes  0.362866
9       Gaussian Process  0.606538
10    Bagging Classifier  0.393950

您可能还需要记住，您的代码中不需要 model.fit() 部分 - cross_val_score 会自行完成所有必要的拟合...