ValueError：在转换期间在第 7 列中发现未知类别 ['?'] - 乳腺癌数据集

Question

我正在尝试使用 1988 年乳腺癌发病率的 UCI 存储库 (https://archive.ics.uci.edu/ml/datasets/Breast+Cancer) 解决分类机器学习问题。我不断收到以下错误，尽管并非始终如一。有时该算法直接运行到训练模型和预测测试准确性，有时它在 OneHotEncoding 上失败并显示以下错误：

ohe = OneHotEncoder()
ohe.fit(X_train)
X_train_encoded = ohe.transform(X_train)
X_test_encoded = ohe.transform(X_test)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-5-2cfd638a5b4d> in <module>()
      2 ohe.fit(X_train)
      3 X_train_encoded = ohe.transform(X_train)
----> 4 X_test_encoded = ohe.transform(X_test)

1 frames
/usr/local/lib/python3.6/dist-packages/sklearn/preprocessing/_encoders.py in _transform(self, X, handle_unknown)
    122                     msg = ("Found unknown categories {0} in column {1}"
    123                            " during transform".format(diff, i))
--> 124                     raise ValueError(msg)
    125                 else:
    126                     # Set the problematic rows to an acceptable value and

ValueError: Found unknown categories ['?'] in column 7 during transform

我尝试在 Colab 和 Spyder 中运行并遇到相同的问题，但不确定我哪里出错了。我在拆分数据集然后编码之前估算缺失值，但即使我删除 SimpleImputer，我仍然会收到错误。

dataset = pd.read_csv('breast-cancer.csv')
X = dataset.iloc[:, :-1].values
y = dataset.iloc[:, -1].values

from sklearn.impute import SimpleImputer
imputer = SimpleImputer(missing_values=np.nan, strategy='most_frequent')
imputer.fit(X)
X_imputed = imputer.transform(X)

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X_imputed, y, test_size = 0.25)

ohe = OneHotEncoder()
ohe.fit(X_train)
X_train_encoded = ohe.transform(X_train)
X_test_encoded = ohe.transform(X_test)

<-- Code stops running here -->

le = LabelEncoder()
le.fit(y_train)
y_train_encoded = le.transform(y_train)
y_test_encoded = le.transform(y_test)

Answer 1

测试数据可能包含训练数据中不存在的新条目。 你能试试这个吗？

ohe = OneHotEncoder(handle_unknown = "ignore")

关于此参数：如果在转换期间存在未知的分类特征，是否引发错误或忽略（默认为引发）。当此参数设置为“忽略”并且在转换过程中遇到未知类别时，此功能的生成的 one-hot 编码列将全为零。

更多：

https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html