Python：使用 scikit-learn 进行预测，给出空白预测

Question

我在客户支持部门工作，我正在使用 scikit-learn 来预测我们的票证的标签，给定一组票证（训练集中大约 40,000 张票证）。

我正在使用基于this one 的分类模型。它预测只是“（）”作为我的许多测试票证的标签，即使训练集中的票证都没有标签。

我的标签训练数据是一个列表列表，例如：

tags_train = [['international_solved'], ['from_build_guidelines my_new_idea eligibility'], ['dropbox other submitted_faq submitted_help'], ['my_new_idea_solved'], ['decline macro_backer_paypal macro_prob_errored_pledge_check_credit_card_us loading_problems'], ['dropbox macro__turnaround_time other plq__turnaround_time submitted_help'], ['dropbox macro_creator__logo_style_guide outreach press submitted_help']]

虽然我的工单描述训练数据只是一个字符串列表，例如：

descs_train = ['description of ticket one', 'description of ticket two', etc]

这是我构建模型的代码的相关部分：

import numpy as np
import scipy
from sklearn.pipeline import Pipeline
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.feature_extraction.text import TfidfTransformer
from sklearn.multiclass import OneVsRestClassifier
from sklearn.svm import LinearSVC

# We have lists called tags_train, descs_train, tags_test, descs_test with the test and train data

X_train = np.array(descs_train)
y_train = tags_train
X_test = np.array(descs_test)  

classifier = Pipeline([
    ('vectorizer', CountVectorizer()),
    ('tfidf', TfidfTransformer()),
    ('clf', OneVsRestClassifier(LinearSVC(class_weight='auto')))])

classifier.fit(X_train, y_train)
predicted = classifier.predict(X_test)

但是，“predicted”给出的列表如下所示：

predicted = [(), ('account_solved',), (), ('images_videos_solved',), ('my_new_idea_solved',), (), (), (), (), (), ('images_videos_solved', 'account_solved', 'macro_launched__edit_update other tips'), ('from_guidelines my_new_idea', 'from_guidelines my_new_idea macro__eligibility'), ()]

我不明白为什么当训练集中没有空白 () 时它会预测为空白 ()。它不应该预测最近的标签吗？谁能推荐我正在使用的模型的任何改进？

非常感谢您提前提供的帮助！

Answer 1

问题在于您的 tags_train 变量。根据OneVsRestClassifier 文档，目标需要是“一系列标签序列”，并且您的目标是 one 元素的列表。

以下是您的代码经过编辑的、独立的、可工作的版本。注意tags_train 的变化，尤其是tags_train 是一个单元素元组。

import numpy as np
import scipy
from sklearn.pipeline import Pipeline
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.feature_extraction.text import TfidfTransformer
from sklearn.multiclass import OneVsRestClassifier
from sklearn.svm import LinearSVC


# We have lists called tags_train, descs_train, tags_test, descs_test with the test and train data
tags_train = [('label', ), ('international' ,'solved'), ('international','open')]
descs_train = ['description of ticket one', 'some other ticket two', 'label']

X_train = np.array(descs_train)
y_train = tags_train
X_test = np.array(descs_train)  

classifier = Pipeline([
    ('vectorizer', CountVectorizer()),
    ('tfidf', TfidfTransformer()),
    ('clf', OneVsRestClassifier(LinearSVC(class_weight='auto')))])

classifier = classifier.fit(X_train, y_train)
predicted = classifier.predict(X_test)

print predicted

输出是

[('international',), ('international',), ('international', 'open')]

Answer 2

即使在将目标从一个元素列表转换为序列之后，仍然面临 () 预测