从多标签类修改数组

Question

我正在尝试为具有 1、2、3 和 4 类的多标签分类定义一组类/标签，但遇到数组问题时意外还包括以下内容：

multilabel.classes_ 数组(['', ',', '1', '2', '3', '4'], dtype=object)

我只想将 1、2、3、4 作为我的标签，但想不出办法将其删除。

我的代码：

import pandas as pd
import numpy as np
import os
import ast
import seaborn as sns #pip install seaborn
import matplotlib.pyplot as plt
import skmultilearn #pip install scikit-multilearn

from preprocessing.transcription_preprocessing import TranscriptionPreprocessor
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MultiLabelBinarizer
from sklearn.preprocessing import LabelBinarizer
from sklearn.linear_model import SGDClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.svm import LinearSVC
from sklearn.multiclass import OneVsRestClassifier

df = pd.read_csv(r'C:\Users\M94969\Desktop\datasets\prod500.csv')

# Define label variable
y = df['tags']

# Make multilabelbinarizer object
#multilabel = MultiLabelBinarizer()
#y = multilabel.fit_transform(y)
#multilabel.classes_
#pd.DataFrame(y,columns=multilabel.classes_)

labelbinarizer = LabelBinarizer()

fit = labelbinarizer.fit_transform(y)

labelbinarizer.classes_

pd.DataFrame(y,columns=labelbinarizer.classes_)

# Turn texts into sparse matrix
tfidf = TfidfVectorizer(analyzer='word', max_features=1000, max_df=0.50, ngram_range=(1,3))
X = tfidf.fit_transform(df['text'])

tfidf.vocabulary_

# Split data into train/test
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=1)

# Build models
sgd = SGDClassifier()
lr = LogisticRegression(solver = 'lbfgs')
svc = LinearSVC()

def j_score(y_true, y_pred):
    jaccard = np.minimum(y_true, y_pred).sum(axis=1)/np.maximum(y_true, y_pred).sum(axis = 1)
    return jaccard.mean()*100

def print_score(y_pred, clf):
    print("Clf: ", clf.__class__.__name__)
    print('Jacard score: {}'.format(j_score(y_test,y_pred)))
    print('----')

for classifier in [sgd, lr, svc]:
    clf = OneVsRestClassifier(classifier)
    clf.fit(X_train,y_train)
    y_pred = clf.predict(X_test)
    print_score(y_pred, classifier)
    

链接到数据子集： https://www.filemail.com/d/whkmmsazgrwzfdp

Answer 1

当你运行时：

df['tags'].unique()

在您的示例数据上，输出如下：

array(['1', '3', '2', '1, 2'], dtype=object)

多标签分配发生在数据框的第 7 行：

df[df['tags']=='1, 2']

结果：

             text   TV  Internet    Mobil   Fastnet tags
7   TIL YOUSEE...   1   2   0   0   1, 2

如果您不希望这种二进制化，您可以简单地删除该行或在您的数据框中分配一个标签。

或者，您可以查看 sklearn LabelBinarizer 以获得更符合您所寻找的标签：

labelbinarizer = LabelBinarizer()

fit = labelbinarizer.fit_transform(y)

labelbinarizer.classes_

# array(['1', '1, 2', '2', '3'], dtype='<U4')