如何修复错误 ValueError: could not convert string to float in a NLP project in python？

Question

我正在使用 jupyter notebook 编写 Python 代码，用于训练和测试数据集以返回正确的情绪。

当我尝试预测短语的情绪时系统崩溃并显示以下错误的问题：

ValueError: 无法将字符串转换为浮点数：'这本书是如此 激怒它让我不开心'

注意我有一个不平衡的数据集，所以我使用 SMOTE 来对数据集进行过度采样

代码：

import pandas as pd
import numpy as np
from imblearn.over_sampling import SMOTE# for inbalance dataset
from sklearn.linear_model import LogisticRegression
from sklearn.feature_extraction.text import TfidfTransformer,TfidfVectorizer
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score,classification_report,confusion_matrix
from sklearn.pipeline import Pipeline

df = pd.read_csv("data/Apple-Twitter-Sentiment-DFE.csv",encoding="ISO-8859-1")

df
# data is cleaned using preprocessing functions

# Solving inbalanced dataset using SMOTE 

vectorizer = TfidfVectorizer()
vect_df =vectorizer.fit_transform(df["clean_text"])
oversample = SMOTE(random_state = 42)
x_smote,y_smote = oversample.fit_resample(vect_df, df["sentiment"])
print("shape x before SMOTE: {}".format(vect_df.shape))
print("shape x after SMOTE: {}".format(x_smote.shape))
print("balance of targets feild %")
y_smote.value_counts(normalize = True)*100


# split the dataset into train and test 
x_train,x_test,y_train,y_test = train_test_split(x_smote,y_smote,test_size = 0.2,random_state =42)


logreg = Pipeline([
                ('tfidf', TfidfTransformer()),
                ('clf', LogisticRegression(n_jobs=1, C=1e5)),
               ])
logreg.fit(x_train, y_train)

y_pred = logreg.predict(x_test)

print('accuracy %s' % accuracy_score(y_pred, y_test))
print(classification_report(y_test, y_pred))

# Make prediction 
exl = "this book was so interstening it made me not happy"

logreg.predict(exl)

Answer 1

您应该将变量exl 定义如下：

exl = vectorizer.transform(["this book was so interstening it made me not happy"])

然后进行预测。

首先，将测试数据放在一个列表中，然后使用vectorizer 使用从您的训练数据中提取的特征进行预测。