ValueError：np.nan 是无效文档

Question

下面是我写的代码：

X2=df['title']
y2=df['news_type']
X2_train, X2_test, y2_train, y2_test = train_test_split(X2, y2, test_size=0.3, random_state=42)
pp=Pipeline([
    ('bow',CountVectorizer(analyzer=final)),
    ('tfidf',TfidfTransformer()),
    ('classifier',RandomForestClassifier())
    ])
pp.fit(X2_train.astype("U"),y2_train.astype("U"))
predictions7=pp.predict(X2_test)

错误：

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-300-2bed28a1314e> in <module>
----> 1 predictions7=pp.predict(X2_test)

/home/monika/snap/jupyter/common/lib/python3.7/site-packages/sklearn/utils/metaestimators.py in <lambda>(*args, **kwargs)
    117 
    118         # lambda, but not partial, allows help() to work with update_wrapper
--> 119         out = lambda *args, **kwargs: self.fn(obj, *args, **kwargs)
    120         # update the docstring of the returned function
    121         update_wrapper(out, self.fn)

/home/monika/snap/jupyter/common/lib/python3.7/site-packages/sklearn/pipeline.py in predict(self, X, **predict_params)
    405         Xt = X
    406         for _, name, transform in self._iter(with_final=False):
--> 407             Xt = transform.transform(Xt)
    408         return self.steps[-1][-1].predict(Xt, **predict_params)
    409 

/home/monika/snap/jupyter/common/lib/python3.7/site-packages/sklearn/feature_extraction/text.py in transform(self, raw_documents)
   1248 
   1249         # use the same matrix-building strategy as fit_transform
-> 1250         _, X = self._count_vocab(raw_documents, fixed_vocab=True)
   1251         if self.binary:
   1252             X.data.fill(1)

/home/monika/snap/jupyter/common/lib/python3.7/site-packages/sklearn/feature_extraction/text.py in _count_vocab(self, raw_documents, fixed_vocab)
   1108         for doc in raw_documents:
   1109             feature_counter = {}
-> 1110             for feature in analyze(doc):
   1111                 try:
   1112                     feature_idx = vocabulary[feature]

/home/monika/snap/jupyter/common/lib/python3.7/site-packages/sklearn/feature_extraction/text.py in _analyze(doc, analyzer, tokenizer, ngrams, preprocessor, decoder, stop_words)
     97 
     98     if decoder is not None:
---> 99         doc = decoder(doc)
    100     if analyzer is not None:
    101         doc = analyzer(doc)

/home/monika/snap/jupyter/common/lib/python3.7/site-packages/sklearn/feature_extraction/text.py in decode(self, doc)
    217 
    218         if doc is np.nan:
--> 219             raise ValueError("np.nan is an invalid document, expected byte or "
    220                              "unicode string.")
    221 

ValueError: np.nan is an invalid document, expected byte or unicode string.

尽一切努力解决此错误，但无法解决。请告诉我在这里做错了什么？ 仅在此行之后抛出错误：predictions7=pp.predict(X2_test)。我已经粘贴了上面的错误。

解决方案：

.Replace "pp.fit(X2_train.astype("U"),y2_train.astype("U"))" by
"pp.fit((X2_train.astype("U").str.lower()),(y2_train.astype("U").str.lower()))"

Replace "predictions7=pp.predict(X2_test)" by "predictions7=pp.predict(X2_test.astype("U"))"

Answer 1

Replace "pp.fit(X2_train.astype("U"),y2_train.astype("U"))" by
"pp.fit((X2_train.astype("U").str.lower()),(y2_train.astype("U").str.lower()))"

Replace "predictions7=pp.predict(X2_test)" by "predictions7=pp.predict(X2_test.astype("U"))"