将多个句子标记为python pandas中的行答案

【问题标题】：Tokenize multiple sentences to rows in python pandas将多个句子标记为python pandas中的行
【发布时间】：2019-11-26 15:18:47
【问题描述】：

我有一个这样的文本数据框，

id      text
1       Thanks.  I appreciate your help.  I really like this chat service as it is very convenient.  I hope you have a wonderful day! thanks!
2       Got it. Thanks for the help; good nite.

我想拆分这些文本句子并将它们与每个 id 匹配。我的预期输出是，

id      text
1       Thanks.
1       I appreciate your help.
1       I really like this chat service as it is very convenient.
1       I hope you have a wonderful day!
1       thanks!
2       Got it.
2       Thanks for the help;
2       good nite.

有没有可以处理这个问题的 nltk 函数？

【问题讨论】：

标签： python pandas dataframe nlp nltk

【解决方案1】：

第一个 split 然后使用 explode ，如果您没有将 pandas 升级到 0.25 ，请检查 How to unnest (explode) a column in a pandas DataFrame?

df.assign(text=df.text.str.split('[.!;]')).explode('text').loc[lambda x : x.text!='']
Out[181]: 
                                                text  id
0                                             Thanks   1
0                             I appreciate your help   1
0    I really like this chat service as it is ver...   1
0                    I hope you have a wonderful day   1
0                                             thanks   1
1                                             Got it   2
1                                Thanks for the help   2
1                                          good nite   2

【讨论】：