【问题标题】:removing multiple \n in python before sentence tokenizing在句子标记之前在python中删除多个\ n
【发布时间】:2014-12-26 23:33:33
【问题描述】:

我是编程新手,我正在从一本书和 Stack Overflow 中自学。我正在尝试在聊天语料库中删除 \n 的多个实例,然后对句子进行标记。如果我不删除 \n,则字符串如下所示:

['answers for 10-19-20sUser139 ... hi 10-19-20sUser101 ;)\n\n\n\n\n\n\n\n\n\nI like it when you do it, 10-19-20sUser83\n\n\n\n\n\n\n\n\n\n\n\niamahotnipwithpics\n\n\n\n10-19-20sUser20 go plan the wedding!']

我尝试了几种不同的方法,例如 chomps、line、rstrip 等,但它们似乎都不起作用。可能是我用错了。整个代码如下所示:

import nltk, re, pprint
from nltk.corpus import nps_chat
chat= nltk.Text(nps_chat.words())
from nltk.corpus import NPSChatCorpusReader
from bs4 import BeautifulSoup
chat=nltk.corpus.nps_chat.raw()
soup= BeautifulSoup(chat)
soup.get_text()
text =soup.get_text()
print(text[:40])
print(len(text))
from nltk.tokenize import sent_tokenize
sent_chat = sent_tokenize(text)
len(sent_chat)
text[:] = [line.rstrip('\n') for line in text]
print(len(sent_chat))
print(sent_chat[:40])

当我使用 line 方法时,我得到了这个错误:

Traceback (most recent call last):
File "C:\Python34\Lib\idlelib\testsubjects\sentencelen.py", line 57, in <module>
text[:] = [line.rstrip('\n') for line in text]
TypeError: 'str' object does not support item assignment

帮助?

【问题讨论】:

    标签: python web-scraping nlp nltk data-cleaning


    【解决方案1】:
    >>> x = 'answers for 10-19-20sUser139 ... hi 10-19-20sUser101 ;)\n\n\n\n\n\n\n\n\n\nI like it when you do it, 10-19-20sUser83\n\n\n\n\n\n\n\n\n\n\n\niamahotnipwithpics\n\n\n\n10-19-20sUser20 go plan the wedding!'
    >>> y = "".join([i if i !="\n" else "\t" for i in x])
    >>> z = [i for i in y.split('\t') if i]
    >>> z
    ['answers for 10-19-20sUser139 ... hi 10-19-20sUser101 ;)', 'I like it when you do it, 10-19-20sUser83', 'iamahotnipwithpics', '10-19-20sUser20 go plan the wedding!']
    

    【讨论】:

    • 感谢您的帮助!如果你使用实际的文本,那会起作用,但如果我想使用像文本这样的定义呢?是否需要添加一些内容,因为它在您使用文本时不起作用,但在您使用实际列表时起作用。
    【解决方案2】:

    其实我偶然发现,如果你先将文本标记为单词,然后再造句,\n 就会消失!谢谢你的帮助!

    【讨论】:

      猜你喜欢
      • 1970-01-01
      • 2023-03-20
      • 1970-01-01
      • 1970-01-01
      • 2017-02-10
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多