【问题标题】:find unique sentences in two files在两个文件中找到唯一的句子
【发布时间】:2016-12-01 04:49:08
【问题描述】:

我有两个文件,我正在尝试在两个文件之间打印唯一的句子。为此,我在 python 中使用 difflib。

text ='Physics is one of the oldest academic disciplines. Perhaps the oldest through its inclusion of astronomy. Over the last two millennia. Physics was a part of natural philosophy along with chemistry.'
text1 ='Physics is one of the oldest academic disciplines. Physics was a part of natural philosophy along with chemistry. Quantum chemistry is a branch of chemistry.'
import difflib

differ = difflib.Differ()
diff = differ.compare(text,text1)
print '\n'.join(diff)

它没有给我想要的输出。它给了我这样的。

  P
  h
  y
  s
  i
  c
  s

  i
  s

  o
  n
  e

  o
  f

  t
  h
  e

我想要的输出只是两个文件之间的唯一句子。

text = 也许是最古老的,因为它包含了天文学。超过 过去两千年。

text1 = 量子化学是化学的一个分支。

另外,似乎 difflib.Differ 是逐行而不是逐句进行的。请有任何建议。我怎么能这样做?

【问题讨论】:

    标签: python python-2.7 python-3.x pattern-matching difflib


    【解决方案1】:

    首先,事实上,Differ().compare() 比较的是行,而不是句子。

    其次,它实际上比较序列,例如字符串列表。但是,您传递的是两个字符串,而不是两个字符串列表。由于字符串也是一个(字符)序列,因此 Differ().compare() 在您的情况下会比较各个字符。

    如果要按句子比较文件,则必须准备两个句子列表。您可以使用 nltk.sent_tokenize(text) 将字符串拆分为句子。

    diff = differ.compare(nltk.sent_tokenize(text),nltk.sent_tokenize(text1))
    print('\n'.join(diff))
    #  Physics is one of the oldest academic disciplines.
    #- Perhaps the oldest through its inclusion of astronomy.
    #- Over the last two millennia.
    #  Physics was a part of natural philosophy along with chemistry.
    #+ Quantum chemistry is a branch of chemistry.
    

    【讨论】:

    • 谢谢 DYZ。谢谢你指出我的错误。我还有一个问题要问你。假设我们有一个字符串“I am boy”,另一个是“I am, boy”。 am 后面有一个 (,)。 diff.compare 表示它们都是独一无二的,因为 (,) 不相似。我们如何在这里考虑这种情况。我与nltk有关。但是我可以在这里处理这个案子吗?
    • 我不熟悉 difflib 包(但我很高兴了解它!)但您也可以在通过 diff 运行文本之前手动删除任何标点符号。在按句点拆分之前检查您的字符串的 strip()。
    • 我建议您使用来自 nltk 的单词标记器来提取单词:" ".join(w for w in nltk.word_tokenize('I am, boy') if w.isalpha())。或者你可以使用正则表达式来提取单词。
    【解决方案2】:

    正如上面 DZinoviev 所说,您将字符串传递给需要列表的函数。您不需要使用 NLTK,而是可以通过拆分句点将字符串转换为句子列表。

    import difflib
    
    text1 ="""Physics is one of the oldest academic disciplines. Perhaps the oldest through its inclusion of astronomy. Over the last two millennia. Physics was a part of natural philosophy along with chemistry."""
    text2 ="""Physics is one of the oldest academic disciplines. Physics was a part of natural philosophy along with chemistry. Quantum chemistry is a branch of chemistry."""
    
    list1 = list(text1.split("."))
    list2 = list(text2.split("."))
    
    differ = difflib.Differ()
    diff = differ.compare(list1,list2)
    print "\n".join(diff)
    

    【讨论】:

    • 谢谢 SummerEla
    • 句子之间可能有其他标点符号,如!、?、...等
    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2023-03-30
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2021-12-13
    相关资源
    最近更新 更多