尝试使用不同的文本文件作为“字典”在文本文件中查找字数

Question

我有一个词汇文件，其中包含我需要在其他文本文档中找到的单词。如果有的话，我需要找出每个单词的数量。例如：

词汇表.txt：

thought
await
thorough
away
red

test.txt：

I thought that if i await thorough enough, my thought would take me away.
Away I thought the thought.

最后，我应该看到有 4 个思想实例，1 个等待，2 个离开，1 个彻底，0 个红色。我试过这样：

for vocabLine in vocabOutFile:
    wordCounter = 0
    print >> sys.stderr, "Vocab word:", vocabLine
    for line in testFile:
        print >> sys.stderr, "Line 1 :", line
        if vocabLine.rstrip('\r\n') in line.rstrip('\r\n'):
            print >> sys.stderr, "Vocab word is in line"
            wordCounter = wordCounter + line.count(vocabLine)
            print >> sys.stderr, "Word counter", wordCounter
    testFile.seek(0, 0)

我有一种奇怪的感觉，由于 vocab 文件中的返回字符，它无法识别文件中的单词，因为在调试过程中我确定它正确计算了字符串末尾的任何单词匹配。但是，使用 rstrip() 后，计数仍然不正确。完成这一切后，我必须从词汇表中删除不超过 2 次的单词。

我做错了什么？

谢谢！

Answer 1

为你的词汇制作一本字典是个好主意。

vocab_counter = {vocabLine.strip().lower(): 0 for vocabLine in vocabOutFile}

然后只扫描 testFile 一次（效率更高），增加每个单词的计数

for line in testFile:
    for word in re.findall(r'\w+', line.lower()):
        if word in vocab_counter:
            vocab_counter[word] += 1

Answer 2

使用regex 和collections.Counter

import re
from collections import Counter
from itertools import chain

with open("voc") as v, open("test") as test:
    #create a set of words from vocabulary file
    words = set(line.strip().lower() for line in v) 

    #find words in test file using regex
    words_test = [ re.findall(r'\w+', line) for line in test ]

    #Create counter of words that are found in words set from vocab file
    counter = Counter(word.lower()  for word in chain(*words_test)\
                                          if word.lower() in words)
    for word in words:
        print word, counter[word]

输出

thought 4
away 2
await 1
red 0
thorough 1