计算文本中特定因素的程序，例如“气候金融”

Question

from collections import Counter
input = 'file.txt'

CounterWords = {}
words = {}
with open(input,'r', encoding='utf-8-sig') as fh:
  for line in fh:
    word_list = line.replace(',','').replace('\'','').replace('.','').lower().split()
    for word in word_list:
      if len(word) < 6
          continue
      elif word not in CounterWords:
          CounterWords[word] = 1
      else:
          CounterWords[word] = CounterWords[word] + 1
N = 50

top_words = Counter(CounterWords).most_common(N)
for word, frequency in top_words:
    print("%s %d" % (word, frequency))

目前，我可以选择两个最常用的单词，其中字符串超过 X 个字符。

程序应筛选文本并计算以下单词：

“气候金融” “市场营销失败” “巴黎 2015”

仍应包含每个字符串的最少字符数，以防止出现“I and”等结果。

Answer 1

您可以简单地使用your_file_content.count(your_string)：

from collections import Counter
input = 'D:\\file.txt'

import itertools
def pairwise(iterable):
    # "s -> (s0,s1), (s1,s2), (s2, s3), ..."
    a, b = itertools.tee(iterable)
    next(b, None)
    return zip(a, b)  

CounterWords = {}
CounterPairs = {}
words = {}
file_content = ''
with open(input,'r', encoding='utf-8-sig', errors='ignore') as fh:
  file_content = fh.read().replace('\n', ' ')
  word_list = file_content.replace(',','').replace('\'','').replace('.','').lower().split()
  word_list = list(dict.fromkeys(word_list)) # to remove duplicates
  word_pairs_list = pairwise(word_list)
  for word in word_list:
    if len(word) < 6:
      continue
    else:
      CounterWords[word] = file_content.count(word)
  for pair in word_pairs_list:
    CounterPairs[pair] = file_content.count(' '.join(pair))
N = 50

# for all single words :
top_words = Counter(CounterWords).most_common(N)
for word, frequency in top_words:
  print("%s %d" % (word, frequency))

# for all pairs :
top_pairs = Counter(CounterPairs).most_common(N)
for pair, frequency in top_pairs:
  print("%s %d" % (pair, frequency))

# for specific pairs :
print("\n%s %d" % ('climate finance', CounterPairs[('climate', 'finance')]))

pairwise 函数取自：Iterate a list as pair (current, next) in Python