用所有可能的短语近似值在字符串中搜索单词/短语

Question

假设我有以下字符串：

string = 'machine learning ml is a type of artificial intelligence ai that allows software applications to become more accurate at predicting outcomes without being explicitly programmed to do so machine12 learning algorithms use historical data as input to predict new output values machines learning is good'

进一步假设我有一个标签定义为：

tag = 'machine learning'

现在我希望在我的字符串中找到标签。从我的string 可以看出，我有三个位置machine learning，一个位于string 的开头，一个位于machine12 learning，最后一个位于machines learning。我希望找到所有这些并将输出列表设为

['machine learning', 'machine12 learning', 'machines learning']

为了能够做到这一点，我尝试使用 nltk 标记我的标签。那是

tag_token = nltk.word_tokenize(tag)

然后我将拥有['machine','learning']。然后我会搜索tag[0]。

我知道string.find(tag_token[0]) 和data.rfind(tag_token[0]) 将给出machine 的位置，用于第一个和最后一个查找，但如果我在文本中有更多machine learning 怎么办（这里我们有3 个）？

在那种情况下，我将无法将它们全部提取出来。所以我最初的想法是找到所有出现的machine 然后learning 会失败。我希望使用fuzzywuzzy 来分析['machine learning', 'machine12 learning', 'machines learning'] 的标签。

所以我的问题是 string 我有，我如何搜索标签及其近似值并将它们列出如下？

['machine learning', 'machine12 learning', 'machines learning']

---

更新：我现在知道我可以做到以下几点：

pattern = re.compile(r"(machine[\s0-9]+learning)",re.IGNORECASE)
matches = pattern.findall(data)
#[output]: ['machine learning', 'machine12 learning']

如果我这样做了

pattern = re.compile(r"(machine[\sA-Za-z]+learning)",re.IGNORECASE)
matches = pattern.findall(data)
#[output]: ['machine learning', 'machines learning']

但可以肯定的是，这并不是一个可推广的解决方案。所以我想知道在这种情况下是否有一种智能的搜索方式？

Answer 1

也许使用这样的模式 (string\w*)？

import re

string = 'machine 12 learning ml is a type of artificial intelligence ai that allows software applications to become more accurate at predicting outcomes without being explicitly programmed to do so machine12 learning algorithms use historical data as input to predict new output values machines learning is good'

tag_token=['machine','learning']

pattern='('+''.join(e+'\w*\s+(?:\S*\s+)?' for e in tag_token)[:-14]+')'

rgx=re.compile(pattern,re.IGNORECASE)
rgx.findall(string)
#output
#['machine 12 learning', 'machine12 learning', 'machines learning']

随着单词在标签中位置的变化，将更难找到匹配项

此代码将从 tag_token 中找到所有组合。例如。 machine s learning 和 machine learning 和 machine12 12 learning 和 learning machine ...您还可以创建包含 2 个以上单词的新字符串和新 tag_token。将找到这些单词的所有组合。

示例 tag_token = ['1', '2', '3'] 将匹配 1 2 3 和 1a 2 b 3 和 2b2 1sss 3 和 333 2tt 1

import re
import itertools

string = 'machine 12 learning ml is a type of artificial intelligence ai that allows software applications to become more accurate at predicting outcomes without being explicitly programmed to do so machine12 learning algorithms use historical data as input to predict new output values machines learning is good. Learning machine can be used to train people. learning the machines is a great job'

tag_token=['machine','learning']

pattern='('
for current_tag in itertools.permutations(tag_token, len(tag_token)):
    pattern+=''.join(e+'\w*\s+(?:\S*\s+)?' for e in current_tag)[:-14]+'|'

pattern=pattern.rstrip('|')+')'
rgx=re.compile(pattern,re.IGNORECASE)

rgx.findall(string)

#output
#['machine 12 learning', 'machine12 learning', 'machines learning', 'Learning machine', 'learning the machines']