Python：检查字符串及其子字符串是否存在于同一个列表中

Question

我在一个标记化的句子中提取了基于 1-gram、2-gram、3-gram 的关键字

list_of_keywords = []
for i in range(0, len(stemmed_words)):
    temp = []
    for j in range(0, len(stemmed_words[i])):
        temp.append([' '.join(x) for x in list(everygrams(stemmed_words[i][j], 1, 3)) if ' '.join(x) in set(New_vocabulary_list)])
    list_of_keywords.append(temp)

我已经获得了关键字列表

['blood', 'pressure', 'high blood', 'blood pressure', 'high blood pressure']
['sleep', 'anxiety', 'lack of sleep']

如何通过删除列表中的所有子字符串来简化结果并保留：

['high blood pressure']
['anxiety', 'lack of sleep']

Answer 1

你可以使用这一个班轮：

b = ['blood', 'pressure', 'high blood', 'blood pressure', 'high blood pressure']
result = [ i for i in b if not any( [ i in a for a in b if a != i]   )]

我承认这是 O(n²) 并且对于大输入可能会降低性能。

这基本上是对以下内容的列表理解：

word_list =  ['blood', 'pressure', 'high blood', 'blood pressure', 'high blood pressure']

result = []
for this_word in word_list:
    words_without_this_word = [ other_word  for other_word in word_list if other_word != this_word]  
    found = False
    for other_word in words_without_this_word:
        if this_word in other_word:
            found = True

    if not found:
        result.append(this_word)

result

Answer 2

如果您的单词列表很大，最好使用suffix tree。

这是 PyPI 上的 package。

创建树后，您可以调用find_all(word) 来获取word 每次出现的索引。您只需要保留只出现一次的字符串：

from suffix_trees import STree
# https://pypi.org/project/suffix-trees/
# pip install suffix-trees

words = ['blood', 'pressure', 'high blood', 'blood pressure', 'high blood pressure'] + ['sleep', 'anxiety', 'lack of sleep']
st = STree.STree(words)

st.find_all('blood')
# [0, 20, 26, 46]

st.find_all('high blood pressure')
# [41]

[word for word in words if len(st.find_all(word)) == 1]
# ['high blood pressure', 'anxiety', 'lack of sleep']

words 需要是唯一的字符串列表，因此您可能需要在生成后缀树之前调用list(set(words))。

据我所知，整个脚本应该在O(n) 中运行，n 是字符串的总长度。

Answer 3

假设您的元素的顺序是从最短字符串到最长字符串，您需要检查每个元素是否是最后一个元素的子字符串，然后将其从列表中删除：

symptoms = ['blood', 'pressure', 'high blood', 'blood pressure', 'high blood pressure']


def removeSubstring(data):
    for symptom in data[:-1]:
        if symptom in data[-1]:
            print("Removing: ", symptom)
            data.remove(symptom)
    print(data)


removeSubstring(symptoms)

Answer 4

words = ['blood', 'pressure', 'high blood', 'blood pressure', 'high blood pressure']

superset_word = ''
#print (words)
for word in words:
    word_list_minus_word = [each for each in words if word != each]
    counter = 0
    for other_word in word_list_minus_word:
        if (other_word not in word):
            break
        else:
            counter += 1
    if (counter == len(word_list_minus_word)):
        superset_word = word
        break
print(superset_word)

Answer 5

grams = ['blood', 'pressure', 'high blood', 'blood pressure', 'high blood pressure']

unique_grams = [grams[i] for i in range(len(grams)) if not grams[i] in ' '.join(grams[i+1:])]