从文本中提取列表中的 unigram 和 bigram答案

【问题标题】：Extracting unigram and bigram in list from text从文本中提取列表中的 unigram 和 bigram
【发布时间】：2023-01-16 22:51:28
【问题描述】：

我有一个固定尺寸的列表：

sizes = ['extra small', 'small', 'medium', 'large', 'extra large']

我想从文本中提取对这些尺寸的任何提及。然而，当我输入这样的文本时，“特小”与“小”、“特大”与“大”之间的关系很复杂：

text1 = 'she wears a small size and he wears an extra large'

在尝试匹配较小的字符串之前，我想出了以下语法来匹配较大的字符串：

import re
sizes = ['extra small', 'small', 'medium', 'large', 'extra large']
text1 = 'she wears a small size and he wears an extra large size'
mentioned_sizes = []

sizes.sort(key=lambda x: len(x.split()), reverse=True)

for x in sizes:
    if len(x.split()) > 1:
        if re.findall(x, text1):
            mentioned_sizes.append(x)
    elif len(x.split()) == 1:
        if (x in text1) and (x not in [item for sublist in [x.split() for x in mentioned_sizes] for item in sublist]):
            mentioned_sizes.append(x)

这给了我 ['extra large', 'small'] for the mentioned_sizes，这就是我想要的。但是，当文本变成这样时，我遇到了一个问题：

text2 = 'she wears a large size and he wears an extra large size'

对于 mentioned_sizes，我现在只得到 ['extra large']，而不是 ['extra large', 'large']。如何提取文本中提到的尺寸？

【问题讨论】：

制作一个结合所有大小的正则表达式模式：(extra small|small|medium|large|extra large) 并使用 findall 与此模式。无需拆分文本或在每个单词上循环。

标签： python

【解决方案1】：

如果您重新排序您的尺寸，以便您的双词尺寸排在第一位，您可以找到这些尺寸，然后将它们从文本中删除，这样在搜索单个词尺寸时就不会找到它们。此外，通过添加到集合中，您不必担心 mentioned_sizes 中的重复大小。

这是一个例子：

代码：

sizes = ['extra small', 'extra large', 'small', 'medium', 'large']

text_list = ['she wears a small size and he wears an extra large size',
             'she wears a large size and he wears an extra large size']

for text in text_list:
    mentioned_sizes = set()
    original_text = text
    for size in sizes:
        if size in text:
            mentioned_sizes.add(size)
            text = text.replace(size, "")
    print(f"Text: {original_text}
Mentioned Sizes: {mentioned_sizes}
")

输出：

Text: she wears a small size and he wears an extra large size
Mentioned Sizes: {'small', 'extra large'}

Text: she wears a large size and he wears an extra large size
Mentioned Sizes: {'large', 'extra large'}

笔记：

如果你想使用正则表达式，你可以做这样的事情来产生相同的输出：

for text in text_list:
    mentioned_sizes = set(re.findall('|'.join(sizes),text))
    print(f"Text: {text}
Mentioned Sizes: {mentioned_sizes}
")

【讨论】：