在python nlp中匹配一组单词和一组句子答案

【问题标题】：Matching set of words with set of sentences in python nlp在python nlp中匹配一组单词和一组句子
【发布时间】：2020-02-11 19:57:06
【问题描述】：

我有一个用例，我想将一个单词列表与一个句子列表进行匹配，并带来最相关的句子

我在 python 中工作。我已经尝试过使用 KMeans，我们将我们的一组文档聚集到集群中，然后预测它所在的结构中的句子。但就我而言，我已经有可用的单词列表。

def getMostRelevantSentences():
    Sentences = ["This is the most beautiful place in the world.",
            "This man has more skills to show in cricket than any other game.",
            "Hi there! how was your ladakh trip last month?",
            "Isn’t cricket supposed to be a team sport? I feel people should decide first whether cricket is a team game or an individual sport."]

    words = ["cricket","sports","team","play","match"]

    #TODO: now this should return me the 2nd and last item from the Sentences list as the words list mostly matches with them

所以我想从上面的代码中返回与提供的单词密切匹配的句子。我不想在这里使用监督机器学习。任何帮助将不胜感激。

【问题讨论】：

这个问题似乎根本没有包括任何解决问题的尝试。请编辑问题以显示您尝试过的内容，并使用Minimal, Complete, and Verifiable example 显示您遇到的特定障碍。欲了解更多信息，请参阅How to Ask。

标签： python nlp data-science unsupervised-learning

【解决方案1】：

所以最后我使用了这个名为 gensim 的超级库来生成相似度。

import gensim
from nltk.tokenize import word_tokenize

def getSimilarityScore(raw_documents, words):
    gen_docs = [[w.lower() for w in word_tokenize(text)] 
            for text in raw_documents]
    dictionary = gensim.corpora.Dictionary(gen_docs)
    corpus = [dictionary.doc2bow(gen_doc) for gen_doc in gen_docs]
    tf_idf = gensim.models.TfidfModel(corpus)
    sims = gensim.similarities.Similarity('/usr/workdir',tf_idf[corpus],
                                      num_features=len(dictionary))

    query_doc_bow = dictionary.doc2bow(words)
    query_doc_tf_idf = tf_idf[query_doc_bow]

    return sims[query_doc_tf_idf]

您可以将此方法用作：


Sentences = ["This is the most beautiful place in the world.",
            "This man has more skills to show in cricket than any other game.",
            "Hi there! how was your ladakh trip last month?",
            "Isn’t cricket supposed to be a team sport? I feel people should decide first whether cricket is a team game or an individual sport."]

words = ["cricket","sports","team","play","match"]

words_lower = [w.lower() for w in words]

getSimilarityScore(Sentences,words_lower)

【讨论】：