检索同义词和相似性[重复]答案

【问题标题】：Retrieve synonyms and similarity [duplicate]检索同义词和相似性[重复]
【发布时间】：2020-04-29 07:33:33
【问题描述】：

我想从www.thesaurus.com 中抓取几页。我对一个词的同义词和反义词都感兴趣。例如，如果我要查找的单词是angry，我对以下图片中的单词感兴趣（页面中还有很多，但我只对前两个块感兴趣）：

和：我可以用这段代码找到这些词（和其他相关的）：

import requests
from bs4 import BeautifulSoup

word = "angry"
url = 'https://www.thesaurus.com/browse/{}'.format(word)
r = requests.get(url)
returned_words_list = []

soup = BeautifulSoup(r.text, 'html.parser')
word_ul = soup.find("ul", {"class":'css-1lc0dpe et6tpn80'})
for idx, elem in enumerate(word_ul.findAll("a")):
    returned_words_list.append(elem.text.strip())

print (returned_words_list)

但我也对相似度（单词的颜色）感兴趣。

看源码，有一个类似JSON的文件：

<script>window.INITIAL_STATE = {"routerReducer":{"location":null},"searchData":{"isFetchingTunaApi":false,"isFetchingSpellSuggestion":false,"isFetchingRelatedWordsApi":false,"searchTerm":"angry","tunaApiData":{"entry":"angry","type":"normal","slugLuna":"angry","slug":"angry","pronunciation":{"audio":{"audio\u002Fogg":"https:\u002F\u002Fstatic.sfdict.com\u002Faudio\u002Flunawav\u002FA04\u002FA0484200.ogg","audio\u002Fmpeg":"https:\u002F\u002Fstatic.sfdict.com\u002Faudio\u002FA04\u002FA0484200.mp3"},"spell":"\u003Cspan class=\"bold\"\u003Eang\u003C\u002Fspan\u003E-gree","ipa":"ˈæŋ gri"},"posTabs":[{"isInformal":null,"isVulgar":"0","definition":"being mad, often extremely mad","thesRid":"842","pos":"adj.","synonyms":[{"similarity":"100","isInformal":"0","isVulgar":null,"term":"annoyed","targetTerm":"annoyed","targetSlug":"annoyed"},{"similarity":"100","isInformal":"0","isVulgar":null,"term":"bitter","targetTerm":"bitter","targetSlug":"bitter"},{"similarity":"100","isInformal":"0","isVulgar":null,"term":"enraged","targetTerm":"enraged","targetSlug":"enraged"},.....

但是我不知道怎么读。最后我想有这样的输出：

"synonyms":[{"similarity":"100","isInformal":"0","isVulgar":null,"term":"annoyed","targetTerm":"annoyed","targetSlug":"annoyed"},{"similarity":"100","isInformal":"0","isVulgar":null,"term":"bitter","targetTerm":"bitter","targetSlug":"bitter"},{"similarity":"100","isInformal":"0","isVulgar":null,"term":"enraged","targetTerm":"enraged","targetSlug":"enraged"},
"antonyms":[{"similarity":"-100","isInformal":"0","isVulgar":null,"term":"calm","targetTerm":"calm","targetSlug":"calm"},{"similarity":"-100","isInformal":"0","isVulgar":null,"term":"cheerful","targetTerm":"cheerful","targetSlug":"cheerful"},

我可以像这样读取term 和similarity（或只是元组的输出列表）：

[("annoyed", 100), ("bitter", 100)...]
[("calm", -100), ("cheerful", -100)...]

【问题讨论】：

前两个区块是什么意思？
仅显示图片中的那些
别再问为什么你的 cmets 被删除了。你有一个问题。尽力描述问题。有很多很好的解释性问题无缘无故被否决。你无法控制人民。最后，你确实有一个问题。所以你有一个答案。其他的不要关注

标签： python json web-scraping beautifulsoup scrape

【解决方案1】：

信用转到Andrej Kesely

import re
import json
import requests

url = 'https://www.thesaurus.com/browse/angry?s=t'

txt = re.findall(r'INITIAL_STATE\s*=\s*({.*})', requests.get(url).text)[0]

data = json.loads(txt)

# print(json.dumps(data, indent=4))  # <-- uncomment to see all data

print(data.keys())

【讨论】：