如何通过其 CSS 选择器字符串选择此元素？

Question

从这个url，我正在尝试提取链接https://www.collinsdictionary.com/dictionary/french-english/conjugation/aimer in

<a class="link-right verbtable" href="https://www.collinsdictionary.com/dictionary/french-english/conjugation/aimer">Full verb table</a>

其 CSS 选择器为 div.content.definitions.dictionary.biling > div.hom > span > span.xr > a。我按照 Automate the Boring Stuff with Python 一书中的说明操作

from bs4 import BeautifulSoup

url = 'https://www.collinsdictionary.com/dictionary/french-english/aimer'
soup = BeautifulSoup(url, 'html.parser')

soup.select('div.content.definitions.dictionary.biling > div.hom > span > span.xr > a')

能否请您详细说明[]的结果如何？

Answer 1

这是因为柯林斯词典使用 Cloudfare 来提高其网站和服务的安全性和性能。因此，当您向其服务器请求时。它不会给你 HTML 文件。

<title>Access denied | www.collinsdictionary.com used Cloudflare to restrict access</title>

为了通过它的安全性。您必须在请求中设置用户代理。

from bs4 import BeautifulSoup
import requests

user_agent = {'User-agent': "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36"}

url = 'https://www.collinsdictionary.com/dictionary/french-english/aimer'
doc = requests.get(url, headers = user_agent).text
soup = BeautifulSoup(doc, 'html.parser')
result = soup.select('div.content.definitions.dictionary.biling > div.hom > span > span.xr > a')
print(result)

这将为您提供结果：

[<a class="link-right verbtable" href="https://www.collinsdictionary.com/dictionary/french-english/conjugation/aimer">Full verb table</a>]