您可能应该调整您的 URL 以进行抓取。
使用 curl 测试
当我使用此 URL 运行 curl 请求时，响应的 HTML 不包含预期的 <span class="a-size-medium a-color-base a-text-normal">。
curl 'https://www.amazon.com/s?k=samsung+tablet&crid=3VMSMTMZYOP78&sprefix=samsung+%2Caps%2C273&ref=nb_sb_ss_ts-doa-p_2_8' | grep "<span class="
但只有以下跨度：
<span class="a-button a-button-primary a-span12">
<span class="a-button-inner">
<span class="a-letter-space"></span>
<span class="a-letter-space"></span>
<span class="a-letter-space"></span>
<span class="a-letter-space"></span>
试汤
您也可以将soup 测试为HedgeHog commented：
import requests # Import the library for sending requests to the server
from bs4 import BeautifulSoup # Import the library for webpage parsing
url ='https://www.amazon.com/s?k=samsung+tablet&crid=3VMSMTMZYOP78&sprefix=samsung+%2Caps%2C273&ref=nb_sb_ss_ts-doa-p_2_8'
response = requests.get(url) # GET-request
soup = BeautifulSoup(response.text, 'html') # adjusted from lxml to html
print(soup) # contains span elements but not expected
elements = soup.find_all('span', attrs={'class_':'a-size-medium a-color-base a-text-normal'})
print(elements) # empty list, the expected spans were not found
您会发现一种机器人预防措施，可能使用验证码来验证人类是否正在使用浏览器：
<h4>Enter the characters you see below</h4>
<p class="a-last">Sorry, we just need to make sure you're not a robot. For best results, please make sure your browser is accepting cookies.</p>
趣事：
您可以将生成的 HTML 复制并粘贴或写入文件并在浏览器中打开。它显示了亚马逊的看门狗：
另见All The Dogs You Can Meet If You're Trying To Get On Amazon But It's Broken
解决方法：传递所需的请求标头
进一步的研究建议在请求中添加 2 个标头（您的浏览器也会自动添加）：
- 有效的
User-Agent（可以模拟特定的浏览器和操作系统/平台）
-
Accept-Language（大多数电子商务页面都需要将内容本地化）
在 requests 中，您可以将它们添加为字典，例如：
HEADERS = ({
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.157 Safari/537.36',
'Accept-Language': 'en-US, en;q=0.5'
})
response = requests.get(URL, headers=HEADERS)
见：