Python Web Scraping 有问题

Question

我正在使用requests-HTML和beautiful来抓取一个网站，下面是代码。奇怪的是，有时我可以在使用 print(soup.get_text()) 时从网上获取文本，而在使用 print(soup) 时，我会得到一些随机代码 - 在所附图像中。

session = HTMLSession()
r = session.get(url)
soup = bs(r.content, "html.parser")
print(soup.get_text())
#print(soup)

The program return this when I tried to look at the soup

Answer 1

我认为该网站受 javascript 保护..试试这个..它可能会有所帮助

import requests
from bs4 import BeautifulSoup

r = requests.get(url)
print(r.text)

#if you want the whole content you can just do slicing stuff on the response stored in r or rather just do it with bs4

soup = BeautifulSoup(r.text, "html.parser")
print(soup.text)