Beautiful Soup 没有解析所有标签

Question

我附上了我要抓取的网站的 HTML 屏幕截图，有一个表格，在其中我想从正文中的行中获取一些数据（它们显然存在），但是，它不起作用，所以我决定打印 tbody，这表明解析找到了表和 tbody，但没有找到其中的行。我不知道如何解决这个问题，任何帮助将不胜感激。HTML from wesbite

Output when printing body

这是我的代码的开头：

url = "https://superfpl.com/player_stats"

results = requests.get(url)

soup = BeautifulSoup(results.text, "html.parser")

players = []

teams = []

positions = []

ownerships = []

print(soup.find("tbody"))

player_div = soup.find_all('tr', role_="row",  class_="odd")

Answer 1

数据是通过 JavaScript 动态加载的。但是你可以使用requests 模块来模拟它：

import requests

url = 'https://superfpl.com/ajax/player_stats'
data = requests.get(url).json()

# uncomment this to see all data (WARNING, huge list!):
# import json
# print(json.dumps(data, indent=4))

# pretty print some data to screen:
for row in data['data']:
    print('{:<20}{:<20}{:<20}'.format(row['web_name'], row['position'], row['points_per_game']))

打印：

Connolly            FWD                 2.4                 
Cresswell           DEF                 2.7                 
Lennon              MID                 0.9                 
Mooy                MID                 2.9                 
Ramsdale            GKP                 3.4                 
Wan-Bissaka         DEF                 3.3                 
Koiki               DEF                 0.0                 
Doucouré            MID                 3.3                 
Idah                FWD                 1.3                 
Lallana             MID                 1.8                 
Masina              DEF                 2.8                 
Adam Smith          DEF                 2.2                 

... and so on.