Selenium：Xpath 返回空白文本？

Question

所以我有这个网址：

https://www.amazon.com/RevitaLash-Cosmetics-RevitaBrow-Advanced-Conditioner/product-reviews/B009QZCAM6/ref=cm_cr_dp_d_show_all_btm?ie=UTF8&reviewerType=all_reviews

我想抓取每条评论给出的评分，但即使我尝试了一些 xpath 变体，它也没有返回任何内容。

当我使用给定的 xpath 在页面中搜索它时，xpaths 确实找到了带有文本“x out of 5 stars”的 10 个元素，这是我目前所拥有的：

from bs4 import BeautifulSoup
import requests
import csv
import os
import pandas as pd
from selenium import webdriver


chromedriver = "path to chromedriver"
driver = webdriver.Chrome(chromedriver)

url = https://www.amazon.com/RevitaLash-Cosmetics-RevitaBrow-Advanced-Conditioner/product-reviews/B009QZCAM6/ref=cm_cr_dp_d_show_all_btm?ie=UTF8&reviewerType=all_reviews

driver.get(url)

ratings = driver.find_elements_by_xpath('//div[@class="a-section a-spacing-none review-views celwidget"]//div[@class="a-row"]/a[@class="a-link-normal"]/i/span')
#ratings = driver.find_elements_by_xpath('/*//div[@id="cm_cr-review_list"]//i[@data-hook="review-star-rating"]/span[@class="a-icon-alt"]')
#ratings = driver.find_elements_by_xpath('/*//div[@id="cm_cr-review_list"]//i[@data-hook="review-star-rating"]/span')

rating_row = []
for rating in ratings:
    rating_row.append(rating.text)

但是当我调用 rating_row 时，它只会在我调用 rating_row 时返回空白文本列表

> rating_row
['', '', '', '', '', '', '', '', '', '']

我在这里做错了什么，我该如何解决？ 对于包含其他评论的其他 url，此结果也是相同的。

Answer 1

将rating_row.append(rating.text) 行替换为rating_row.append(rating.get_attribute('innerHTML'))

Answer 2

你可以试试下面的xpath：

//i[@data-hook='review-star-rating']/span

在代码中：

rating_row = []
for rating in driver.find_elements(By.XPATH, "//i[@data-hook='review-star-rating']/span"):
    rating_row.append(rating.text)

我建议您也让driver.implicitly_wait(30) 隐式等待。你可以在这个地方写这个：

driver = webdriver.Chrome(chromedriver)
driver.implicitly_wait(30)

阅读更多关于Implicit wait here