刮掉没有类名的 Span 标签，并且不会出现在所有元素中

Question

我正在使用 Python 中的 Selenium 抓取评论页面。我想提取每条评论的评分（即从评论中的 7/10 中提取 7）。 HTML 元素的结构如下：

    <div class ="review">
         <div class="rating-bar">
            <span class="user-rating">
               <svg class="ipl-icon ipl-star-icon 
                "xmlns="http://www.w3.org/2000/svg" fill="#000000" height="24" 
                 viewBox="0 0 24 24" width="24"> <path d="M0 0h24v24H0z" 
                 fill="none"></path> <path d="M12 17.27L18.18 21l-1.64-7.03L22 
                 9.24l-7.19-.61L12 2 9.19 8.63 2 9.24l5.46 4.73L5.82 21z"> 
                </path> <path d="M0 0h24v24H0z" fill="none"></path> </svg>
               <span>7</span>             # What I want to extract
               <span class='scale'>/10</span>
             </span>
            </div>

该元素没有任何类名，所以我假设使用span标签下的类user-rating提取它：

    rating = driver.find_elements_by_class_name('user-rating')

但是我应该如何在另一个 span 标签中提取 span 标签？我不能将它引用到任何类名。

另外，不是每条评论都包含评分，所以当它刮到没有评分的评论时，它会提示我错误：

    NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".rating-other-user-rating"} (Session info: chrome=87.0.4280.66)

这是我迄今为止尝试过的：

    review = driver.find_elements_by_class_name("review")
    rating_ls = []
    
    for i in review:
        rating = i.find_element_by_class_name('rating-other-user-rating').text
        # If rating exists, append it to the list, otherwise append "N/A" 
        rating_ls.append(rating[0] if rating else "N/A")   

如果有人可以帮助我解决这个问题，我将不胜感激。提前非常感谢！

Answer 1

尝试等待元素（可能是JS代码添加的）：

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

reviews = WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.CLASS_NAME, "review-container")))

for review in reviews:
    _rating = review.find_elements_by_class_name('rating-other-user-rating')
    rating = _rating[0].text if _rating else 'N/A' 
    _comment = review.find_elements_by_class_name('content')
    comment = _comment[0].text if _comment else 'N/A' 
    print(rating + ": " + comment)

Answer 2

要使用Selenium 和python 提取每条评论的评分（即从评论中的7/10 中提取7），您必须为visibility_of_all_elements_located() 诱导WebDriverWait，您可以使用以下任一方法Locator Strategies:

• 使用XPATH, span index 和 text 属性：

  print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@class='review']//span[@class='user-rating']//following::span[1]")))])
  

• 使用XPATH、属性和get_attribute()：

  print([my_elem.get_attribute("innerHTML") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@class='review']/span[@class='user-rating']//span[not(contains(@class,'scale'))]")))])
  

• 注意：您必须添加以下导入：

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  

---

结尾

链接到有用的文档：

• get_attribute() 方法Gets the given attribute or property of the element.
• text 属性返回 The text of the element.
• Difference between text and innerHTML using Selenium