无法循环单击“更多”按钮以获取所有完整评论

Question

我在 python 中结合 selenium 创建了一个脚本，用于从谷歌地图的某个页面获取所有评论。该页面中有很多评论，并且只有在该页面向下滚动时才能看到它们。我的脚本可以成功完成所有这些。

但是，我目前面临的唯一问题是某些评论有 More 按钮，该按钮旨在单击以显示完整评论。

其中一个是这样的：

website address

我试过了：

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

link = "https://www.google.com/maps/place/Pizzeria+Di+Matteo/@40.8512552,14.255779,17z/data=!4m7!3m6!1s0x133b0841ef6e38e5:0xece6ea09987e9baf!8m2!3d40.8512512!4d14.2579677!9m1!1b1"

driver = webdriver.Chrome()
driver.get(link)
wait = WebDriverWait(driver,10)

while True:
    try:
        elem = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "[class='section-loading-spinner']")))
        driver.execute_script("arguments[0].scrollIntoView();",elem)
    except Exception:
        break

    for see_more in wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "button[class^='section-expand-review']"))):
        see_more.click()


for item in wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, ".section-review-content"))):
    name = item.find_element_by_css_selector("[class='section-review-title'] > span").text
    try:
        review = item.find_element_by_css_selector("[class='section-review-text']").text
    except AttributeError:
        review = ""
    print(name)

driver.quit()

目前上面的脚本在遇到for see_more in wait.until().click()这一行时会抛出stale element错误。

如何循环点击More 按钮以获取所有完整评论？

Answer 1

如果使用WebdriverWait 和presence_of_all_elements_located，它会等待在给定时间内搜索元素，如果它未附加到 html，则会收到错误消息。

但是检查网页中元素的长度，如果有，然后点击元素。

if len(driver.find_elements_by_css_selector("button[class^='section-expand-review']"))>0:
  driver.find_element_by_css_selector("button[class^='section-expand-review']").click()

这里是代码。

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

link = "https://www.google.com/maps/place/Ecstasy/@23.7399982,90.3732109,17z/data=!3m1!4b1!4m7!3m6!1s0x3755b8caa669d5e3:0x41f47ddcc39a556e!8m2!3d23.7399933!4d90.3753996!9m1!1b1"

driver = webdriver.Chrome()
driver.get(link)
wait = WebDriverWait(driver,10)

while True:
    try:
        elem = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "[class='section-loading-spinner']")))
        driver.execute_script("arguments[0].scrollIntoView();",elem)
    except Exception:
        break

    if len(driver.find_elements_by_css_selector("button[class^='section-expand-review']"))>0:
        driver.find_element_by_css_selector("button[class^='section-expand-review']").click()
        print('pass')



for item in wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, ".section-review-content"))):
    name = item.find_element_by_css_selector("[class='section-review-title'] > span").text
    try:
        review = item.find_element_by_css_selector("[class='section-review-text']").text
    except AttributeError:
        review = ""
    print(name)

driver.quit()

已编辑

 if len(driver.find_elements_by_css_selector("button[class^='section-expand-review']"))>0:

    for item in driver.find_elements_by_css_selector("button[class^='section-expand-review']"):
        item.location_once_scrolled_into_view
        item.click()
        time.sleep(2)

Answer 2

这对我有用：- 您可以将其放在 for 循环或您的方法中以获取所有评论。

try:                
   driver.find_element_by_class_name("mapsConsumerUiSubviewSectionReview__section-expand-review").click()
except:
   continue