循环遍历列表时出现 StaleElementReferenceException

Question

我正在尝试为this 网站制作网络爬虫。这个想法是代码通过选择机构的名称（首先是 3B-Wonen）来遍历所有机构，关闭弹出屏幕，单击下载按钮，然后对列表中的所有项目再次执行所有操作。

但是，在第一个循环之后，它在选择循环中选择第二个机构时抛出StaleElementReferenceException。从我读到的内容来看，这意味着第一个循环中定义的元素不再可访问。我已经阅读了多篇文章，但我不知道如何克服这种特殊情况。

谁能指出我正确的方向？顺便说一句，我正在使用 Python 的 selenium，而且我是编程的初学者，所以我还在学习。如果您能指出我的大致方向，那将对我有很大帮助！我的代码如下：

#importing and setting up parameters for geckodriver/firefox
...

# webpage
driver.get("https://opendata-dashboard.cijfersoverwonen.nl/dashboard/opendata-dashboard/beleidswaarde")

WebDriverWait(driver, 30)

# Get rid of cookie notification
# driver.find_element_by_class_name("cc-compliance").click()

# Store position of download button
element_to_select = driver.find_element_by_id("utilsmenu")
action = ActionChains(driver)
WebDriverWait(driver, 30)

# Drop down menu
driver.find_element_by_id("baseGeo").click()

# Add institutions to array
corporaties=[]
corporaties = driver.find_elements_by_xpath("//button[@role='option']")

# Iteration
for i in corporaties:
    i.click()                                                     # select institution
    driver.find_element_by_class_name("close-button").click()     # close pop-up screen
    action.move_to_element(element_to_select).perform()           # select download button
    driver.find_element_by_id("utilsmenu").click()                # click download button
    driver.find_element_by_id("utils-export-spreadsheet").click() # pick export to excel
    driver.find_element_by_id("baseGeo").click()                  # select drop down menu for next iteration

Answer 1

这段代码对我有用。但我不做driver.find_element_by_id("utils-export-spreadsheet").click()

from selenium import webdriver
import time
from selenium.webdriver.common.action_chains import ActionChains

driver = webdriver.Chrome(executable_path="path")
driver.maximize_window()
driver.implicitly_wait(10)

driver.get("https://opendata-dashboard.cijfersoverwonen.nl/dashboard/opendata-dashboard/beleidswaarde")

act = ActionChains(driver)
driver.find_element_by_xpath("//a[text()='Sluiten en niet meer tonen']").click() # Close pop-up
# Get the count of options
driver.find_element_by_id("baseGeoContent").click()
cor_len = len(driver.find_elements_by_xpath("//button[contains(@class,'sel-listitem')]"))
print(cor_len)
driver.find_element_by_class_name("close-button").click()

# No need to start from 0, since 1st option is already selected. Start from downloading and then move to next items.
for i in range(1,cor_len-288): # Tried only for 5 items
    act.move_to_element(driver.find_element_by_id("utilsmenu")).click().perform()
    #Code to click on downloading option
    print("Downloaded:{}".format(driver.find_element_by_id("baseGeoContent").get_attribute("innerText")))
    driver.find_element_by_id("baseGeoContent").click()
    time.sleep(3) # Takes time to load.
    coritems = driver.find_elements_by_xpath("//button[contains(@class,'sel-listitem')]")
    coritems[i].click()
    driver.find_element_by_class_name("close-button").click()
driver.quit()

输出：

295
Downloaded:3B-Wonen
Downloaded:Acantus
Downloaded:Accolade
Downloaded:Actium
Downloaded:Almelose Woningstichting Beter Wonen
Downloaded:Alwel

Answer 2

问题说明：

看到这里的问题是，你定义了一个列表corporaties = driver.find_elements_by_xpath("//button[@role='option']")，然后是iterating这个list和clicking on first element，这可能会导致一些重定向到一个新页面，或in a new tab等

所以当Selenium 尝试与来自同一个list 的second webelement 交互时，它必须回到original page，而在它回来的那一刻，所有elements 变成stale in nature。

解决办法：

在这种情况下，基本解决方案之一是再次定义列表，这样该元素就不会过时。请看下图：-

代码：

corporaties=[]
corporaties = driver.find_elements_by_xpath("//button[@role='option']")
# Iteration
j = 0
for i in range(len(corporaties)):
    elements = driver.find_elements_by_xpath("//button[@role='option']")
    elements[j].click()
    j = j + 1 # select institution
    driver.find_element_by_class_name("close-button").click()     # close pop-up screen
    action.move_to_element(element_to_select).perform()           # select download button
    driver.find_element_by_id("utilsmenu").click()                # click download button
    driver.find_element_by_id("utils-export-spreadsheet").click() # pick export to excel
    driver.find_element_by_id("baseGeo").click()  # select drop down menu for next iteration
    time.sleep(2)