使用 selenium (python) 循环遍历 webelements 列表以检索属性会产生 StaleElementReferenceException

Question

我使用 selenium webdriver 在网站上进行搜索，该网站为我提供了多个页面，每个页面上有多个链接。我的目标是创建搜索产生的所有链接的列表。

它适用于第一页，但是当我在第二页时，我尝试提取“href”属性的 for 循环会产生 StaleElementReferenceException。

我相信应该有某种方法可以使用 WebDriverWait 解决此问题，但我不太清楚具体方法。感谢您的任何建议。

links =[]

while True:

  result = driver.find_element_by_id('results')
# list of all relevant elements on the page
  docs = result.find_elements_by_xpath('//li[@class="row1"]')

  for d in docs:
# this line produces StaleElementReferenceException
    link = d.find_element_by_tag_name('a')
    links.append(link.get_attribute('href'))

# try block checks if "next page" button exists and clicks it (this works fine)
  try:
    next_page = WebDriverWait(driver, 30).until(EC.presence_of_element_located((By.XPATH, '//a[contains(@class,"la-TriangleRight")]')))
    driver.execute_script('arguments[0].click();', next_page)

# if "next page" button doesn't exist we break out of the loop
  except:
    break

编辑： 错误信息：

StaleElementReferenceException            Traceback (most recent call last)
<ipython-input-6-ad6166a696e6> in <module>
     23 #         link = WebDriverWait(d, 30).until(EC.presence_of_element_located((By.XPATH, '//a')))
     24         link = d.find_element_by_tag_name('a')
---> 25         links.append(link.get_attribute('href'))
     26 
     27 #     for i in range(25):

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py in get_attribute(self, name)
    139             attributeValue = self.parent.execute_script(
    140                 "return (%s).apply(null, arguments);" % getAttribute_js,
--> 141                 self, name)
    142         else:
    143             resp = self._execute(Command.GET_ELEMENT_ATTRIBUTE, {'name': name})

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py in execute_script(self, script, *args)
    634         return self.execute(command, {
    635             'script': script,
--> 636             'args': converted_args})['value']
    637 
    638     def execute_async_script(self, script, *args):

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py in execute(self, driver_command, params)
    319         response = self.command_executor.execute(driver_command, params)
    320         if response:
--> 321             self.error_handler.check_response(response)
    322             response['value'] = self._unwrap_value(
    323                 response.get('value', None))

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py in check_response(self, response)
    240                 alert_text = value['alert'].get('text')
    241             raise exception_class(message, screen, stacktrace, alert_text)
--> 242         raise exception_class(message, screen, stacktrace)
    243 
    244     def _value_or_default(self, obj, key, default):

StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
  (Session info: chrome=75.0.3770.100)

Answer 1

对于每个循环，请尝试使用带范围的 for 循环。

links =[]

while True:

  result = driver.find_element_by_id('results')
# list of all relevant elements on the page
  docs = result.find_elements_by_xpath('//li[@class="row1"]')

  for i in range(len(docs)):
# this line produces StaleElementReferenceException
    WebDriverWait(driver, 30).until(EC.staleness_of(docs[i]))
    link = docs[i].find_element_by_tag_name('a')
    links.append(link.get_attribute('href'))
    result = driver.find_element_by_id('results')
    docs = result.find_elements_by_xpath('//li[@class="row1"]')

# try block checks if "next page" button exists and clicks it (this works fine)
  try:
    next_page = WebDriverWait(driver, 30).until(EC.presence_of_element_located((By.XPATH, '//a[contains(@class,"la-TriangleRight")]')))
    driver.execute_script('arguments[0].click();', next_page)

# if "next page" button doesn't exist we break out of the loop
  except:
    break