无法使用 Selenium 单击下一步按钮

Question

我已经做了好几个小时了，但没有任何进展。我正在尝试单击此页面上的下一步按钮here

这是我的代码：

#!/usr/local/bin python3

import sys
import time
import re
import logging
from selenium import webdriver
from selenium.webdriver.firefox.options import Options as options
from bs4 import BeautifulSoup as bs
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.common.action_chains import ActionChains


_USE_VIRTUAL_DISPLAY = False
_FORMAT = '%(asctime)s - %(levelname)s - %(name)s - %(message)s'
# logging.basicConfig(filename=LOG_FILENAME,level=logging.DEBUG)
logging.basicConfig(format=_FORMAT, level=logging.INFO)
_LOGGER = logging.getLogger(sys.argv[0])
_DEFAULT_SLEEP = 0.5


try:
    options = options()
    # options.headless = True
    

    driver = webdriver.Firefox(options=options, executable_path=r"/usr/local/bin/geckodriver")
    
    print("Started Browser and Driver")


except:
    _LOGGER.info("Can not run headless mode.")

url = 'https://www.govinfo.gov/app/collection/uscourts/district/alsd/2021/%7B%22pageSize%22%3A%22100%22%2C%22offset%22%3A%220%22%7D'

driver.get(url)
time.sleep(5)

page = driver.page_source
soup = bs(page, "html.parser")


next_page = WebDriverWait(driver,5).until(EC.element_to_be_clickable((By.XPATH,'//*[@id="collapseOne1690"]/div/span[1]/div/ul/li[8]/a')))
if next_page:
    print('*****getting next page*****')
    # driver.execute_script('arguments[0].click()', next_page)
    next_page.click()
    time.sleep(3)
    
else:
    print('no next page')
    

driver.quit()

我收到超时错误。我试过改变 XPath。我试过 ActionChains 滚动到视图中，但没有一个有效。任何帮助表示赞赏。

Answer 1

1 您的 XPATH 不起作用，因为它使用动态类名 collapseOne1690，如前所述。 而且，即使你使用了这个类名的一部分，它也不是很稳定。 如果您更喜欢 XPaths，我建议您使用这个：//span[@class='custom-paginator']//li[@class='next fw-pagination-btn']/a 或只是 //li[@class='next fw-pagination-btn']/a。你也可以使用css选择器：.next.fw-pagination-btn

2 我把日志代码去掉了，因为它也有一些问题，重新检查一下。

3 5 秒显式等待太小。让它至少 10 秒，最好 15 秒。这只是一个建议。

点击按钮并使用 Firefox 的最小可重现代码是：

from selenium import webdriver
from selenium.webdriver.firefox.options import Options as options
from bs4 import BeautifulSoup as bs
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By

options = options()
# options.headless = True

driver = webdriver.Firefox(options=options)

print("Started Browser and Driver")

url = 'https://www.govinfo.gov/app/collection/uscourts/district/alsd/2021/%7B%22pageSize%22%3A%22100%22%2C%22offset%22%3A%220%22%7D'

driver.get(url)

page = driver.page_source
soup = bs(page, "html.parser")
print(soup)

next_page = WebDriverWait(driver, 15).until(
    EC.element_to_be_clickable((By.XPATH, "//span[@class='custom-paginator']//li[@class='next fw-pagination-btn']/a")))
next_page.click()

# driver.quit()

Answer 2

当我加载此页面时，显示 div id 是动态分配的。第一次加载页面，id是collapseOne5168，第二次是collapseOne1136

您可以考虑改用find_element_by_class_name("next fw-pagination-btn")？