点击显示更多按钮；用 python 刮硒

Question

我正在尝试使用显示更多按钮抓取网站；我无法点击它。 网址是：https://www.wtatennis.com/rankings/singles

我的代码是：

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver import ActionChains
from tqdm import tqdm
import time

options = Options()
options.add_argument("--headless")

browser = webdriver.Chrome(ChromeDriverManager().install(),options=options)
browser.get('https://www.wtatennis.com/rankings/singles')

action = ActionChains(browser)
showmore = browser.find_elements_by_xpath(".//button[contains(@class, 'btn widget-footer__more-button rankings__show-more js-show-more-button')]")
action.move_to_element(showmore).perform()
showmore.click()
time.sleep(5)

有人知道吗？谢谢！

Answer 1

当您从根目录开始搜索时，请勿在定位器中使用“.//”，因为没有当前元素，您的定位器将找不到任何元素。您也可以使用任何属性来唯一地查找元素。见以下代码：

browser = webdriver.Chrome(options=options)
browser.get('https://www.wtatennis.com/rankings/singles')


WebDriverWait(browser,10).until(EC.element_to_be_clickable((By.XPATH,
    '//*[@data-text="Accept Cookies"]'))).click()


WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH,
    '//*[@data-text = "Show More"]'))).click()

使用 webdriver 等待和数据属性

你使用等待导入：

from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By

要等到所有元素都加载完毕，你必须确保最后一个元素没有变化，如果它变化保持滚动。

browser.get('https://www.wtatennis.com/rankings/singles')


WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH,
                                                             '//*[@data-text="Accept Cookies"]'))).click()

value = "start"
WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH,
                                                             '//*[@data-text = "Show More"]'))).click()


while(browser.find_element_by_xpath("(//tr[@class='rankings__row'])[last()]").text != value):
    elem = browser.find_element_by_xpath(
        '(//*[contains(text(),"Loading")])[2]')
    value = browser.find_element_by_xpath(
        "(//tr[@class='rankings__row'])[last()]").text

    browser.execute_script("arguments[0].scrollIntoView()", elem)

    WebDriverWait(browser, 10).until(EC.presence_of_all_elements_located((By.XPATH,
                                                                          "//tr[@class='rankings__row']")))

    try:
       WebDriverWait(browser, 10).until_not(EC.text_to_be_present_in_element((By.XPATH,
                                                                           "(//tr[@class='rankings__row'])[last()]"), value))
    except:
        None