Selenium 的隐式延迟答案

【问题标题】：Implicit delay with SeleniumSelenium 的隐式延迟
【发布时间】：2021-12-02 13:25:33
【问题描述】：

我正在练习硒。我尝试用beautifulsoup 抓取这个页面，但是随着页面需要一些时间来加载，我用selenium 尝试了它并添加了延迟时间。我也尝试过 drive.delay 和 timeout 。但对我来说没有什么好用的。我在所有尝试中都遇到了相同的错误，我附上了我最近的尝试和我遇到的错误。

这是我的朋友：

from selenium import webdriver
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By


url = 'https://www.checkatrade.com/search?page=1&categoryId=1164&location=London'

driver =webdriver.Chrome(executable_path='D:\Web_Scraping\chromedriver_win32\chromedriver.exe')

driver.get(url)
driver.implicitly_wait(20)

items = driver.find_element_by_class_name('sc-8keyp7-0 kbPUFb sc-1a2v1qv-0 gwBNSh sc-3jeiz1-1 bfuahU')

for item in items:
    company = item.find_element_by_xpath('//*[@id="1052063"]/div/div[2]/div[2]/a/div[1]/h2').text
    print(company)

但它给我抛出了这个错误，我无法弄清楚。

DeprecationWarning: executable_path has been deprecated, please pass in a Service object
  driver =webdriver.Chrome(executable_path='D:\Web_Scraping\chromedriver_win32\chromedriver.exe')

DevTools listening on ws://127.0.0.1:1090/devtools/browser/9ed2753e-5d6e-4382-a9ac-a0ed2fbb8379
d:/Web_Scraping/seleniumtest.py:21: DeprecationWarning: find_element_by_* commands are deprecated. Please use find_element() instead
  items = driver.find_element_by_class_name('sc-8keyp7-0 kbPUFb sc-1a2v1qv-0 gwBNSh sc-3jeiz1-1 bfuahU')
[5536:25384:1014/115806.197:ERROR:chrome_browser_main_extra_parts_metrics.cc(228)] crbug.com/1216328: Checking Bluetooth availability started. Please report if there is no report that this ends.
[5536:25384:1014/115806.198:ERROR:chrome_browser_main_extra_parts_metrics.cc(231)] crbug.com/1216328: Checking Bluetooth availability ended.
[5536:25384:1014/115806.199:ERROR:chrome_browser_main_extra_parts_metrics.cc(234)] crbug.com/1216328: Checking default browser status started. Please report if there is no report that this ends.
[5536:23016:1014/115806.200:ERROR:device_event_log_impl.cc(214)] [11:58:06.200] USB: usb_device_handle_win.cc:1048 Failed to read descriptor from node connection: A device attached to the system is not functioning. (0x1F)
[5536:25384:1014/115806.733:ERROR:chrome_browser_main_extra_parts_metrics.cc(238)] crbug.com/1216328: Checking default browser status ended.
Traceback (most recent call last):
  File "d:/Web_Scraping/seleniumtest.py", line 21, in <module>
    items = driver.find_element_by_class_name('sc-8keyp7-0 kbPUFb sc-1a2v1qv-0 gwBNSh sc-3jeiz1-1 bfuahU')
  File "C:\Python38\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 754, in find_element_by_class_name
    return self.find_element(by=By.CLASS_NAME, value=name)
  File "C:\Python38\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 1238, in find_element
    return self.execute(Command.FIND_ELEMENT, {
  File "C:\Python38\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 418, in execute
    self.error_handler.check_response(response)
  File "C:\Python38\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 243, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".sc-8keyp7-0 kbPUFb sc-1a2v1qv-0 gwBNSh sc-3jeiz1-1 bfuahU"}
  (Session info: chrome=94.0.4606.81)
Stacktrace:
Backtrace:
        Ordinal0 [0x0107BDE3+2473443]
        Ordinal0 [0x01016661+2057825]
        Ordinal0 [0x00F22438+1057848]
        Ordinal0 [0x00F4C7DB+1230811]
        Ordinal0 [0x00F76712+1402642]
        Ordinal0 [0x00F6571A+1333018]
        Ordinal0 [0x00F74ABF+1395391]
        Ordinal0 [0x00F655AB+1332651]
        Ordinal0 [0x00F42104+1188100]
        Ordinal0 [0x00F42F59+1191769]
        GetHandleVerifier [0x01202266+1549718]
        GetHandleVerifier [0x012AD4A7+2250711]
        GetHandleVerifier [0x0110718B+521403]
        GetHandleVerifier [0x01106229+517465]
        Ordinal0 [0x0101B79D+2078621]
        Ordinal0 [0x0101FB58+2095960]
        Ordinal0 [0x0101FC92+2096274]
        Ordinal0 [0x01029541+2135361]
        BaseThreadInitThunk [0x764DFA29+25]
        RtlGetAppContainerNamedObjectPath [0x77A77A9E+286]
        RtlGetAppContainerNamedObjectPath [0x77A77A6E+238]

是什么导致了这个错误？

【问题讨论】：

标签： python selenium selenium-webdriver css-selectors selenium-chromedriver

【解决方案1】：

您使用了错误的定位器。
find_element_by_class_name 方法旨在在您将多个类名值传递给它“find_element_by_class_name”时接收一个类名。
要传递多个类名，您应该使用 css 选择器或 XPath。
所以不是

items = driver.find_element_by_class_name('sc-8keyp7-0 kbPUFb sc-1a2v1qv-0 gwBNSh sc-3jeiz1-1 bfuahU')

可能是

items = driver.find_element_by_css_selector('.sc-8keyp7-0.kbPUFb.sc-1a2v1qv-0.gwBNSh.sc-3jeiz1-1.bfuahU')

或者

items = driver.find_element_by_xpath("//div[@class='sc-8keyp7-0 kbPUFb sc-1a2v1qv-0 gwBNSh sc-3jeiz1-1 bfuahU']")

【讨论】：