使用类名、css-selector 和 xpath 从网站中查找元素

Question

我正在寻找有关我的代码的帮助。

我正在尝试从网站打印出一个元素，但我从我尝试的方法中收到不同的错误消息，它们都不起作用。

我已经尝试使用带有以下代码的 class_name：

from selenium import webdriver    
browser = webdriver.Chrome('/home/cena/Downloads/chromedriver')

poeisz_url = "https://webwinkel.poiesz-supermarkten.nl/boodschappen/producten/631258"
poeisz_classname = "full-product full-product--price__for"

browser.get(poeisz_url)
elem = browser.find_element_by_class_name(poeisz_classname)
print(elem.text)

这返回了以下错误消息：

Traceback (most recent call last):
  File "/home/cena/.PyCharmCE2019.1/config/scratches/scratch_2.py", line 9, in <module>
    elem = browser.find_element_by_class_name(poeisz_classname)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 564, in find_element_by_class_name
    return self.find_element(by=By.CLASS_NAME, value=name)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 978, in find_element
    'value': value})['value']
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: Compound class names not permitted
  (Session info: chrome=74.0.3729.169)
  (Driver info: chromedriver=74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729@{#29}),platform=Linux 4.15.0-50-generic x86_64)

所以我尝试使用 xpath 和 css_selector 来解决它：

from selenium import webdriver

browser = webdriver.Chrome('/home/cena/Downloads/chromedriver')

poeisz_url = "https://webwinkel.poiesz-supermarkten.nl/boodschappen/producten/631258"
poeisz_classname = "full-product full-product--price__for"

browser.get(poeisz_url)
elem = browser.find_element_by_xpath("//*[@class='" + poeisz_classname + "']")
print(elem.text)

from selenium import webdriver

browser = webdriver.Chrome('/home/cena/Downloads/chromedriver')

poeisz_url = "https://webwinkel.poiesz-supermarkten.nl/boodschappen/producten/631258"
poeisz_classname = "full-product full-product--price__for"

browser.get(poeisz_url)
elem = browser.find_element_by_css_selector("*[class^='" + poeisz_classname + "']")
print(elem.text)

他们按顺序返回了这些错误：

Traceback (most recent call last):
  File "/home/cena/.PyCharmCE2019.1/config/scratches/scratch_2.py", line 9, in <module>
    elem = browser.find_element_by_xpath("//*[@class='" + poeisz_classname + "']")
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 394, in find_element_by_xpath
    return self.find_element(by=By.XPATH, value=xpath)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 978, in find_element
    'value': value})['value']
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@class='full-product full-product--price__for']"}
  (Session info: chrome=74.0.3729.169)
  (Driver info: chromedriver=74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729@{#29}),platform=Linux 4.15.0-50-generic x86_64)

Traceback (most recent call last):
  File "/home/cena/.PyCharmCE2019.1/config/scratches/scratch_2.py", line 9, in <module>
    elem = browser.find_element_by_css_selector("*[class^='" + poeisz_classname + "']")
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 598, in find_element_by_css_selector
    return self.find_element(by=By.CSS_SELECTOR, value=css_selector)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 978, in find_element
    'value': value})['value']
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"*[class^='full-product full-product--price__for']"}
  (Session info: chrome=74.0.3729.169)
  (Driver info: chromedriver=74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729@{#29}),platform=Linux 4.15.0-50-generic x86_64)

我对编程很陌生，所以如果它是愚蠢的，我很抱歉。提前感谢您的帮助！

Answer 1

你应该使用WebDriverWait:

例如 XPath：

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC


driver = webdriver.Chrome('/home/cena/Downloads/chromedriver')
url = "https://webwinkel.poiesz-supermarkten.nl/boodschappen/producten/631258"
driver.get(url)
my_el = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, '//div[@id="products"]//*[@class="full-product full-product--price__for"]')))
print(my_el.text)

输出是：

0 99

使用WebDriverWait 的更好方法是将其用作var：

wait = WebDriverWait(driver, 20)
my_el = wait.until(EC.presence_of_element_located((By.XPATH, '//div[@id="products"]//*[@class="full-product full-product--price__for"]')))

正如@DebanjanB 所说，您可能想使用visibility_of_element_located() 而不是presence_of_element_located。

希望对您有所帮助！

Answer 2

您遇到的问题在第一条错误消息中进行了说明。

无效的选择器：不允许复合类名

find_element_by_class_name() 将一个单个类名作为参数，但“full-product full-product--price__for”包含两个类名（注意：类名用空格分隔，因此有这里有两个...“full-product”和“full-product--price__for”。

您可以改为将定位器转换为 CSS 选择器 .full-product.full-product--price__for，它应该可以正常工作。

要了解有关 CSS 选择器的更多信息，请参阅下面的链接。

Selenium Tips: CSS Selectors
WC3 reference, Selectors Level 4
Taming Advanced CSS Selectors