使用 selenium 提取网站的面包屑

Question

我需要提取本网站的面包屑：https://www.woolworths.com.au/Shop/Browse/drinks/cordials-juices-iced-teas/iced-teas

我试图检查元素并复制 xpath 但它没有提取它

from selenium import webdriver
driver = webdriver.Firefox()
driver.get('https://www.woolworths.com.au/Shop/Browse/drinks/cordials-juices-iced-teas/iced-teas')
driver.find_elements_by_xpath('//*[@id="center-panel"]/div/wow-tile-list-with-content/ng-transclude/wow-browse-tile-list/wow-tile-list/div/div[1]/div[1]/wow-breadcrumbs/div/ul/li[4]/span/span')

driver.find_element_by_css_selector('#center-panel > div > wow-tile-list-with-content > ng-transclude > wow-browse-tile-list > wow-tile-list > div > div.tileList > div.tileList-headerContainer > wow-breadcrumbs > div > ul > li:nth-child(4) > span > span')

我该如何继续？

Answer 1

要打印网站的面包屑：https://www.woolworths.com.au/Shop/Browse/drinks/cordials-juices-iced-teas/iced-teas，您必须为所需的visibility_of_element_located() 诱导 WebDriverWait，您可以使用以下任一Locator Strategies：

• 使用CSS_SELECTOR 和get_attribute() 方法：

  print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "ul.breadcrumbs-linkList li:nth-child(4) span span"))).get_attribute("innerHTML"))
  

• 使用XPATH 和text 属性：

  print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//ul[@class='breadcrumbs-linkList']//following-sibling::li[4]//span//span"))).text)
  

• 注意：您必须添加以下导入：

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  

---

结尾

根据文档：

• get_attribute() 方法Gets the given attribute or property of the element.
• text 属性返回 The text of the element.
• Difference between text and innerHTML using Selenium

Answer 2

您尝试抓取的页面是用 Angular 编写的，这意味着大多数 DOM elements 是由 JavaScript AJAX 代码动态加载的，并且一旦页面加载它们就不会出现。 (driver.get函数返回)

你应该使用waitsuntil函数来定位这些元素。

这是使用您提供的 XPATH 的工作示例：

driver.get('https://www.woolworths.com.au/Shop/Browse/drinks/cordials-juices-iced-teas/iced-teas')
try:
    element = WebDriverWait(driver, 1).until(
        EC.presence_of_element_located((By.XPATH, '//*[@id="center-panel"]/div/wow-tile-list-with-content/ng-transclude/wow-browse-tile-list/wow-tile-list/div/div[1]/div[1]/wow-breadcrumbs/div/ul/li[4]/span/span'))
    )
    print(element.text) ' this outputs Iced Teas
except TimeoutException:
    print("Timeout")

Answer 3

以下一项适用于我的验证

//*[span='first text' and span='Search results for "second text"']