如何使用 Selenium 和 Python 提取第一个搜索结果的 href 属性

Question

我的 excel 上有一个书籍清单，对于每本书，我想用摘要填写一列。为此，我要去goodreads.com，搜索“哈利波特”打开第一个结果，然后复制粘贴摘要文本。但是，无法获取第一个搜索结果的链接。这是我的代码。我参考的链接：Python Selenium - get href value

from selenium import webdriver
from selenium.webdriver.common.keys import Keys

driver=webdriver.Chrome()
driver.get('https://goodreads.com')


loginbox=driver.find_element_by_xpath('//*[@id="userSignInFormEmail"]')
loginbox.send_keys('shivam01anand@gmail.com')
passwordbox=driver.find_element_by_xpath('//*[@id="user_password"]')
passwordbox.send_keys('shivam03')
loginButton=driver.find_element_by_xpath('//*[@id="sign_in"]/div[3]/input[1]')
loginButton.click()

searchbox=driver.find_element_by_xpath('/html/body/div[2]/div/header/div[2]/div/div[2]/form/input[1]')
searchbox.send_keys('harry potter')

searchButton=driver.find_element_by_xpath('/html/body/div[2]/div/header/div[2]/div/div[2]/form/button')
searchButton.click()

elem=driver.find_element_by_css_selector("bookTitle").get_attribute("href")
print(elem)
#elem = driver.find_element_by_css_selector("bookTitle [href]")

Error: NoSuchElementException: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[2]/div/header/div[2]/div/div[2]/form/input[1]"}
  (Session info: chrome=83.0.4103.116)

此错误仅在我编写 elem 行时出现，这很奇怪，因为错误是前一行。完全糊涂了。

Answer 1

要打印第一个搜索结果的 href 属性的值，您必须为visibility_of_element_located() 引入WebDriverWait，您可以使用以下任一Locator Strategies：

• 使用CSS_SELECTOR：

  driver.get("https://goodreads.com")
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[@id='userSignInFormEmail']"))).send_keys("shivam01anand@gmail.com")
  driver.find_element_by_xpath("//input[@id='user_password']").send_keys("shivam03")
  driver.find_element_by_xpath("//input[@value='Sign in']").click()
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.NAME, "q"))).send_keys("harry potter")
  driver.find_element_by_xpath("//button[@aria-label='Search']").click()
  # extracting the _href_ attribute of the first search result using CSS_SELECTOR
  print(WebDriverWait(driver, 5).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "table.tableList > tbody > tr td a.bookTitle"))).get_attribute("href"))
  

• 使用XPATH：

  driver.get("https://goodreads.com")
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[@id='userSignInFormEmail']"))).send_keys("shivam01anand@gmail.com")
  driver.find_element_by_xpath("//input[@id='user_password']").send_keys("shivam03")
  driver.find_element_by_xpath("//input[@value='Sign in']").click()
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.NAME, "q"))).send_keys("harry potter")
  driver.find_element_by_xpath("//button[@aria-label='Search']").click()
  # extracting the _href_ attribute of the first search result using XPATH
  print(WebDriverWait(driver, 5).until(EC.visibility_of_element_located((By.XPATH, "//table[@class='tableList']/tbody/tr//td//a[@class='bookTitle']"))).get_attribute("href"))
  

• 注意：您必须添加以下导入：

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  

• 控制台输出：

  https://www.goodreads.com/book/show/3.Harry_Potter_and_the_Sorcerer_s_Stone?from_search=true&from_srp=true&qid=3nIjRXwsfG&rank=1
  

---

参考文献

您可以在NoSuchElementException 上找到一些相关讨论：

• selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element while trying to click Next button with selenium
• selenium in python : NoSuchElementException: Message: no such element: Unable to locate element