在 Selenium Python 中使用 Keys.PAGE_DOWN 滚动

Question

大家好，任何人都可以帮我滚动https://www.grainger.com/category/black-pipe-fittings/pipe-fittings/pipe-tubing-and-fittings/plumbing/ecatalog/N-qu1?searchRedirect=products

我想用

滚动这个

actions = ActionChains(browser)
actions.send_keys(Keys.PAGE_DOWN)
actions.perform()

直到它到达滚动的底部，它会找到一个元素“加载更多”

loadMoreButton = browser.find_element_by_css_selector(
                        ".btn.list-view__load-more.list-view__load-more--js")
loadMoreButton.click()

然后突然点击加载更多按钮，它必须再次执行滚动操作，然后再次加载更多操作，直到加载更多按钮不可用。

我必须使用此页面向下操作，因为元素在页面滚动到元素之前不会加载，如果有人可以提出一些解决方案将有很大帮助

Answer 1

这对我来说是零问题...

from selenium.webdriver.common.keys import Keys

driver.find_element_by_tag_name('body').send_keys(Keys.PAGE_DOWN)

Answer 2

滚动页面https://www.grainger.com/category/black-pipe-fittings/pipe-fittings/pipe-tubing-and-fittings/plumbing/ecatalog/N-qu1?searchRedirect=products直到它到达页面底部，它会找到一个文本为查看更多的元素，然后单击该元素直到该元素不可用，您可以使用以下解决方案：

• 代码块：

  from selenium import webdriver
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.support import expected_conditions as EC
  from selenium.common.exceptions import StaleElementReferenceException
  from selenium.common.exceptions import TimeoutException
  
  options = webdriver.ChromeOptions() 
  options.add_argument("start-maximized")
  options.add_argument('disable-infobars')
  browser=webdriver.Chrome(chrome_options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
  browser.get("https://www.grainger.com/category/black-pipe-fittings/pipe-fittings/pipe-tubing-and-fittings/plumbing/ecatalog/N-qu1?searchRedirect=products")
  while True:
      try:
          browser.execute_script("return arguments[0].scrollIntoView(true);", WebDriverWait(browser,10).until(EC.visibility_of_element_located((By.XPATH, "//a[@class='btn list-view__load-more list-view__load-more--js' and normalize-space()='View More']"))))
          browser.execute_script("arguments[0].click();", WebDriverWait(browser,10).until(EC.element_to_be_clickable((By.XPATH, "//a[@class='btn list-view__load-more list-view__load-more--js' and normalize-space()='View More']"))))
          print("View More button clicked")
      except (TimeoutException, StaleElementReferenceException) as e:
          print("No more View More buttons")
          break
  browser.quit()
  

• 控制台输出：

  View More button clicked
  View More button clicked
  No more View More buttons
  

Answer 3

@PedroLobito 我正在尝试检索产品链接，你能帮我吗 在这

---

在这种情况下不需要selenium，只需sniffxhr 通过developer tools 请求并直接获得黄金 (json)。

---

产品的url结构如下：

https://www.x.com/product/anything-Item#

只需在网址末尾的json object 中添加Item # 值，例如：

1. https://www.x.com/product/anything-5P540
2. https://www.x.com/product/anything-5P541

...

---

py3 示例（对于py2，只需更改format 语法）：

import json
import requests

main_cat = "WP7115916"
sub_cat = "4836"

x = requests.get(f"https://www.x.com/product/tableview/GRAINGER-APPROVED-Square-Head-Plugs-{main_cat}/_/N-qu1?searchRedirect=products&breadcrumbCatId={sub_cat}&s_pp=false").json()

for p in x['records']:
    for childs in p['children']:
        for item in json.loads(childs['collapseValues']):
            url = f"https://www.x.com/product/lol-{item['sku']}"
            print(url)

---

https://www.x.com/product/lol-5P540
https://www.x.com/product/lol-5P541
https://www.x.com/product/lol-5P542
https://www.x.com/product/lol-5P543
https://www.x.com/product/lol-5P544
https://www.x.com/product/lol-5P545
https://www.x.com/product/lol-5P546
https://www.x.com/product/lol-5P547
https://www.x.com/product/lol-5P548
...