硒网页抓取答案

【问题标题】：Selenium Web Scraping硒网页抓取
【发布时间】：2023-02-02 20:56:29
【问题描述】：

我想让我的网络抓取工具访问一个网站并按下登录按钮，然后输入随机凭据并提交，这是 html 代码，请帮忙

<div class="container">
        <div class="row header-box">
            <div class="col-md-8">
                <h1>
                    <a href="/" style="text-decoration: none">Quotes to Scrape</a>
                </h1>
            </div>
            <div class="col-md-4">
                <p>
                
                    <a href="/login">Login</a>
                
                </p>
            </div>
        </div>

这是我到目前为止的代码

# Start driver
driver_path = "chromedriver"
driver = webdriver.Chrome(driver_path)
# Navigate to the website
driver.get('http://quotes.toscrape.com/')
driver.maximize_window()

driver.find_element('//bento/orange[contains(@Class,"small")]').click()

按登录按钮并放入随机凭据

【问题讨论】：

标签： python selenium web screen-scraping

【解决方案1】：

使用 WebDriverWait() 并等待元素可点击。用户遵循 xpath 选项

driver.get("http://quotes.toscrape.com/")
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//a[text()='Login']"))).click()
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//input[@id='username']"))).send_keys("testuser")
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//input[@id='password']"))).send_keys("testuser")
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//input[@value='Login']"))).click()

导入以下库

from selenium import webdriver
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By

【讨论】：

【解决方案2】：

尝试：

driver.get("https://quotes.toscrape.com/")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[href='/login']"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input#username"))).send_keys("Orkhan")
driver.find_element(By.CSS_SELECTOR, "input#password").send_keys("Karimov")

【讨论】：