Python，Selenium：从字符串中提取价格

Question

我能够拼凑出一个程序，该程序可以打开网页、登录并使用 Selenium/Python 搜索特定部分。

我的最终目标是循环浏览一系列零件编号，然后返回价格，打印到列表中。

目前，我正在尝试提取零件的价格数据，但不确定如何。 我在此站点中搜索时返回了一堆零件。没有零件具有与价格相关的属性。我不知道如何隔离我需要的部分。

我专门制作了一个联合国/密码来提问。这个非私有的 UN/PW 是：

userName = "FirstName.SurName321123@gmail.com"
password = "PasswordForThis123"

网站是Tessco.com

我假设第一个挑战是从返回的列表中找到我需要的部分。我知道我可以使用以下语法找到一个项目：

driver.find_element(By.ID, "someID").get_attribute("attribute")

但是如果项目没有属性，我如何提取它的数据？一些提取字符串的方法？

我正在考虑调用 IF 函数，说明是否有“MFG PART #:”==“有问题的字符串”，在本例中为 HL4RP-50，然后打印价格。

如果我能够分离出有问题的部分，我如何提取它的价格？

Tessco 网站的 HTML 代码是：

    <div class="row">
                                                <div class="col-xs-5 col-sm-2 col-md-2 productImage">
                                                    <a href="/product/1-2-plenum-air-cable-off-white-574840" class="CoveoResultLink" onclick="ClickToProductDetailGA({name: &quot;1/2\&quot; Plenum Air Cable, Off White&quot;, sku: 574840, price: &quot;$1.89&quot;, brand: &quot;CommScope&quot;, category: &quot;Cable Products| Coaxial Cable, Connectors &amp; Jumpers| Air Coaxial Cable| 1/2\&quot; Air Cable&quot;, position: 0, pageType: &quot;Search Page&quot;, url: &quot;/product/574840&quot; });" tabindex="0">
                                                        <img src="https://avalanche.tessco.com/productimages/250x250/1462639.jpg" alt="CommScope">
                                                    </a>
                                                    <a class="hidden-xs" href="/search#f:manufacturerName=[CommScope]">
                                                        CommScope
                                                    </a>

                                                    <span class="badge blueBadge">GSA</span>

                                                </div>
                                                <div class="col-xs-7 visible-xs detailMobile">
                                                    <a href="/product/1-2-plenum-air-cable-off-white-574840" class="productName CoveoResultLink" onclick="ClickToProductDetailGA({name: &quot;1/2\&quot; Plenum Air Cable, Off White&quot;, sku: 574840, price: &quot;$1.89&quot;, brand: &quot;CommScope&quot;, category: &quot;Cable Products| Coaxial Cable, Connectors &amp; Jumpers| Air Coaxial Cable| 1/2\&quot; Air Cable&quot;, position: 0, pageType: &quot;Search Page&quot;, url: &quot;/product/574840&quot; });" tabindex="0">1/2" Plenum Air Cable, Off White</a>
                                                </div>
                                                <div class="col-xs-12 col-sm-6 col-md-7 detail">
                                                    <div>
                                                        <a href="/product/1-2-plenum-air-cable-off-white-574840" class="productName CoveoResultLink hidden-xs" onclick="ClickToProductDetailGA({name: &quot;1/2\&quot; Plenum Air Cable, Off White&quot;, sku: 574840, price: &quot;$1.89&quot;, brand: &quot;CommScope&quot;, category: &quot;Cable Products| Coaxial Cable, Connectors &amp; Jumpers| Air Coaxial Cable| 1/2\&quot; Air Cable&quot;, position: 0, pageType: &quot;Search Page&quot;, url: &quot;/product/574840&quot; });" tabindex="0">1/2" Plenum Air Cable, Off White</a>
                                                        <div class="row">
                                                            <div class="col-xs-12">
                                                                <ul class="unlisted info">
                                                                    <li><span>TESSCO SKU:</span> 574840</li>
                                                                    <li><span>QTY:</span> 1 FOOT</li>
                                                                    <li><span>UPC:</span> 888063388620</li>
                                                                    <li><span>MFG PART #:</span> HL4RPV-50</li>
                                                                </ul>
                                                            </div>
                                                        </div>
                                                        <p class="more">ANDREW 1/2" Plenum Air 50 ohm cable. HL4RPV-50. Uses LDF4 connectors. Off…</p>
                                                    </div>
                                                </div>
                                                <div class="col-xs-12 col-sm-4 col-md-3 purchase">
                                                    <div>
                                                        <add-product-to-cart params="
                                                                             sku: 574840,
                                                                             price: &quot;$1.89&quot;,
                                                                             listPrice: &quot;$6.37&quot;,
                                                                             canPurchase: &quot;true&quot;,
                                                                             isAuthenticated: true,
                                                                             name: &quot;1/2\&quot; Plenum Air Cable, Off White&quot;,
                                                                             brand: &quot;CommScope&quot;,
                                                                             category: &quot;Cable Products| Coaxial Cable, Connectors &amp; Jumpers| Air Coaxial Cable| 1/2\&quot; Air Cable&quot;,
                                                                             pageType: &quot;Search Page&quot;,
                                                                             brandProtectionLink:&quot;/brand-protection-program&quot;,
                                                                             viewProductPricingText: &quot;viewAccountPricingOnTCOM&quot;,
                                                                             userRoles: &quot;canBuy, authorizedBuyerOnTCOM, viewAccountAvailabilityOnTCOM, viewAccountPricingOnTCOM, viewOrderHistoryOnTCOM, overrideShiptoAddressOnTCOM&quot;,
                                                                             minQuantity:1,
                                                                             minQuantityBefore: &quot;Minimum &quot;,
                                                                             minQuantityAfter: &quot; to Order&quot;,
                                                                             isOnSale: &quot;No&quot;,
                                                                             saleClass:&quot;redBadge&quot;,
                                                                             saleText:&quot;Sale&quot;,
                                                                             isCutCable: &quot;true&quot;,
                                                                             maximumReelLength: 2000,
                                                                             isCableJumper: false,
                                                                             isPricingWrapperAlive: true,
                                                                             context: &quot;search&quot;,
                                                                             index: 0, index: 0" data-sellingrestrictioncode="N/A"><div class="price" data-bind="visible: ((canPurchase()===true) &amp;&amp; (isAuthenticated()===true)), css: {sale: isOnSale} ">
        <span data-bind="text: 'List: ' + listPrice()">List: $6.37</span>
        <span data-bind="visible: isOnSale, css:saleClass, text: saleText" class="badge large redBadge" style="display: none;">Sale</span><!--
        --><!--ko text: canViewPricing()===true ? price : listPrice-->$1.89<!--/ko-->
    </div>

到目前为止，我的 Selenium 代码是：

    import time
    #Need Selenium for interacting with web elements
    from selenium import webdriver
    from selenium.webdriver.support import expected_conditions as EC
    #Need numpy/pandas to interact with large datasets
    import numpy as np
    import pandas as pd

    chrome_path = r"C:\Users\James\Documents\Python Scripts\jupyterNoteBooks\ScrapingData\chromedriver_win32\chromedriver.exe"
    driver = webdriver.Chrome(chrome_path)
    driver.get("https://www.tessco.com/login")

    userName = "FirstName.SurName321123@gmail.com"
    password = "PasswordForThis123"

    #Set a wait, for elements to load into the DOM
    wait = WebDriverWait(driver, 10)

    elem = wait.until(EC.element_to_be_clickable((By.ID, "userID"))) 
    elem.send_keys(userName)

    elem = wait.until(EC.element_to_be_clickable((By.ID, "password"))) 
    elem.send_keys(password)

    #Press the login button
    driver.find_element_by_xpath("/html/body/account-login/div/div[1]/form/div[6]/div/button").click()

    #Expand the search bar
    searchIcon = wait.until(EC.element_to_be_clickable((By.XPATH, "/html/body/header/div[2]/div/div/ul/li[2]/i"))) 
    searchIcon.click()

    searchBar = wait.until(EC.element_to_be_clickable((By.XPATH, '/html/body/header/div[3]/input'))) 
    searchBar.click()

    #load in manufacture part number from a collection of components, via an Excel file

    #Enter information into the search bar
    searchBar.send_keys("HL4RPV-50" + '\n')

任何指针将不胜感激。

Answer 1

这是您需要的逻辑。

# wait for the products information loaded
products = WebDriverWait(driver,30).until(EC.presence_of_all_elements_located((By.XPATH,"//div[@class='CoveoResult']")))
# create a dictionary to store product and price
productInfo = {}
# iterate through all products in the search result and add details to dictionary
for product in products:
    # get product name
    productName = product.find_element_by_xpath(".//a[@class='productName CoveoResultLink hidden-xs']").text
    # get price
    price = product.find_element_by_xpath(".//div[@class='price']").text.split('\n')[1]
    # add details to dictionary
    productInfo[productName] = price
# print products information   
print(productInfo)

下面是输出：

{'1/2" Plenum Air Cable, Off White': '$6.37', '1/2" Plenum Air Cable, Blue': '$6.37', '4.3-10 Male for 1/2" AL4RPV-50 ,LDF4-50A,HL4RPV-50': '$25.91', '4.3-10M RA for 1/2" AL4RPV-50, LDF4-50A, HL4RPV-50': '$51.28', '4.3-10 Male connector for 1/ 2” Plenum 电缆”：“34.32 美元”，“1/2” Plenum 的 4.3-10 母接头”：“35.00 美元”，“1/2” Plenum 的 4.3-10 R/A 公接头”：“47.50 美元”，“ 4.3-10 女性 1/2 in AL4RPV-50, LDF4-50A': '$25.91'}

Answer 2

您需要以某种方式获取该价格的路径，然后获取外部 html 代码，使用 substring on in 来获取您需要的那个，即外部 html 中的值。

price_element = driver.find_element_by_xpath('#xpath of the price here')
price_html = price_element.get_attribute('outerHTML')
price_html = price_html[#substring here]

很难复制或显示，因为您尝试抓取的网站需要登录才能查看价格。

但我希望这能给你一个想法。祝你好运：D。