由 BeautifulSoup 刮刮生产图像答案

【问题标题】：Scraping Produce image by BeautifulSoup由 BeautifulSoup 刮刮生产图像
【发布时间】：2022-11-10 23:40:45
【问题描述】：

我正在尝试废弃此页面的产品图片：

https://www.noon.com/egypt-ar/golden-wood-edp-100ml/N39185122A/p/?o=d55236b5f16d3c9d

但我仍然有一个错误，我使用 beaurifaulSoup，但它没有给我：

我得到的就是：

import requests
from bs4 import BeautifulSoup
import time
import json
import re



page_numbers = 1




 
while True:

    URL = "https://www.noon.com/egypt-ar/beauty-and-health/beauty/eg-nov22-clearance-2/?limit=50&page="+str(page_numbers)

    headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36", "Accept-Encoding":"gzip, deflate", "Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "DNT":"1","Connection":"close", "Upgrade-Insecure-Requests":"1"}

    page = requests.get(URL, headers=headers)
    soup = BeautifulSoup(page.text, "html.parser")
    products = soup.find_all("span",class_="sc-5e739f1b-0 gEERDr wrapper productContainer")
    


    time.sleep(1) 
    product_Source =soup.find("div",class_="sc-95314d04-3 dnAXkE")
    product_String = product_Source.text
    all_words = product_String.split()
    products_Numbers= all_words[0]

#loop to Get Data

    for product in products:

        product_link = product.find("a").attrs["href"]                                                         #Get product product links
        
        productFullLink = "https://www.noon.com"+product_link
        Moreinfo = requests.get(productFullLink,headers=headers)                                         # Get informattion from page
        Product_Details = BeautifulSoup(Moreinfo.text,"html.parser")
        #time.sleep(1)

       
            
        product_image = Product_Details.find_all("div",class_="sc-8cbb8e24-2 eDXUdi")
       
        

        print(product_image)

【问题讨论】：

这些链接不包含任何具有“sc-8cbb8e24-2”类的 div。它们有 -0、-1 和 -3，但没有 -2。
我尝试使用 3 个选项，但没有得到图像 div。在检查中它有 2 对此不确定，@ 987654323@ 请你帮我提供确切的代码吗？谢谢

标签： python web-scraping beautifulsoup

【解决方案1】：

对我来说，以下selector 有效：

product_image = Product_Details.select(".swiper-wrapper .lazyload-wrapper div")
print(product_image)

对于前 3 个产品，它会打印

[<div class="sc-8cbb8e24-2 eDXUdi"><img alt="عطر جولدن وود EDP 100مل - v1666705361/N39185122A_1" aria-hidden="true" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_80/v1666705361/N39185122A_1.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="عطر جولدن وود EDP 100مل - v1618317832/N39185122A_2" aria-hidden="true" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_80/v1618317832/N39185122A_2.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="عطر جولدن وود EDP 100مل - v1618317832/N39185122A_3" aria-hidden="true" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_80/v1618317832/N39185122A_3.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="عطر جولدن وود EDP 100مل - v1617954175/N39185122A_5" aria-hidden="true" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_80/v1617954175/N39185122A_5.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="عطر جولدن وود EDP 100مل - v1617954175/N39185122A_4" aria-hidden="true" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_80/v1617954175/N39185122A_4.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="v1666705361/N39185122A_1" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_400/v1666705361/N39185122A_1.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="v1618317832/N39185122A_2" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_400/v1618317832/N39185122A_2.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="v1618317832/N39185122A_3" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_400/v1618317832/N39185122A_3.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="v1617954175/N39185122A_5" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_400/v1617954175/N39185122A_5.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="v1617954175/N39185122A_4" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_400/v1617954175/N39185122A_4.jpg"/></div>]
[<div class="sc-8cbb8e24-2 eDXUdi"><img alt="عطر سايلنسز EDP 100مل - v1562242787/N11201003A_1" aria-hidden="true" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_80/v1562242787/N11201003A_1.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="عطر سايلنسز EDP 100مل - v1562243187/N11201003A_2" aria-hidden="true" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_80/v1562243187/N11201003A_2.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="v1562242787/N11201003A_1" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_400/v1562242787/N11201003A_1.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="v1562243187/N11201003A_2" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_400/v1562243187/N11201003A_2.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="عطر سايلنسز EDP 100مل" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_80/v1562242787/N11201003A_1.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="مزيل عرق فريشلي فري ببكرة دوارة برائحة الجريب فروت والليتشي 50مل" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_80/v1654077085/N42721327A_1.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="عطر سايلنسز EDP 100مل" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_80/v1562242787/N11201003A_1.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="مزيل عرق فريشلي فري ببكرة دوارة برائحة الجريب فروت والليتشي 50مل" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_80/v1654077085/N42721327A_1.jpg"/></div>]
[<div class="sc-8cbb8e24-2 eDXUdi"><img alt="عطر رويال أمبر EDP 100مل - v1515389873/N11202278A_1" aria-hidden="true" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_80/v1515389873/N11202278A_1.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="عطر رويال أمبر EDP 100مل - v1515389757/N11202278A_2" aria-hidden="true" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_80/v1515389757/N11202278A_2.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="عطر رويال أمبر EDP 100مل - v1560784604/N11202278A_3" aria-hidden="true" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_80/v1560784604/N11202278A_3.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="v1515389873/N11202278A_1" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_400/v1515389873/N11202278A_1.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="v1515389757/N11202278A_2" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_400/v1515389757/N11202278A_2.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="v1560784604/N11202278A_3" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_400/v1560784604/N11202278A_3.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="عطر رويال أمبر EDP 100مل" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_80/v1515389873/N11202278A_1.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="عطر بيور بلوم EDP 100مل" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_80/v1572605693/N31750079A_1.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="عطر رويال أمبر EDP 100مل" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_80/v1515389873/N11202278A_1.jpg"/></div>, <div class="sc-8cbb8e24-2 eDXUdi"><img alt="عطر بيور بلوم EDP 100مل" class="sc-b51db3f-1 iCVkuj" src="https://f.nooncdn.com/products/tr:n-t_80/v1572605693/N31750079A_1.jpg"/></div>]

但是，即使.find_all("div", class_="sc-8cbb8e24-2 eDXUdi") 给了我同样的结果。另外，获取img 标签不是更有用吗？（您可以使用.swiper-wrapper img 选择它们。）

您确定要获取正确的 html 吗？您的标头对我不起作用，而且我根本无法编写工作请求，因此我实际上使用 selenium 来获取 html。如果有兴趣，看看我的edited version of your code。我用我在一些爬虫中使用的可重用函数[经过一些调整]替换了你的 for 循环，输出看起来像

[
    {
        "page_number": 1,
        "page_link": "https://www.noon.com/egypt-ar/beauty-and-health/beauty/eg-nov22-clearance-2/?limit=50&page=1",
        "products_count": 182,
        "products": [
                {
                "product_id": "N39185122A",
                "product_brand": "إس. تي. ديبون",
                "product_name": "عطر جولدن وود EDP 100مل",
                "product_price_text": "رقم الموديل : 16904A قبل : 1298.00 جنيه الآن : 975.00 جنيه يشمل ضريبة القيمة المضافة وفّرت : 323.00 جنيه خصم 24٪",
                "current_price": "975.00",
                "product_link": "https://www.noon.com/egypt-ar/golden-wood-edp-100ml/N39185122A/p/?o=d55236b5f16d3c9d",
                "imageLinks_list": ["https://f.nooncdn.com/products/tr:n-t_80/v1666705361/N39185122A_1.jpg", "https://f.nooncdn.com/products/tr:n-t_80/v1618317832/N39185122A_2.jpg", "https://f.nooncdn.com/products/tr:n-t_80/v1618317832/N39185122A_3.jpg", "https://f.nooncdn.com/products/tr:n-t_80/v1617954175/N39185122A_5.jpg", "https://f.nooncdn.com/products/tr:n-t_80/v1617954175/N39185122A_4.jpg", "https://f.nooncdn.com/products/tr:n-t_400/v1666705361/N39185122A_1.jpg", "https://f.nooncdn.com/products/tr:n-t_400/v1618317832/N39185122A_2.jpg", "https://f.nooncdn.com/products/tr:n-t_400/v1618317832/N39185122A_3.jpg", "https://f.nooncdn.com/products/tr:n-t_400/v1617954175/N39185122A_5.jpg", "https://f.nooncdn.com/products/tr:n-t_400/v1617954175/N39185122A_4.jpg"]
            },
            {
                "product_id": "N11201003A",
                "product_brand": "جاكومو باريس",
                "product_name": "عطر سايلنسز EDP 100مل",
                "product_price_text": "4.3 12 تقييمات قبل : 811.00 جنيه الآن : 675.00 جنيه يشمل ضريبة القيمة المضافة وفّرت : 136.00 جنيه خصم 16٪",
                "current_price": "675.00",
                "product_link": "https://www.noon.com/egypt-ar/silences-edp-100ml/N11201003A/p/?o=b886ef68a4cc3e4c",
                "imageLinks_list": ["https://f.nooncdn.com/products/tr:n-t_80/v1562242787/N11201003A_1.jpg", "https://f.nooncdn.com/products/tr:n-t_80/v1562243187/N11201003A_2.jpg", "https://f.nooncdn.com/products/tr:n-t_400/v1562242787/N11201003A_1.jpg", "https://f.nooncdn.com/products/tr:n-t_400/v1562243187/N11201003A_2.jpg", "https://f.nooncdn.com/products/tr:n-t_80/v1562242787/N11201003A_1.jpg", "https://z.nooncdn.com/s/app/com/noon/icons/checkbox-square_checked_v2.svg", "https://z.nooncdn.com/s/app/com/noon/images/fulfilment_express_v2-ar.svg", "https://f.nooncdn.com/products/tr:n-t_80/v1654077085/N42721327A_1.jpg", "https://z.nooncdn.com/s/app/com/noon/icons/checkbox-square_checked_v2.svg", "https://z.nooncdn.com/s/app/com/noon/images/fulfilment_express_v2-ar.svg", "https://f.nooncdn.com/products/tr:n-t_80/v1562242787/N11201003A_1.jpg", "https://z.nooncdn.com/s/app/com/noon/icons/checkbox-square_checked_v2.svg", "https://z.nooncdn.com/s/app/com/noon/images/fulfilment_express_v2-ar.svg", "https://f.nooncdn.com/products/tr:n-t_80/v1654077085/N42721327A_1.jpg", "https://z.nooncdn.com/s/app/com/noon/icons/checkbox-square_checked_v2.svg", "https://z.nooncdn.com/s/app/com/noon/images/fulfilment_express_v2-ar.svg"]
            }
        ]
    },
    {
        "page_number": 2,
        "page_link": "https://www.noon.com/egypt-ar/beauty-and-health/beauty/eg-nov22-clearance-2/?limit=50&page=2",
        "products_count": 182,
        "products": [
                {
                "product_id": "N42677090A",
                "product_brand": "جاكومو",
                "product_name": "طَقم هدايا مكون من أو دي بارفان وبخاخ مناسب للسفر من قطعتين 100مل",
                "product_price_text": "رقم الموديل : 3392865093684 قبل : 1241.00 جنيه الآن : 679.00 جنيه يشمل ضريبة القيمة المضافة وفّرت : 562.00 جنيه خصم 45٪",
                "current_price": "679.00",
                "product_link": "https://www.noon.com/egypt-ar/2-piece-eau-de-parfum-and-travel-spray-gift-set-100ml/N42677090A/p/?o=d8dc699010d8086b",
                "imageLinks_list": ["https://f.nooncdn.com/products/tr:n-t_80/v1608984530/N42677090A_1.jpg", "https://f.nooncdn.com/products/tr:n-t_400/v1608984530/N42677090A_1.jpg"]
            },
            {
                "product_id": "N11200698A",
                "product_brand": "لانفن باريس",
                "product_name": "ماء تواليت إكلا داربيج للرجال 100مل",
                "product_price_text": "5.0 2 تقييمات قبل : 1365.00 جنيه الآن : 687.00 جنيه يشمل ضريبة القيمة المضافة وفّرت : 678.00 جنيه خصم 49٪",
                "current_price": "687.00",
                "product_link": "https://www.noon.com/egypt-ar/eclat-d-arpege-pour-homme-edt-100ml/N11200698A/p/?o=c5824e03ee50f26a",
                "imageLinks_list": ["https://f.nooncdn.com/products/tr:n-t_80/v1544679664/N11200698A_1.jpg", "https://f.nooncdn.com/products/tr:n-t_80/v1544679664/N11200698A_2.jpg", "https://f.nooncdn.com/products/tr:n-t_80/v1544679665/N11200698A_3.jpg", "https://f.nooncdn.com/products/tr:n-t_400/v1544679664/N11200698A_1.jpg", "https://f.nooncdn.com/products/tr:n-t_400/v1544679664/N11200698A_2.jpg", "https://f.nooncdn.com/products/tr:n-t_400/v1544679665/N11200698A_3.jpg"]
            }
        ]
    },
    {
        "page_number": 3,
        "page_link": "https://www.noon.com/egypt-ar/beauty-and-health/beauty/eg-nov22-clearance-2/?limit=50&page=3",
        "products_count": 182,
        "products": [
                {
                "product_id": "N39185122A",
                "product_brand": "إس. تي. ديبون",
                "product_name": "عطر جولدن وود EDP 100مل",
                "product_price_text": "رقم الموديل : 16904A قبل : 1298.00 جنيه الآن : 975.00 جنيه يشمل ضريبة القيمة المضافة وفّرت : 323.00 جنيه خصم 24٪",
                "current_price": "975.00",
                "product_link": "https://www.noon.com/egypt-ar/golden-wood-edp-100ml/N39185122A/p/?o=d55236b5f16d3c9d",
                "imageLinks_list": ["https://f.nooncdn.com/products/tr:n-t_80/v1666705361/N39185122A_1.jpg", "https://f.nooncdn.com/products/tr:n-t_80/v1618317832/N39185122A_2.jpg", "https://f.nooncdn.com/products/tr:n-t_80/v1618317832/N39185122A_3.jpg", "https://f.nooncdn.com/products/tr:n-t_80/v1617954175/N39185122A_5.jpg", "https://f.nooncdn.com/products/tr:n-t_80/v1617954175/N39185122A_4.jpg", "https://f.nooncdn.com/products/tr:n-t_400/v1666705361/N39185122A_1.jpg", "https://f.nooncdn.com/products/tr:n-t_400/v1618317832/N39185122A_2.jpg", "https://f.nooncdn.com/products/tr:n-t_400/v1618317832/N39185122A_3.jpg", "https://f.nooncdn.com/products/tr:n-t_400/v1617954175/N39185122A_5.jpg", "https://f.nooncdn.com/products/tr:n-t_400/v1617954175/N39185122A_4.jpg"]
            },
            {
                "product_id": "N11202278A",
                "product_brand": "إس. تي. ديبون",
                "product_name": "عطر رويال أمبر EDP 100مل",
                "product_price_text": "4.7 9 تقييمات قبل : 1899.00 جنيه الآن : 699.00 جنيه يشمل ضريبة القيمة المضافة وفّرت : 1200.00 جنيه خصم 63٪",
                "current_price": "699.00",
                "product_link": "https://www.noon.com/egypt-ar/royal-amber-edp-100ml/N11202278A/p/?o=dd94cf4150e7d47c",
                "imageLinks_list": ["https://f.nooncdn.com/products/tr:n-t_80/v1515389873/N11202278A_1.jpg", "https://f.nooncdn.com/products/tr:n-t_80/v1515389757/N11202278A_2.jpg", "https://f.nooncdn.com/products/tr:n-t_80/v1560784604/N11202278A_3.jpg", "https://f.nooncdn.com/products/tr:n-t_400/v1515389873/N11202278A_1.jpg", "https://f.nooncdn.com/products/tr:n-t_400/v1515389757/N11202278A_2.jpg", "https://f.nooncdn.com/products/tr:n-t_400/v1560784604/N11202278A_3.jpg", "https://f.nooncdn.com/products/tr:n-t_80/v1515389873/N11202278A_1.jpg", "https://z.nooncdn.com/s/app/com/noon/icons/checkbox-square_checked_v2.svg", "https://z.nooncdn.com/s/app/com/noon/images/fulfilment_express_v2-ar.svg", "https://f.nooncdn.com/products/tr:n-t_80/v1572605693/N31750079A_1.jpg", "https://z.nooncdn.com/s/app/com/noon/icons/checkbox-square_checked_v2.svg", "https://z.nooncdn.com/s/app/com/noon/images/fulfilment_express_v2-ar.svg", "https://f.nooncdn.com/products/tr:n-t_80/v1515389873/N11202278A_1.jpg", "https://z.nooncdn.com/s/app/com/noon/icons/checkbox-square_checked_v2.svg", "https://z.nooncdn.com/s/app/com/noon/images/fulfilment_express_v2-ar.svg", "https://f.nooncdn.com/products/tr:n-t_80/v1572605693/N31750079A_1.jpg", "https://z.nooncdn.com/s/app/com/noon/icons/checkbox-square_checked_v2.svg", "https://z.nooncdn.com/s/app/com/noon/images/fulfilment_express_v2-ar.svg"]
            }
        ]
    },
    {
        "page_number": 4,
        "page_link": "https://www.noon.com/egypt-ar/beauty-and-health/beauty/eg-nov22-clearance-2/?limit=50&page=4",
        "products_count": 182,
        "products": [
                {
                "product_id": "N31126667A",
                "product_brand": "فيكتور أند رولف",
                "product_name": "ماء عطر فلاور بومب ميدنايبت 100مل",
                "product_price_text": "رقم الموديل : 10507A 4.5 2 تقييمات قبل : 2999.00 جنيه الآن : 2395.00 جنيه يشمل ضريبة القيمة المضافة وفّرت : 604.00 جنيه خصم 20٪",
                "current_price": "2395.00",
                "product_link": "https://www.noon.com/egypt-ar/flowerbomb-midnight-edp-100ml/N31126667A/p/?o=e34956fccd70220b",
                "imageLinks_list": ["https://f.nooncdn.com/products/tr:n-t_80/v1572005046/N31126667A_1.jpg", "https://f.nooncdn.com/products/tr:n-t_80/v1572005047/N31126667A_2.jpg", "https://f.nooncdn.com/products/tr:n-t_400/v1572005046/N31126667A_1.jpg", "https://f.nooncdn.com/products/tr:n-t_400/v1572005047/N31126667A_2.jpg"]
            },
            {
                "product_id": "N42264463A",
                "product_brand": "كييلز",
                "product_name": "محلول تصحيح لعلاج البقع الداكنة بوضوح 30مل",
                "product_price_text": "رقم الموديل : 3605970202637 5.0 تقييم عميل واحد قبل : 2678.57 جنيه الآن : 1715.00 جنيه يشمل ضريبة القيمة المضافة وفّرت : 963.57 جنيه خصم 35٪",
                "current_price": "1715.00",
                "product_link": "https://www.noon.com/egypt-ar/clearly-corrective-dark-spot-solution-30ml/N42264463A/p/?o=bba8c04cdb7cce0e",
                "imageLinks_list": ["https://f.nooncdn.com/products/tr:n-t_80/v1605337673/N42264463A_5.jpg", "https://f.nooncdn.com/products/tr:n-t_400/v1605337673/N42264463A_5.jpg"]
            }
        ]
    }
]

（每页仅包含 2 个产品）

【讨论】：