他们向我显示错误“所有数组必须具有相同的长度”答案

【问题标题】：They show me the error 'All arrays must be of the same length'他们向我显示错误“所有数组必须具有相同的长度”
【发布时间】：2021-09-13 11:02:27
【问题描述】：

我尝试了很多方法，但他们都无法解决他们向我显示错误：

所有数组的长度必须相同

from bs4 import BeautifulSoup

import requests
import pandas as pd

review = []
ratings = []
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101 Firefox/66.0",
    "Accept-Encoding": "gzip, deflate",
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
    "DNT": "1",
    "Connection": "close",
    "Upgrade-Insecure-Requests": "1",
}
for page in range(1, 5):
    r = requests.get(
        "https://www.amazon.com/s?k=redmi&page=2&qid=1631528810&ref=sr_pg_={page}".format(
            page=page
        ),
        headers=headers,
    )
    soup = BeautifulSoup(r.content, "lxml")
    for d in soup.findAll("div", attrs={"class": "s-result-item"}):
        rating = d.find("span", attrs={"class": "a-icon-alt"})
        if rating is not None:
            ratings.append(rating.text)

        reviews = d.find("span", class_="a-size-base")
        if reviews is not None:
            review.append(reviews.text)


df = pd.DataFrame({"rating": ratings, "reviews": review})
df.to_csv("products .csv", index=False, encoding="utf-8")

【问题讨论】：

我想你的ratings 和reviews 的长度不一样。您需要修复抓取代码以考虑到这一点。

标签： python web-scraping beautifulsoup python-requests

【解决方案1】：

ratings 和 reviews 的长度不同，并且您刮错了容器。我做了必要的修改，现在它应该可以工作了：

from bs4 import BeautifulSoup

import requests
import pandas as pd

review = []
ratings = []
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101 Firefox/66.0",
    "Accept-Encoding": "gzip, deflate",
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
    "DNT": "1",
    "Connection": "close",
    "Upgrade-Insecure-Requests": "1",
}
for page in range(1, 5):
    cookies = {'session': '17ab96bd8ffbe8ca58a78657a918558'}
    r = requests.get(
        "https://www.amazon.com/s?k=redmi&page=2&qid=1631528810&ref=sr_pg_={page}".format(
            page=page
        ),
        headers=headers,
        cookies =cookies
    )
    soup = BeautifulSoup(r.content, "lxml")
    for d in soup.select(".s-result-item[data-component-type='s-search-result']"):
        rating = d.find("span", attrs={"class": "a-icon-alt"})
        if rating is not None:
            ratings.append(rating.text)
        else:
            ratings.append("-")

        reviews = d.find("span", class_="a-size-base")
        if reviews is not None and rating is not None:
            review.append(reviews.text)
        else:
            review.append("-")


df = pd.DataFrame({"rating": ratings, "reviews": review})
df.to_csv("products.csv", index=False, encoding="utf-8")

【讨论】：

他们现在给了我一个空的 csv 文件
这是因为亚马逊在连续发送请求时阻止了您。您可以通过为request.get() 方法提供cookie 来解决此问题。我相应地更新了代码，请试一试。