通过多个页面进行网页抓取不会保存每个结果 -beautifulsoup答案

【问题标题】：Web scraping through multiple pages doesnt save each result -beautifulsoup通过多个页面进行网页抓取不会保存每个结果 -beautifulsoup
【发布时间】：2021-03-18 05:11:56
【问题描述】：

我的问题是，它会遍历页面，但它不会在我的列表中写入任何内容。

最后我打印len(title)，它仍然是0。

from bs4 import BeautifulSoup
import requests

for page in range(20, 200, 20):
    current_page = 'https://auto.bazos.sk/{}/?hledat=kolesa&hlokalita=&humkreis=&cen'.format(page)
    web_req = requests.get(current_page).text
    soup = BeautifulSoup(requests.get(current_page).content, 'html.parser')

    title_data = soup.select('.nadpis')
    title = []
    for each_title in title_data:
        title.append(each_title.text)
    print(current_page)

print(len(title))

【问题讨论】：

打印 each_title.text 会发生什么？如果这是预期的文本内容，您的 .append 方法可能有问题。如果没有，请尝试打印其来源等以找到根本原因

标签： python web web-scraping beautifulsoup

【解决方案1】：

将title 移出loop 即可。

import requests
from bs4 import BeautifulSoup

title = []
for page in range(20, 40, 20):
    current_page = 'https://auto.bazos.sk/{}/?hledat=kolesa&hlokalita=&humkreis=&cen'.format(page)
    soup = BeautifulSoup(requests.get(current_page).content, 'html.parser')
    title_data = soup.select('.nadpis')
    for each_title in title_data:
        title.append(each_title.text)
    print(current_page)
print(title)

输出：

['ELEKTRONY SKODA OCTAVIA SCOUT DISKY “PROTEUS” R17', 'Fiat Sedici 1.6, 4x4, r.v 04/2009, 79 kw, slovenské ŠPZ', 'Bmw e46 328ci', '255/50 R19', 'Honda Jazz 1.3', 'Predám 4 ks kolesá', 'Audi A5 3.2 FSI quattro tiptronic  S LINE R20  TOP STAV', 'Peugeot 407 combi 1,6 hdi', 'Škoda Superb 2.0TDI 4x4 od 260€ mesačne, bez akontácia', 'Predam elektrony Audi 5x112 R17 a letne pneu', 'ROZPREDÁM MAZDA 3 2.0i 110kW NA NÁHRADNÉ DIELY', 'Predám Astra j Turbo Noblesse bronz', 'ŠKODA KAROQ  1.6 TDI - full výbava', 'VW CHICAGO 5x112 + letné pneu 215/40 R18', 'Fiat 500 SPORT 1.3 multijet 70kw', 'Volvo FL280 - TROJSTRANNÝ SKLÁPAČ + HYDRAULICKÁ RUKA', 'ŠKODA SUPERB COMBI 2.0 TDI 190K 4X4 L&K DSG', 'FORD FOCUS 2.0 TDCI TITANIUM', 'FORD EDGE 2.0 TDCi - 154 kW  VIGNALE : 27.000 km', 'R18 5x112 originalne Vw Seat Audi Skoda']

【讨论】：