如何从网络抓取的 2 个列表中制作字典答案

【问题标题】：How to make a dictionary from 2 lists from web scraping如何从网络抓取的 2 个列表中制作字典
【发布时间】：2022-11-28 15:03:59
【问题描述】：

我想通过网络抓取此页面制作一个数据框：https://www.airlinequality.com/airline-reviews/british-airways。

我拥有的价值是来自乘客的评论和乘客给出的评分，但我不知道如何让它成为数据框

这是我的代码：

import requests
from bs4 import BeautifulSoup
import pandas as pd

base_url = "https://www.airlinequality.com/airline-reviews/british-airways"
pages = 5 #10
page_size = 1 #100

reviews = []
aircraft = []
seat_type = []
route = []
recommended = []
rating = []
category = []

for i in range(1, pages + 1):

    print(f"Scraping page {i}")

    # Create URL to collect links from paginated data
    url = f"{base_url}/page/{i}/?sortby=post_date%3ADesc&pagesize={page_size}"

    # Collect HTML data from this page
    response = requests.get(url)

    # Parse content
    content = response.content
    parsed_content = BeautifulSoup(content, 'html.parser')
    for para in parsed_content.find_all("div", {"class": "text_content"}):
        reviews.append(para.get_text())
        
    for para2 in parsed_content.find_all("div", {"class" : "review-stats"}):
        for para3 in para2.find_all('td',{'class' : 'review-value'}):
            rating.append(para3.get_text())
        recomend = rating[-1]
        rating = rating[:-1]
        for para4 in para2.find_all('td',{'class' : 'review-rating-stars stars'}):
            para5 = len(para4.find_all('span', {'class' : 'star fill'}))
            rating.append(para5)
        rating.append(recomend)
        #print(rating)
        for para6 in para2.find_all('td',{'class' : 'review-rating-header'}):
            category.append(para6.get_text())
        #print(category)
        
    print(f"   ---> {len(reviews)} total reviews")

我得到的输出：

我想要什么（示例）：

【问题讨论】：

使用pd.Dataframe() 并传递其中的数据。
我试过了，刚刚创建了两列，我输入了你的输入结果

标签： python dataframe web-scraping beautifulsoup

【解决方案1】：

你有列值，只需构建DataFrame

例如。，

from pandas import DataFrame

category = ["Aircraft", 'Type of Traveller', 'Seat Type']
rating = ['A320', 'Solo', 'Business Class']

# Create the records from both list, using zip and dict calls.
data_dict = dict(zip(category, rating))

# Build the dataframe from the dictionary.
df = DataFrame.from_records(data_dict, columns=category, index=[0])

print(df)

看起来像这样。

【讨论】：