如何将excel数据解析为特定格式的数据框？答案

【问题标题】：How to parse excel data into dataframe with particular format?如何将excel数据解析为特定格式的数据框？
【发布时间】：2021-12-03 14:07:32
【问题描述】：

我有一个这样的 Excel，在 Excel 的“元素”表中，如下所示：https://i.stack.imgur.com/pT0PY.png

Item              Category  Value
TRANSPORT         A         1
Bus               A         2
Car               A         3
Automobile        A         4
Bike              A         5
ACCOMODATION      A         6
House             A         7
Apartment         A         8
DELIVERY          B         9
Glovo             B         10
Emag              B         11
Transporter       B         12
ACCOMODATION      B         13
Apartment1        B         14
Apartment2        B         15
ACCOMODATION      C         16
Rental            C         17
Apartment         C         18

我想将元素（公共汽车、汽车、汽车、自行车...）中的项目（运输、住宿、送货）分离到如下数据框中：

      Element          Item Category  Value
0         Bus     TRANSPORT        A      2
1         Car     TRANSPORT        A      3
2  Automobile     TRANSPORT        A      4
3        Bike     TRANSPORT        A      5
4       House  ACCOMODATION        A      7
5   Apartment  ACCOMODATION        A      8

我已经设法编写代码来将元素与类别 A 分开，但是当我将其用于类别 B 或 C 或其他类别时，代码会以某种方式中断。它触发：IndexError: index 8 is out of bounds for axis 0 with size 7，因为代码中的索引被修剪。我只需要为元素提取值列的值，而不是为项目提取值，并且由于长度不匹配而中断。我需要最终的数据框来包含 Excel 中所有类别的所有信息，而不仅仅是一个类别。到目前为止我尝试过的（仅适用于 A 类）：

import pandas as pd
import numpy as np

df = pd.read_excel('elements.xlsx',
                   ['Elements'], engine='openpyxl')

category_names = df['Elements']['Category'].unique()

df['Elements'] = df['Elements'].groupby(['Category'])

categ_group = ['TRANSPORT', 'ACCOMODATION', 'DELIVERY']


def create_category_df(category_name='A'):
    helper_df = df['Elements'].get_group(category_name)

    # get index for items
    item_index = helper_df[helper_df["Item"].isin(categ_group)].index.to_list()

    # get elements and associated items
    item_data = np.split(helper_df['Item'].to_numpy(), item_index)

    helper_df = helper_df.drop(helper_df.index[item_index])  # drop rows for items

    helper_df = helper_df.reset_index(drop=True)

    resulted_df = pd.DataFrame(columns=['Element', 'Item', 'Category', 'Value'])

    item_list = []
    for index in range(len(item_data)):
        if item_data[index].size != 0:
            resulted_df = resulted_df.append(pd.DataFrame(item_data[index][1:], columns=['Element']))
            item_list += len(item_data[index][1:]) * [
                item_data[index][0]]  # multiply items by number of times it is present and add it to df

    resulted_df['Category'] = category_name  # 'Hardware EA'
    resulted_df['Item'] = item_list
    resulted_df['Value'] = helper_df['Value'].values
    resulted_df = resulted_df.reset_index(drop=True)
    print(resulted_df.to_string())
    return resulted_df


create_category_df()

【问题讨论】：

标签： python excel pandas dataframe group-by

【解决方案1】：

首先替换列名称，然后用Series.where 中的NaNs 替换列表中不匹配的值，因此可能在DataFrame.insert 中为第二个新列前向填充缺失值，如果在两个列中的值相等，则最后删除行boolean indexing:

categ_group = ['TRANSPORT', 'ACCOMODATION', 'DELIVERY']

df = df.rename(columns={'Item':'Element'})
df.insert(1, 'Item', df['Element'].where(df['Element'].isin(categ_group)).ffill())
df =df[ df['Element'].ne(df['Item'])]

print (df)
        Element          Item Category  Value
1           Bus     TRANSPORT        A      2
2           Car     TRANSPORT        A      3
3    Automobile     TRANSPORT        A      4
4          Bike     TRANSPORT        A      5
6         House  ACCOMODATION        A      7
7     Apartment  ACCOMODATION        A      8
9         Glovo      DELIVERY        B     10
10         Emag      DELIVERY        B     11
11  Transporter      DELIVERY        B     12
13   Apartment1  ACCOMODATION        B     14
14   Apartment2  ACCOMODATION        B     15
16       Rental  ACCOMODATION        C     17
17    Apartment  ACCOMODATION        C     18

【讨论】：