如何根据熊猫中的多个查找条件转置数据 - python答案

【问题标题】：How to transpose data based on multiple lookup criterias in pandas - python如何根据熊猫中的多个查找条件转置数据 - python
【发布时间】：2022-01-26 04:33:54
【问题描述】：

我是一名工资单专家，经常遇到非常奇怪的报告，其中一名员工的姓名和身份证号显示在多行（A 列和 B 列）中，并且相应的数据分布在许多列之间。示例：

id#, Name, PTO Code, Accrued Amount $, Accrued Hours, Used Amount, Used Hours, LeftAmount, LeftHours
101, Empl1, NY Sick,         0,            112,          0,         56,          0,           56
101, Empl1, Plan1, TO Am,    3600,          0,         1500,          0,         2100,        0               
101, Empl1, Plan1, PTO Hrs,   0,           240,          0,         100,           0,         140
101, Empl1, Plan2, PTO Am,   6000,          0,         6000,         0,            0,          0
101, Empl1, Plan2, PTO Hrs,    0,         400,           0,           400,          0,         0
201, Empl2,

等等……

这种报道让人头疼…… 我写了一个组织数据的代码（看到它请不要笑）。这是输出：

id#, Name, NYC Sick,       NYC Sick     NYC Sick    Plan1 PTO Am  Plan1 PTOAm  Plan1 PTOAm  Plan1PTO 
           Hours Accrued  Hours Used   Hours Left      Accrued       Used         Left    HrsAccrued
101  Empl1    112              56          56            3600         1500         2100       240
201  Empl2

等等……

我的目标已经实现，但如果能看到一些可以执行相同任务的一流（干）代码，那就太好了。请在下面查看我的代码。

import pandas as pd
df = pd.read_excel('PTO report.xlsx')

创建一个新的数据框；只留下 ID 列和 Name 列；摆脱重复值。将是一个输出文件

 df_new = df[['ID#', 'Name']].drop_duplicates()

添加新列

df_new[[
    'NYC Sick Hours Accrued', 'NYC Sick Hours Used', 
    'NYC Sick Hours Left', 'Plan1 PTO Amount Accrued', 
    'Plan1 PTO Amount Used', 'Plan1 PTO Amount Left', 'Plan1 PTO Hours Accrued', 
    'Plan1 PTO Hours Used', 'Plan1 PTO Hours Left', 'Plan2 PTO Amount Accrued', 
    'Plan2 PTO Amount Used', 'Plan2 PTO Amount Left', 'Plan2 PTO Hours Accrued', 
    'Plan2 PTO Hours Used', 'Plan2 PTO Hours Left']] = 0

遍历每个员工并执行类似于 vlookup 操作。在我看来，这部分代码必须改进。

for i in df_new['ID#']:

    #NY Sick Hours Accrued ****************************************
    filt = (df['PTO Code'] == 'NYC Sick Hours') & (df['ID#'] == i )
    filt2 = df_new['ID#'] == i 
    df_new.loc[filt2, 'NYC Sick Hours Accrued'] = int(df.loc[filt,'Accrued Hours'])
       
    #NY Sick Hours Used
    filt = (df['PTO Code'] == 'NYC Sick Hours') & (df['ID#'] == i)
    filt2 = df_new['ID#'] == i 
    df_new.loc[filt2, 'NYC Sick Hours Used'] = int(df.loc[filt,'Used Hours'])
    
    #NY Sick Hours Left
    filt = (df['PTO Code'] == 'NYC Sick Hours') & (df['ID#'] == i)
    filt2 = df_new['ID#'] == i 
    df_new.loc[filt2, 'NYC Sick Hours Left'] = int(df.loc[filt,'Ending Balance Hours'])
        
    #Plan1 PTO Amount Accrued *************************************
    filt = (df['PTO Code'] == 'Plan1 PTO Amount') & (df['ID#'] == i)
    filt2 = df_new['ID#'] == i 
    df_new.loc[filt2, 'Plan1 PTO Amount Accrued'] = int(df.loc[filt,'Accrued Amount $'])
    
    #Plan1 PTO Amount Used
    filt = (df['PTO Code'] == 'Plan1 PTO Amount') & (df['ID#'] == i)
    filt2 = df_new['ID#'] == i 
    df_new.loc[filt2, 'Plan1 PTO Amount Used'] = int(df.loc[filt,'Used Amount $'])
    
    #Plan1 PTO Amount Left 
    filt = (df['PTO Code'] == 'Plan1 PTO Amount') & (df['ID#'] == i)
    filt2 = df_new['ID#'] == i 
    df_new.loc[filt2, 'Plan1 PTO Amount Left'] = int(df.loc[filt,'Ending Balance Amount $'])
    
    
    #Plan1 PTO Hours Accrued 
    filt = (df['PTO Code'] == 'Plan1 PTO Hours') & (df['ID#'] == i)
    filt2 = df_new['ID#'] == i 
    df_new.loc[filt2, 'Plan1 PTO Hours Accrued'] = int(df.loc[filt,'Accrued Hours']) 
    
    #Plan1 PTO Hours Used
    filt = (df['PTO Code'] == 'Plan1 PTO Hours') & (df['ID#'] == i)
    filt2 = df_new['ID#'] == i 
    df_new.loc[filt2, 'Plan1 PTO Hours Used'] = int(df.loc[filt,'Used Hours']) 
    
    #Plan1 PTO Hours Left
    filt = (df['PTO Code'] == 'Plan1 PTO Hours') & (df['ID#'] == i)
    filt2 = df_new['ID#'] == i 
    df_new.loc[filt2, 'Plan1 PTO Hours Left'] = int(df.loc[filt,'Ending Balance Hours'])    

       
   #Plan2 PTO Amount Accrued **************************************
    filt = (df['PTO Code'] == 'Plan2 PTO Amount') & (df['ID#'] == i)
    filt2 = df_new['ID#'] == i 
    df_new.loc[filt2, 'Plan2 PTO Amount Accrued'] = int(df.loc[filt,'Accrued Amount $'])
    
    #Plan2 PTO Amount Used
    filt = (df['PTO Code'] == 'Plan2 PTO Amount') & (df['ID#'] == i)
    filt2 = df_new['ID#'] == i 
    df_new.loc[filt2, 'Plan2 PTO Amount Used'] = int(df.loc[filt,'Used Amount $'])
    
    #Plan2 PTO Amount Left 
    filt = (df['PTO Code'] == 'Plan2 PTO Amount') & (df['ID#'] == i)
    filt2 = df_new['ID#'] == i 
    df_new.loc[filt2, 'Plan2 PTO Amount Left'] = int(df.loc[filt,'Ending Balance Amount $'])
     
    #Plan2 PTO Hours Accrued 
    filt = (df['PTO Code'] == 'Plan2 PTO Hours') & (df['ID#'] == i)
    filt2 = df_new['ID#'] == i 
    df_new.loc[filt2, 'Plan2 PTO Hours Accrued'] = int(df.loc[filt,'Accrued Hours'])
    
    #Plan2 PTO Hours Used
    filt = (df['PTO Code'] == 'Plan2 PTO Hours') & (df['ID#'] == i)
    filt2 = df_new['ID#'] == i 
    df_new.loc[filt2, 'Plan2 PTO Hours Used'] = int(df.loc[filt,'Used Hours'])
    
    #Plan2 PTO Hours Left
    filt = (df['PTO Code'] == 'Plan2 PTO Hours') & (df['ID#'] == i)
    filt2 = df_new['ID#'] == i 
    df_new.loc[filt2, 'Plan2 PTO Hours Left'] = int(df.loc[filt,'Ending Balance Hours'])

导出为新文件

df_new.to_excel('PTO report transposed.xlsx', index = False)

我在 VBA excel 中编写了一个宏来完成同样的工作。我使用了一个类对象而不是“vlookup”。必须有一个简单的解决方案。这是我的第一篇文章，所以如果我的问题不清楚或标题错误，请告诉我。

感谢您的宝贵时间！

【问题讨论】：

请提供字典形式的示例代码。你可以通过df.sample(10).to_dict())
@serge 感谢您为第一个问题所做的努力。从读取文件到保存，您不需要发布所有代码，这与您的问题直接相关；)

标签： python pandas

【解决方案1】：

使用DataFrame.set_index 和DataFrame.stack 重塑为MultiIndex Series，然后使用Series.unstack 删除0 行，最后展平MultiIndex in columns：

s = df.set_index(['id#','Name','PTO Code']).stack()
df1 = s[s.ne(0)].unstack([-2,-1])
#alternatives if duplicates
#df1 = s[s.ne(0)].groupby(level=[0,1,2,3]).sum().unstack([-2,-1])
df1.columns = df1.columns.map(lambda x: f'{x[0]} {x[1]}')
df1 = df1.reset_index()

print (df1)
   id#   Name  NY Sick Accrued Hours  NY Sick Used Hours  NY Sick LeftHours  \
0  101  Empl1                    112                  56                 56   

   Plan1 TO Am Accrued Amount $  Plan1 TO Am Used Amount  \
0                          3600                     1500   

   Plan1 TO Am LeftAmount  Plan1 PTO Hrs Accrued Hours  \
0                    2100                          240   

   Plan1 PTO Hrs Used Hours  Plan1 PTO Hrs LeftHours  \
0                       100                      140   

   Plan2 PTO Am Accrued Amount $  Plan2 PTO Am Used Amount  \
0                           6000                      6000   

   Plan2 PTO Hrs Accrued Hours  Plan2 PTO Hrs Used Hours  
0                          400                       400

【讨论】：

感谢 JEZRAEL 的帮助！这正是我想要的。只需 4 行代码就可以实现相同的结果。太奇妙了！！！感谢您的宝贵时间！
@serge - 欢迎您。如果我的回答有帮助，请不要忘记accept。谢谢。

【解决方案2】：

您想要的内容并不完全清楚，尤其是您的列中存在潜在错误（PTO 列中是否有逗号？）。

无论如何，假设 PTO 代码是单列，这本质上是 pivot 并删除了零/NaN

(df.replace(0, float('nan'))
   .pivot(index=['id#','Name'], columns='PTO Code')
   .dropna(how='all', axis=1)
 )

输出：

          Accrued Amount $              Accrued Hours                            Used Amount              Used Hours                             LeftAmount LeftHours             
PTO Code       Plan1 TO Am Plan2 PTO Am       NY Sick Plan PTO Hrs Plan2 PTO Hrs Plan1 TO Am Plan2 PTO Am    NY Sick Plan PTO Hrs Plan2 PTO Hrs Plan1 TO Am   NY Sick Plan PTO Hrs
id# Name                                                                                                                                                                          
101 Empl1           3600.0       6000.0         112.0        240.0         400.0      1500.0       6000.0       56.0        100.0         400.0      2100.0      56.0        140.0
201 Empl2              NaN          NaN           NaN          NaN           NaN         NaN          NaN        NaN          NaN           NaN         NaN       NaN          NaN

【讨论】：

感谢 MOZWAY 的帮助！我以前从未使用过 .pivot() 。这是一个很好的方法。感谢您的宝贵时间！