如何检查熊猫数据框组是否具有相同的数据

Question

我有一个如下的熊猫数据框

id  name  Base   field1    field2           field3
1   AA     Y      Yes      Consumer         Not Applicable 
1   BB     N      Yes      Consumer         Not Applicable 
2   CC     Y      Yes      Consumer         Not Applicable 
2   DD     N      Yes      Not Applicable   Not Applicable 
2   EE     N      No       Not Applicable   Modified
3   FF     Y      Yes      Not Applicable   Applicable 
3   GG     N      Yes      Not Applicable   Not Applicable 
3   HH     N      Yes      Not Applicable   Not Applicable 

预期的结果是根据ID列对这个dataframe进行分组，并检查其他所有列上的数据是否是每个组中的相同数据，最后写入结果。

我试过这个来验证每个组的数据，但它总是说 TRUE

代码：

result_list=[]
for col in df.columns:
        result = df.groupby(level=0)[col].apply(lambda x: len(set(x))==1)
        result_list.append(result)

final = pd.concat(result_list,1)

预期结果是

id  name     field1   field2           field3           Error
1   AA       Yes      Consumer         Not Applicable   Pass 
1   BB       Yes      Consumer         Not Applicable   Pass
2   CC       Yes      Consumer         Not Applicable   field1, field2, field3 mismatch for ID: 2
2   DD       Yes      Not Applicable   Not Applicable   field1, field2, field3 mismatch for ID: 2
2   EE       No       Not Applicable   Modified         field1, field2, field3 mismatch for ID: 2
3   FF       Yes      Not Applicable   Applicable       field3 mismatch for ID: 3
3   GG       Yes      Not Applicable   Not Applicable   field3 mismatch for ID: 3
3   HH       Yes      Not Applicable   Not Applicable   field3 mismatch for ID: 3

有什么帮助吗？

Answer 1

你可能会得到你想要的代码（假设df有名为id的索引）：

def handler(df):
    for col in ['field1', 'field2', 'field3']:
        if df.loc[:, col].nunique() > 1:
            return 'error in {} for id {}'.format(col, df.index[0])
    else:
        return 'pass'

result = df.groupby(level=0).apply(handler)
result = df.reset_index().merge(result.to_frame().reset_index(), on='id')

result 是：

   id name field1          field2          field3                         0
0   1   AA    Yes        Consumer  Not Applicable                      pass
1   1   BB    Yes        Consumer  Not Applicable                      pass
2   2   CC    Yes        Consumer  Not Applicable  error in field1 for id 2
3   2   DD    Yes  Not Applicable  Not Applicable  error in field1 for id 2
4   2   EE     No  Not Applicable        Modified  error in field1 for id 2
5   3   FF    Yes  Not Applicable      Applicable  error in field3 for id 3
6   3   GG    Yes  Not Applicable  Not Applicable  error in field3 for id 3
7   3   HH    Yes  Not Applicable  Not Applicable  error in field3 for id 3

EDIT - 处理程序中的次要版本

def handler(df):
    cols = list()
    for col in ['field1', 'field2', 'field3']:
        if df.loc[:, col].nunique() > 1:
            cols.append(col)
    if cols:
        return 'error in {} for id {}'.format(', '.join(cols), df.index[0])
    else:
        return 'pass'

Answer 2

您可以groupby id 然后agg 每列计算每个组的unique 值的数量，然后您就知道该数字大于 1 是错误的：

df[df.columns.drop('name')].groupby('id').agg(lambda x: len(x.unique()))>1

有了这个输出，你可以根据它来构造你的字符串。

    field1  field2  field3
id          
1   False   False   False
2   True    True    True
3   False   False   True