python pandas groupby 带有二进制类的多个组答案

【问题标题】：python pandas groupby multiple groups with binary classpython pandas groupby 带有二进制类的多个组
【发布时间】：2015-12-30 22:22:10
【问题描述】：

我有一个如下的DataFrame：

id class
A   1
B   1
C   0 
D   0
E   1
F   1

我想把它分成 3 组，G1:A,B, G2:C,D, G3:E,F。有没有办法通过循环遍历所有行来为每个 id 分配一个新类？

【问题讨论】：

您能否发布所需的输出以便我们更好地理解您的问题？

标签： python pandas group-by data-analysis

【解决方案1】：

遍历'class'，每次class和上一个不一样的时候开始一个新的group，例如：

创建 DF：

import pandas as pd
df = pd.DataFrame()
df['id'] = ['a','b','c','d','e','f']
df['class'] = [1,1,0,0,1,1]

遍历“类”以创建组索引：

group_index = [0]
for i in df.index[1:]:
    if df['class'][i]==df['class'][i-1]:
        group_index.append(group_index[-1])
    else:
        group_index.append(group_index[-1]+1)

将 group_index 添加到 DF：

df['group_index'] = group_index

输出应该是：

    id  class   group_index
  0 a     1        0
  1 b     1        0
  2 c     0        1
  3 d     0        1
  4 e     1        2
  5 f     1        2

【讨论】：

【解决方案2】：

这是一个单行代码。 :p 它利用相邻行的差分信息和累积求和来为每一行分配组ID。

>>> df = pd.DataFrame({'id': ['A','B','C','D','E','F'],
                       'class': [1, 1, 0, 0, 1, 1]},
                       columns=['id', 'class'])

>>> pd.concat([df, pd.Series(map(lambda x: 1 if abs(x) > 0 else 0,
df['class'].diff().fillna(0)), name='groupid').cumsum()], axis=1)

  id  class  groupid
0  A      1        0
1  B      1        0
2  C      0        1
3  D      0        1
4  E      1        2
5  F      1        2

现在，您可以使用 groupby() 来获取 groupy 对象。

>>> g = pd.concat([df, pd.Series(map(lambda x: 1 if abs(x) > 0 else 0,
df['class'].diff().fillna(0)), name='groupid').cumsum()], axis=1).groupby('groupid')

>>> for index, group_df in g:
        print(group_df)

  id  class  groupid
0  A      1        0
1  B      1        0
  id  class  groupid
2  C      0        1
3  D      0        1
  id  class  groupid
4  E      1        2
5  F      1        2

附上完整代码。

import pandas as pd

def groupby_binaryflag(df, key='class'):
    return pd.concat([df,
                      pd.Series(map(lambda x: 1
                                    if abs(x) > 0
                                    else 0, df['class'].diff().fillna(0)),
                                name='groupid').cumsum()], axis=1).groupby('groupid')

if __name__ == '__main__':
    df1 = pd.DataFrame({'id': ['A','B','C','D','E','F'],
                        'class': [1, 1, 0, 0, 1, 1]}, columns=['id', 'class'])

    df2 = pd.DataFrame({'id': ['A','B','C','D','E','F', 'G', 'H', 'I', 'J', 'K', 'L'],
                        'class': [1, 1, 0, 0, 1, 1, 0, 0, 0, 1, 1, 1]}, columns=['id', 'class'])

    for df in [df1, df2]:
        for index, group_df in groupby_binaryflag(df):
            print(group_df)
        print("=====\n")

输出：

  id  class  groupid
0  A      1        0
1  B      1        0
  id  class  groupid
2  C      0        1
3  D      0        1
  id  class  groupid
4  E      1        2
5  F      1        2
=====

  id  class  groupid
0  A      1        0
1  B      1        0
  id  class  groupid
2  C      0        1
3  D      0        1
  id  class  groupid
4  E      1        2
5  F      1        2
  id  class  groupid
6  G      0        3
7  H      0        3
8  I      0        3
   id  class  groupid
9   J      1        4
10  K      1        4
11  L      1        4
=====

【讨论】：

【解决方案3】：

您可以使用diff、astype 和cumsum：

print df
    id  class
0    A      0
1    B      1
2   B1      1
3    C      0
4    D      0
5    E      1
6    F      1
7   F1      1
8    G      0
9    H      0
10   I      1
11   J      1

df['count'] = (df['class'].diff(1) != 0).astype('int').cumsum()
print df

    id  class  count
0    A      0      1
1    B      1      2
2   B1      1      2
3    C      0      3
4    D      0      3
5    E      1      4
6    F      1      4
7   F1      1      4
8    G      0      5
9    H      0      5
10   I      1      6
11   J      1      6

for name,  group in df.groupby('count'):
    print name
    print group[['id', 'class']]

测试性能：

这些时间将在很大程度上取决于 df 的大小以及0 和1 的数量（和位置）：

import pandas as pd

df = pd.DataFrame({'id': ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J'], 'class': [0, 1, 1, 0, 0, 1, 1, 1, 0, 0]}, columns=['id', 'class'])

#uncomment for test len(df) = 1000
#df =  pd.concat([df]*1000).reset_index(drop=True)

def jez(df):
    df['count'] = (df['class'].diff(1) != 0).astype('int').cumsum()
    return df

def eze(df):
    group_index = [0]
    for i in df.index[1:]:
        if df['class'][i]==df['class'][i-1]:
            group_index.append(group_index[-1])
        else:
            group_index.append(group_index[-1]+1)

    df['group_index'] = group_index        
    return df

def sy2(df):
    df = pd.concat([df, pd.Series(map(lambda x: 1 if abs(x) > 0 else 0, df['class'].diff().fillna(0)), name='groupid').cumsum()], axis=1)
    return df

print jez(df)
print eze(df)
print sy2(df)

测试len(df) = 10:

In [28]: %timeit jez(df)
The slowest run took 5.08 times longer than the fastest. This could mean that an intermediate result is being cached 
1000 loops, best of 3: 454 µs per loop

In [29]: %timeit eze(df)
The slowest run took 4.83 times longer than the fastest. This could mean that an intermediate result is being cached 
1000 loops, best of 3: 422 µs per loop

In [30]: %timeit sy2(df)
The slowest run took 4.57 times longer than the fastest. This could mean that an intermediate result is being cached 
1000 loops, best of 3: 1.46 ms per loop

测试len(df) = 10000:

In [32]: %timeit jez(df)
The slowest run took 4.78 times longer than the fastest. This could mean that an intermediate result is being cached 
1000 loops, best of 3: 543 µs per loop

In [33]: %timeit eze(df)
1 loops, best of 3: 245 ms per loop

In [34]: %timeit sy2(df)
The slowest run took 4.11 times longer than the fastest. This could mean that an intermediate result is being cached 
100 loops, best of 3: 9.11 ms per loop

【讨论】：