Python DataFrame：根据自定义条件分隔行？

Question

我的数据框包含三列 name、content 和 day

df

        content          day           name
    0     first_day      01-01-2017      marcus
    1     present        10-01-2017      marcus
    2     first_day      01-02-2017      marcus
    3     first_day      01-03-2017      marcus
    4     absent         05-03-2017      marcus
    5     present        20-03-2017      marcus
    6     first_day      01-04-2017      bruno
    7     present        11-04-2017      bruno
    8     first_day      01-05-2017      bruno
    9     absent         02-05-2017      bruno
    10    first_day      01-06-2017      bruno
    11    absent         02-06-2017      bruno
    12    payment        09-06-2017      bruno

我正在尝试找出用户month wise 的行连续有first_day、absent 和present。

样本输出：

        content          day           name         absent_after_present
    0     first_day      01-01-2017      marcus         False
    1     first_day      01-02-2017      marcus         False
    2     first_day      01-03-2017      marcus         True
    3     first_day      01-04-2017      bruno          False
    4     first_day      01-05-2017      bruno          False
    5     first_day      01-06-2017      bruno          True

例如：marcus 有 first_day,absent 和 present 连续来自 01-03-2017,05-03-2017 和 20-03-2017 在same month。所以marcus状态应该是True

Answer 1

您可以尝试提取每月内容，然后按名称和月份分组，如下所示。

import pandas as pd

data = pd.DataFrame({'content' : ['first_day','present', 'first_day', 'first_day', 'absent', 
'present', 'first_day', 'present', 'first_day', 'absent', 'first_day', 'absent', 'present'],
'day' : ['2017-01-01', '2017-01-10', '2017-02-01', '2017-03-01', '2017-03-05', '2017-03-20',
'2017-04-01', '2017-04-11', '2017-05-01', '2017-05-02', '2017-06-01', '2017-06-02', '2017-06-09'],
'name' : ['marcus', 'marcus', 'marcus', 'marcus', 'marcus', 'marcus', 'bruno', 'bruno', 'bruno',
'bruno', 'bruno', 'bruno', 'bruno']})

data['day'] = pd.to_datetime(data['day'])

data['month'] = data.day.dt.month

data_new = pd.DataFrame(data.groupby(['name', 'month'])['content'].unique()).join(pd.DataFrame(data.groupby(['name', 'month'])['day'].unique()), on=['name', 'month'])

data_new['absent_after_present'] = data_new['content'].apply(lambda x : True if len(x) == 3 and len(set(x)) == 3 else False)
data_new['day'] = data_new['day'].apply(lambda x : x[0])
data_new['content'] = data_new['content'].apply(lambda x : x[0])

data_new = data_new.droplevel(1)



data_new


name    content        day  absent_after_present

bruno   first_day   2017-04-01  False
bruno   first_day   2017-05-01  False
bruno   first_day   2017-06-01  True
marcus  first_day   2017-01-01  False
marcus  first_day   2017-02-01  False
marcus  first_day   2017-03-01  True

Answer 2

如果某个组包含，您的预期输出包含 True '缺席' 之前 '现在'（不是之后）。

所以我将源 DataFrame 定义为：

      content         day    name
0   first_day  01-01-2017  marcus
1     present  10-01-2017  marcus
2   first_day  01-02-2017  marcus
3   first_day  01-03-2017  marcus
4      absent  05-03-2017  marcus
5     present  20-03-2017  marcus
6   first_day  01-04-2017   bruno
7     present  11-04-2017   bruno
8   first_day  01-05-2017   bruno
9      absent  02-05-2017   bruno
10  first_day  01-06-2017   bruno
11     absent  02-06-2017   bruno
12    present  09-06-2017   bruno

（注意最后一行的变化）。

开始于：

import itertools

然后定义一个函数，它从源组（grp）返回第一行， 最后（新）列的附加值：

def getRow(grp):
    lst = [k for k, g in itertools.groupby(grp.content)]
    isAbs = lst[-2] == 'absent' and lst[-1] == 'present' if len(lst) > 1 else False
    return grp.iloc[0].append(pd.Series([isAbs], index=['absent_before_present']))

为了得到预期的结果，运行：

result = df.groupby([pd.to_datetime(df.day, dayfirst=True)
    .apply(lambda x: x.strftime('%Y-%m')), 'name']).apply(getRow)\
    .reset_index(drop=True)

结果是：

     content         day    name  absent_before_present
0  first_day  01-01-2017  marcus                  False
1  first_day  01-02-2017  marcus                  False
2  first_day  01-03-2017  marcus                   True
3  first_day  01-04-2017   bruno                  False
4  first_day  01-05-2017   bruno                  False
5  first_day  01-06-2017   bruno                   True

请注意，上面的代码实际上使用了 2 种不同的 groupby 方法：

• 来自Pandas（按年、月和名称分组df），
• 来自 itertools，其中每个新值（现在除外）都会创建一个新值 输出组。

Answer 3

尝试以下方法：

def pattern_in_group(s):
    s_list = s.to_list()
    for i in range(0, len(s_list)-2):
        if ['first_day', 'absent', 'present' ] == s_list[i:i+3]:
            return True
    return False

df['day1'] = pd.to_datetime(df['day'], dayfirst=True)
df['absent_after_present'] = df.groupby(['name', df['day1'].dt.year, df['day1'].dt.month])['content'].transform(pattern_in_group)

df2 = df.groupby(['name', df['day1'].dt.year, df['day1'].dt.month], as_index=False).first().drop(columns='day1')

print(df2)



     name    content         day  absent_after_present
0   bruno  first_day  01-04-2017                 False
1   bruno  first_day  01-05-2017                 False
2   bruno  first_day  01-06-2017                  True
3  marcus  first_day  01-01-2017                 False
4  marcus  first_day  01-02-2017                 False
5  marcus  first_day  01-03-2017                  True

由于您在最后一行的示例数据中有一个错字，我已将其更正如下：

测试数据构建

data = {'content': ['first_day', 'present', 'first_day', 'first_day', 'absent', 'present', 'first_day', 'present', 'first_day', 'absent', 'first_day', 'absent', 'present'], 
 'day': ['01-01-2017', '10-01-2017', '01-02-2017', '01-03-2017', '05-03-2017', '20-03-2017', '01-04-2017', '11-04-2017', '01-05-2017', '02-05-2017', '01-06-2017', '02-06-2017', '09-06-2017'],
 'name': ['marcus', 'marcus', 'marcus', 'marcus', 'marcus', 'marcus', 'bruno', 'bruno', 'bruno', 'bruno', 'bruno', 'bruno', 'bruno']}   

df = pd.DataFrame(data)

print(df)

      content         day    name
0   first_day  01-01-2017  marcus
1     present  10-01-2017  marcus
2   first_day  01-02-2017  marcus
3   first_day  01-03-2017  marcus
4      absent  05-03-2017  marcus
5     present  20-03-2017  marcus
6   first_day  01-04-2017   bruno
7     present  11-04-2017   bruno
8   first_day  01-05-2017   bruno
9      absent  02-05-2017   bruno
10  first_day  01-06-2017   bruno
11     absent  02-06-2017   bruno
12    present  09-06-2017   bruno