【问题标题】:pandas - removing one of the duplicates if the date is consecutivepandas - 如果日期是连续的,则删除其中一个重复项
【发布时间】:2021-04-26 18:23:59
【问题描述】:

假设以下数据框:

date | id
2020-12-11 | A
2020-12-15 | A
2020-04-09 | B
2020-04-09 | C
2020-04-08 | C
2021-03-11 | D
2021-03-12 | D
2021-01-24 | E
2021-01-19 | E

期望的输出:

date | id
2020-12-11 | A
2020-04-09 | B
2020-04-09 | C
2020-04-08 | C
2021-03-11 | D
2021-03-12 | D
2021-01-19 | E

基本上,如果 id 重复,我们要检查日期是否连续。如果连续,则保留两者,否则仅保留较小的日期。目前这就是我所拥有的,但我觉得必须有一种更有效的方法来做到这一点。

df['date'] = df['date'].apply(pd.to_datetime)
for i in range(1, len(df)):
    if date['id'].iloc[i-1] == date['id'].iloc[i]:
        if (abs(df['date'].iloc[i-1] - df['date'].iloc[i])) > datetime.timedelta(days=1): ## check if days are more than 1 day from each other
            print (max(df['date'].iloc[i], df['date'].iloc[i-1])) ## drop this entry, keep the other

【问题讨论】:

    标签: python pandas date datetime


    【解决方案1】:

    让我们试试groupby

    # to_datetime accepts list-like, no need to apply
    df['date'] = pd.to_datetime(df['date'])
    
    s = df.groupby('id')['date']
    mins, maxs = s.transform('min'), s.transform('max')
    
    df[maxs.sub(mins).le(pd.to_timedelta('1D'))  # ID duplicates and consecutive
       | df['date'].eq(mins)                     # always keep the mins
      ]
    

    输出:

            date id
    0 2020-12-11  A
    2 2020-04-09  B
    3 2020-04-09  C
    4 2020-04-08  C
    5 2021-03-11  D
    6 2021-03-12  D
    8 2021-01-19  E
    

    【讨论】:

    • 谢谢!我有一个后续问题...... pd.to_timedelta 有没有办法在这里检查工作日?因此,不要使用 pd.to_timedelta('1D'),而是检查 pd.to_timedelta(one business day)
    • 你能看看Business Day offset
    猜你喜欢
    • 1970-01-01
    • 2012-10-09
    • 2020-06-12
    • 1970-01-01
    • 1970-01-01
    • 2020-04-19
    • 2020-09-08
    • 2022-01-16
    • 1970-01-01
    相关资源
    最近更新 更多