按周分组一个数据框

Question

我有一个数据框：

Date        Articles
2010-01-04  ((though, reliant, advertis, revenu, internet,...
2010-01-05  ((googl, expect, nexus, one, rival, iphon, hel...
2010-01-06  ((while, googl, introduc, first, piec, hardwar...
2010-01-07  ((googl, form, energi, subsidiari, appli, gove...
2010-01-08  ((david, pogu, review, googl, new, offer, nexu...
2010-01-12  ((the, compani, agre, hand, list, book, scan, ...

日期是索引，文章是元组的元组。

我有另一个数据框：

Date        Price
2010-01-08  602.020
2010-01-15  580.000
2010-01-22  550.010
2010-01-29  529.944

其中 Date 也是索引，但分为周。

我的问题是我想在第二个数据框中创建另一列，该列将包含该特定周之前的所有文章，由索引指示。就像我的第二个数据框中的第一行一样，我希望所有文章都从 2010 年 1 月 8 日之前的第一个数据框中提取（所以这将是我的第一个数据框中的前 4 个条目）。与 2010-01-15 一样，我需要从 2010-01-08 到 2010-01-14 的所有文章，依此类推。

任何帮助将不胜感激。谢谢。

Answer 1

我们可以使用IntervalIndex.from_breaks和pd.cut

df1 = pd.DataFrame({'Articles': 
                   {pd.Timestamp('2010-01-04 00:00:00'): [0, 1],
                    pd.Timestamp('2010-01-05 00:00:00'): [2, 3],
                    pd.Timestamp('2010-01-06 00:00:00'): [4, 5],
                    pd.Timestamp('2010-01-07 00:00:00'): [6, 7],
                    pd.Timestamp('2010-01-08 00:00:00'): [8, 9],
                    pd.Timestamp('2010-01-12 00:00:00'): [10, 11]}})

            Articles
2010-01-04  [0, 1]
2010-01-05  [2, 3]
2010-01-06  [4, 5]
2010-01-07  [6, 7]
2010-01-08  [8, 9]
2010-01-12  [10, 11]

mybins = pd.IntervalIndex.from_breaks(
             pd.date_range("2010-1-1", periods=5, freq="7D"),
             closed="left"
         )

df1["bin"] = pd.cut(df1.index, bins=mybins)
df1.groupby("bin")["Articles"].sum()

bin
[2010-01-01, 2010-01-08)    [0, 1, 2, 3, 4, 5, 6, 7]
[2010-01-08, 2010-01-15)              [8, 9, 10, 11]
[2010-01-15, 2010-01-22)                        None
[2010-01-22, 2010-01-29)                        None
Name: Articles, dtype: object

Answer 2

这是一个使用merge_asof 和allow_exact_matches=False 的两步解决方案，以便每个文章行与日期严格大于（不等于）文章行的第一个价格匹配日期。

.agg(sum) 使用了这样一个事实，即添加两个元组会将它们组合成一个元组。

假设您的 DataFrame 被命名为 df 和 df2：

# Test data adapted from your examples.
# Sorry that this is difficult to copy-paste into pandas

df
            Articles
2010-01-04  (though, reliant, advertis, revenu, internet)        
2010-01-05  ((googl, expect, nexus), (one, rival, iphon))        
2010-01-06  ((while, googl, introduc), (first,), (piec, hardwar))
2010-01-07  ((googl, form), (energi, subsidiari), (appli,))      
2010-01-08  ((david, pogu, review), (googl, new, offer))         
2010-01-12  ((the, compani), (agre, hand, list), (book, scan)) 

df2
            Price               
2010-01-08  602.020
2010-01-15  580.000
2010-01-22  550.010
2010-01-29  529.944


# Solution

price2articles = (pd.merge_asof(df, 
                               df2, 
                               left_index=True, 
                               right_index=True, 
                               allow_exact_matches=False,
                               direction='forward')
                .groupby('Price')
                .agg(sum))

result = pd.merge(df2, price2article, left_on='Price', right_index=True)
# To see full contents of wide data, set
# pd.options.display.max_colwidth = 150 or higher (-1 for no limit)
result

            Articles                                                                                                                                                                                                          
2010-01-08  (though, reliant, advertis, revenu, internet, (googl, expect, nexus), (one, rival, iphon), (while, googl, introduc), (first,), (piec, hardwar), (googl, form), (energi, subsidiari), (appli,))  
2010-01-15  ((david, pogu, review), (googl, new, offer), (the, compani), (agre, hand, list), (book, scan))

Answer 3

我认为需要 cut 的值 df2['Date'] 与 groupby 并将元组连接到 lists：

print (df1)
        Date          Articles
0 2010-01-04  ((t, r), (s, q))
1 2010-01-07  ((g, f), (y, l))
2 2010-01-08  ((d, p), (t, o))
3 2010-01-12  ((t, c), (r, p))

b = pd.concat([df2['Date'], 
               pd.Series(pd.to_datetime(['1970-01-01','2100-01-01']))]).sort_values()

df1['Dates'] = pd.cut(df1['Date'], bins=b, labels=b[1:], right=False)
df3 = (df1.groupby('Dates')['Articles']
         .apply(lambda x: [i for s in x for i in s])
         .iloc[:-1]
         .reset_index())
print (df3)
       Dates                          Articles
0 2010-01-08  [(t, r), (s, q), (g, f), (y, l)]
1 2010-01-15  [(d, p), (t, o), (t, c), (r, p)]
2 2010-01-22                                []
3 2010-01-29                                []

最后如果想过滤掉空的lists:

df3 = df3[df3['Articles'].astype(bool)]
print (df3)
       Dates                          Articles
0 2010-01-08  [(t, r), (s, q), (g, f), (y, l)]
1 2010-01-15  [(d, p), (t, o), (t, c), (r, p)]

Answer 4

也许这个相当简单的两线也可以工作： （这利用了日历周，不是在 2010 年 1 月 8 日而是在 1 月 11 日左右休息）

m = {ind:dfx['Articles'].tolist() for ind,dfx in df1.groupby(df1.index.week)} 
df2['new'] = pd.Series(df2.index.week).map(m).values

如果您想要实际日期，我们可以修改此代码以使用日历日的 div：

m = {ind+1:dfx['Articles'].tolist() for ind,dfx in df1.groupby((df1.index.dayofyear-1)//7)}
df2['new'] = pd.Series(df2.index.week).map(m).values

---

完整示例：

import pandas as pd

data1 = '''\
Date        Articles
2010-01-04  1
2010-01-05  2
2010-01-06  3
2010-01-07  4
2010-01-08  5'''

data2 = '''\
Date        Price
2010-01-08  602.020
2010-01-15  580.000
2010-01-22  550.010
2010-01-29  529.944'''

df1 = pd.read_csv(pd.compat.StringIO(data1), sep='\s+', index_col='Date', parse_dates=['Date'])
df2 = pd.read_csv(pd.compat.StringIO(data2), sep='\s+', index_col='Date', parse_dates=['Date'])

m = {ind:dfx['Articles'].tolist() for ind,dfx in df1.groupby(df1.index.week)}

df2['new'] = pd.Series(df2.index.week).map(m).values

df2:

              Price              new
Date                                
2010-01-08  602.020  [1, 2, 3, 4, 5]
2010-01-15  580.000              NaN
2010-01-22  550.010              NaN
2010-01-29  529.944              NaN

或：

              Price           new
Date                             
2010-01-08  602.020  [1, 2, 3, 4]
2010-01-15  580.000           [5]
2010-01-22  550.010           NaN
2010-01-29  529.944           NaN