基于字典值映射将多索引数据帧与自身合并答案

【问题标题】：Merging multiindex dataframe with itself based on a dictionary value mappings基于字典值映射将多索引数据帧与自身合并
【发布时间】：2021-09-28 15:28:15
【问题描述】：

我有一个多索引数据框：

df = pd.DataFrame.from_dict({('group', ''): {0: 'A',
  1: 'A',
  2: 'A',
  3: 'A',
  4: 'A',
  5: 'A',
  6: 'A',
  7: 'A',
  8: 'B',
  9: 'B',
  10: 'B',
  11: 'B',
  12: 'B',
  13: 'B',
  14: 'B',
  15: 'B'},
 ('category', ''): {0: 'Amazon',
  1: 'Netflix',
  2: 'Apple',
  3: 'Facebook',
  4: 'Tesla',
  5: 'Uber',
  6: 'Google',
  7: 'total',
  8: 'Amazon',
  9: 'Netflix',
  10: 'Apple',
  11: 'Facebook',
  12: 'Tesla',
  13: 'Uber',
  14: 'Google',
  15: 'total'},
 (pd.Timestamp('2020-06-29 00:00:00'), 'total_orders'): {0: 195.0,
  1: 37.0,
  2: 61.0,
  3: 106.0,
  4: 13.0,
  5: 4.0,
  6: 61.0,
  7: 477.0,
  8: 50.0,
  9: 17.0,
  10: 50.0,
  11: 75.0,
  12: 14.0,
  13: 3.0,
  14: 43.0,
  15: 252.0},
 (pd.Timestamp('2020-06-29 00:00:00'), 'total_sales'): {0: 1268.85,
  1: 15323.800000000001,
  2: 18274.385000000002,
  3: 19722.65,
  4: 1688.6749999999997,
  5: 1906.0,
  6: 55547.255,
  7: 113731.615,
  8: 3219.6499999999996,
  9: 5918.5,
  10: 15852.060000000001,
  11: 17743.7,
  12: 1708.75,
  13: 937.01,
  14: 37795.15,
  15: 83174.82},
 (pd.Timestamp('2021-06-28 00:00:00'), 'total_orders'): {0: 96.0,
  1: 34.0,
  2: 56.0,
  3: 106.0,
  4: 13.0,
  5: 9.0,
  6: 44.0,
  7: 358.0,
  8: 101.0,
  9: 13.0,
  10: 22.0,
  11: 120.0,
  12: 8.0,
  13: 1.0,
  14: 40.0,
  15: 305.0},
 (pd.Timestamp('2021-06-28 00:00:00'), 'total_sales'): {0: 5194.95,
  1: 11461.25,
  2: 19102.219999999994,
  3: 22796.420000000002,
  4: 992.6,
  5: 3671.15,
  6: 30853.115,
  7: 94071.705,
  8: 6022.299999999998,
  9: 4749.000000000001,
  10: 7373.6,
  11: 33514.0,
  12: 902.01,
  13: 349.3,
  14: 35943.45,
  15: 88853.66},
 (pd.Timestamp('2020-07-06 00:00:00'), 'total_orders'): {0: 26.0,
  1: 10.0,
  2: 39.0,
  3: 79.0,
  4: 10.0,
  5: 5.0,
  6: 49.0,
  7: 218.0,
  8: 89.0,
  9: 21.0,
  10: 34.0,
  11: 133.0,
  12: 20.0,
  13: 3.0,
  14: 66.0,
  15: 366.0},
 (pd.Timestamp('2020-07-06 00:00:00'), 'total_sales'): {0: 3978.15,
  1: 4280.15,
  2: 12138.96,
  3: 19084.175,
  4: 1495.1,
  5: 1764.15,
  6: 40033.46000000001,
  7: 82774.145,
  8: 8314.92,
  9: 6962.844999999999,
  10: 12776.649999999996,
  11: 28048.075,
  12: 3053.2000000000003,
  13: 1264.655,
  14: 55104.21000000002,
  15: 115524.55500000001},
 (pd.Timestamp('2021-07-07 00:00:00'), 'total_orders'): {0: 45.0,
  1: 13.0,
  2: 47.0,
  3: 87.0,
  4: 8.0,
  5: 2.0,
  6: 45.0,
  7: 247.0,
  8: 81.0,
  9: 9.0,
  10: 36.0,
  11: 143.0,
  12: 9.0,
  13: 1.0,
  14: 56.0,
  15: 335.0},
 (pd.Timestamp('2021-07-07 00:00:00'), 'total_sales'): {0: 7556.414999999998,
  1: 4024.97,
  2: 14985.05,
  3: 16790.899999999998,
  4: 1034.45,
  5: 1385.65,
  6: 36202.729999999996,
  7: 81980.16499999998,
  8: 5600.544999999999,
  9: 3885.1499999999996,
  10: 11209.92,
  11: 32832.61,
  12: 1191.5,
  13: 599.0,
  14: 42137.44500000001,
  15: 97456.17000000001}}).set_index(['group','category'])

我正在尝试在total_sales 旁边添加last_year_sales。例如，在 df 中，我们有 2020-06-29 和 2021-06-28，它们都是 7 月的第一周。

实际上，周的顺序是从2018 到2021。

我对从 7 月的第一周到当前日期的日期感兴趣。所以问题是我不确定如何将数据框与自身合并，将上周的销售额重命名为last_year_sales 并将其合并到正确的位置，即同一周，只是不同的年份。我想使用这本字典，其中注明了 2020 和 2021 对应的周数：

date_maps = {'2020-06-29' : '2021-06-28',
'2020-07-06' : '2021-07-05',
'2020-07-13' : '2021-07-12',
'2020-07-20' : '2021-07-19',
'2020-07-27' : '2021-07-26',
'2020-08-03' : '2021-08-02',
'2020-08-10' : '2021-08-09',
'2020-08-17' : '2021-08-16',
'2020-08-24' : '2021-08-23',
'2020-08-31' : '2021-08-30',
'2020-09-07' : '2021-09-06',
'2020-09-14' : '2021-09-13',
'2020-09-21' : '2021-09-20',
'2020-09-28' : '2021-09-27',
'2020-10-05' : '2021-10-04',
'2020-10-12' : '2021-10-11',
'2020-10-19' : '2021-10-18',
'2020-10-26' : '2021-10-25',
'2020-11-02' : '2021-11-01',
'2020-11-09' : '2021-11-08',
'2020-11-16' : '2021-11-15',
'2020-11-23' : '2021-11-22',
'2020-11-30' : '2021-11-29',
'2020-12-07' : '2021-12-06',
'2020-12-14' : '2021-12-13',
'2020-12-21' : '2021-12-20',
'2020-12-28' : '2021-12-27',
'2021-01-04' : '2022-01-03',
'2021-01-11' : '2022-01-10',
'2021-01-18' : '2022-01-17',
'2021-01-25' : '2022-01-24',
'2021-02-01' : '2022-01-31',
'2021-02-08' : '2022-02-07',
'2021-02-15' : '2022-02-14',
'2021-02-22' : '2022-02-21',
'2021-03-01' : '2022-02-28',
'2021-03-08' : '2022-03-07',
'2021-03-15' : '2022-03-14',
'2021-03-22' : '2022-03-21',
'2021-03-29' : '2022-03-28',
'2021-04-05' : '2022-04-04',
'2021-04-12' : '2022-04-11',
'2021-04-19' : '2022-04-18',
'2021-04-26' : '2022-04-25',
'2021-05-03' : '2022-05-02',
'2021-05-10' : '2022-05-09',
'2021-05-17' : '2022-05-16',
'2021-05-24' : '2022-05-23',
'2021-05-31' : '2022-05-30',
'2021-06-07' : '2022-06-06',
'2021-06-14' : '2022-06-13',
'2021-06-21' : '2022-06-20',
'2021-06-28' : '2022-06-27'}

我想要的输出如下所示：

                        2021-06-28                                                                         2021-07-07      
                        total_orders    total_sales     last_year_sales     last_year_total_orders         total_orders     total_sales     last_year_sales     last_year_total_orders
group       category                                
A           Amazon      195.000         1,268.850       5,194.950           195.000                         ...            ...              ...                 ...
            Netflix     37.000          15,323.800      11,461.250          20.00
            Apple       61.000          18,274.385      19,102.220          30.00   
            Facebook    106.000         19,722.650      22,796.420          50.00
            Tesla       13.000          1,688.675       992.600             25.00
            Uber        4.000           1,906.000       3,671.150           15.00
            Google      61.000          55,547.255      30,853.115          10.00
            total       477.000         113,731.615     94,071.705          56.00
B           Amazon      50.000          3,219.650       6,022.300           400.00
            Netflix     17.000          5,918.500       4,749.000           30.00
            Apple       50.000          15,852.060      7,373.600           27.00
            Facebook    75.000          17,743.700      33,514.000          15.00
            Tesla       14.000          1,708.750       902.010             66.00
            Uber        3.000           937.010         349.300             94.00
            Google      43.000          37,795.150      35,943.450          65.00
            total       252.000         83,174.820      88,853.660          61.00

其中last_year_x 表示去年一周的sales 和orders，在date_maps 字典中表示。我会提供我尝试过的东西，但我不知道从哪里开始。

【问题讨论】：

标签： python pandas multi-index

【解决方案1】：

您可以重命名列以获得所需的内容，假设日期始终按this_year 和last_year 的顺序排列

# Get the current column names in level 1
l = df.droplevel(0, axis=1).columns
l

Index(['total_orders', 'total_sales', 'total_orders', 'total_sales',
       'total_orders', 'total_sales', 'total_orders', 'total_sales'],
      dtype='object')

# Pair up the column names by this year and last year
temp = list(zip(l[::2],l[1::2]))
temp

[('total_orders', 'total_sales'),
 ('total_orders', 'total_sales'),
 ('total_orders', 'total_sales'),
 ('total_orders', 'total_sales')]

# Do a text replacement on the odd pairs (last years')
pairs = [(x[0].replace('total_','last_years_'), x[1].replace('total_','last_years_')) if i%2 == 1 else x for i, x in enumerate(temp)]
pairs

[('total_orders', 'total_sales'),
 ('last_years_orders', 'last_years_sales'),
 ('total_orders', 'total_sales'),
 ('last_years_orders', 'last_years_sales')]

# Flatten the list
flatten_list = [item for subl in pairs for item in subl]
flatten_list

['total_orders',
 'total_sales',
 'last_years_orders',
 'last_years_sales',
 'total_orders',
 'total_sales',
 'last_years_orders',
 'last_years_sales']

# Get a copy of the dataframe with level 0 dropped
dfdrop = df.droplevel(0, axis=1)

# Change the column names to the flattened list
# Note this could also be achieved with df.rename...
dfdrop.columns = flatten_list
dfdrop

    total_orders    total_sales last_years_orders   last_years_sales    total_orders    total_sales last_years_orders   last_years_sales
group   category                                
A   Amazon  195.0   1268.850    96.0    5194.950    26.0    3978.150    45.0    7556.415
Netflix 37.0    15323.800   34.0    11461.250   10.0    4280.150    13.0    4024.970
Apple   61.0    18274.385   56.0    19102.220   39.0    12138.960   47.0    14985.050
Facebook    106.0   19722.650   106.0   22796.420   79.0    19084.175   87.0    16790.900
Tesla   13.0    1688.675    13.0    992.600 10.0    1495.100    8.0 1034.450
Uber    4.0 1906.000    9.0 3671.150    5.0 1764.150    2.0 1385.650
Google  61.0    55547.255   44.0    30853.115   49.0    40033.460   45.0    36202.730
total   477.0   113731.615  358.0   94071.705   218.0   82774.145   247.0   81980.165
B   Amazon  50.0    3219.650    101.0   6022.300    89.0    8314.920    81.0    5600.545
Netflix 17.0    5918.500    13.0    4749.000    21.0    6962.845    9.0 3885.150
Apple   50.0    15852.060   22.0    7373.600    34.0    12776.650   36.0    11209.920
Facebook    75.0    17743.700   120.0   33514.000   133.0   28048.075   143.0   32832.610
Tesla   14.0    1708.750    8.0 902.010 20.0    3053.200    9.0 1191.500
Uber    3.0 937.010 1.0 349.300 3.0 1264.655    1.0 599.000
Google  43.0    37795.150   40.0    35943.450   66.0    55104.210   56.0    42137.445
total   252.0   83174.820   305.0   88853.660   366.0   115524.555  335.0   97456.170

# Get a list of the datetimes
dt = pd.DataFrame(df.T.index.get_level_values(0), columns=["Date"])
dt["Date"] = pd.to_datetime(dt["Date"])
dt

Date
0   2020-06-29
1   2020-06-29
2   2021-06-28
3   2021-06-28
4   2020-07-06
5   2020-07-06
6   2021-07-07
7   2021-07-07

# Make a multiindex and add back into dfdrop
idx = pd.MultiIndex.from_arrays([dfdrop.columns.tolist(), dt["Date"]])
dfdrop.columns = idx
dfdrop


        total_orders    total_sales last_years_orders   last_years_sales    total_orders    total_sales last_years_orders   last_years_sales
Date    2020-06-29  2020-06-29  2021-06-28  2021-06-28  2020-07-06  2020-07-06  2021-07-07  2021-07-07
group   category                                
A   Amazon  195.0   1268.850    96.0    5194.950    26.0    3978.150    45.0    7556.415
Netflix 37.0    15323.800   34.0    11461.250   10.0    4280.150    13.0    4024.970
Apple   61.0    18274.385   56.0    19102.220   39.0    12138.960   47.0    14985.050
Facebook    106.0   19722.650   106.0   22796.420   79.0    19084.175   87.0    16790.900
Tesla   13.0    1688.675    13.0    992.600 10.0    1495.100    8.0 1034.450
Uber    4.0 1906.000    9.0 3671.150    5.0 1764.150    2.0 1385.650
Google  61.0    55547.255   44.0    30853.115   49.0    40033.460   45.0    36202.730
total   477.0   113731.615  358.0   94071.705   218.0   82774.145   247.0   81980.165
B   Amazon  50.0    3219.650    101.0   6022.300    89.0    8314.920    81.0    5600.545
Netflix 17.0    5918.500    13.0    4749.000    21.0    6962.845    9.0 3885.150
Apple   50.0    15852.060   22.0    7373.600    34.0    12776.650   36.0    11209.920
Facebook    75.0    17743.700   120.0   33514.000   133.0   28048.075   143.0   32832.610
Tesla   14.0    1708.750    8.0 902.010 20.0    3053.200    9.0 1191.500
Uber    3.0 937.010 1.0 349.300 3.0 1264.655    1.0 599.000
Google  43.0    37795.150   40.0    35943.450   66.0    55104.210   56.0    42137.445
total   252.0   83174.820   305.0   88853.660   366.0   115524.555  335.0   97456.170

这接近您想要的，但它保留了所有日期。

【讨论】：

日期按我在问题中提到的从2018 到现在的顺序排列。这适用于有序列名吗？
啊，从您的数据框中看，日期似乎是按月/日排序的：2020-06-29 00:00:00 2021-06-28 00:00:00 2020-07-06 00:00:00 2021-07-07 00:00:00。只要这是真的，它就可以工作。此外，我注意到我已经从列名中删除了日期，因此将其添加回来。
我已将日期作为多索引添加回来。
我明白了，但问题是数据结构不同，我无法使用它。
如果您按周数或按月/日对列进行排序，那么上述方法应该可以工作。你可以使用这样的东西：dt["week"] = dt["Date"].dt.isocalendar().week idx = pd.MultiIndex.from_arrays([dt["Date"], dt["week"], df.T.index.get_level_values(1)]) df2 = df df2.columns = idx df2.sort_values(by="week", axis=1) 假设你已经按照答案中的代码声明了dt。