计算 Pandas 的每月异常值

Question

你好 StackOverflow 社区，

我一直对使用 Python 3.9.1 和 Numpy 1.19.5 计算 pandas 1.2.0 中的数据异常感兴趣，但一直在努力找出完成这项任务的最“Pythonic”和“pandas”方式（或任何方式）。下面我创建了一些虚拟数据并将其放入熊猫DataFrame。此外，我试图清楚地概述我计算虚拟数据的月度异常的方法。

我要做的是获取“n”年的月度值（在本例中，2 年的月度数据 = 25 个月）并计算所有年份的月度平均值（例如，将所有 1 月的值组合在一起并计算均值）。我已经能够使用 pandas 做到这一点。

接下来，我想取每个月的平均值，然后从我的 DataFrame 中属于该特定月份的所有元素中减去它（例如从 1 月份的整体平均值中减去每个 1 月份的值）。在下面的代码中，您将看到一些尝试执行减法的代码行，但无济于事。

如果有人对解决此问题的好方法有任何想法或提示，我非常感谢您的洞察力。如果您需要进一步澄清，请告诉我。感谢您的时间和想法。

-玛丽安

#Import packages
import numpy as np
import pandas as pd
#-------------------------------------------------------------
#Create a pandas dataframe with some data that will represent:
#Column of dates for two years, at monthly resolution
#Column of corresponding values for each date.

#Create two years worth of monthly dates
dates = pd.date_range(start='2018-01-01', end='2020-01-01', freq='MS')

#Create some random data that will act as our data that we want to compute the anomalies of
values = np.random.randint(0,100,size=25)

#Put our dates and values into a dataframe to demonsrate how we have tried to calculate our anomalies
df = pd.DataFrame({'Dates': dates, 'Values': values})
#-------------------------------------------------------------
#Anomalies will be computed by finding the mean value of each month over all years
#And then subtracting the mean value of each month by each element that is in that particular month

#Group our df according to the month of each entry and calculate monthly mean for each month
monthly_means = df.groupby(df['Dates'].dt.month).mean()
#-------------------------------------------------------------
#Now, how do we go about subtracting these grouped monthly means from each element that falls
#in the corresponding month. 
#For example, if the monthly mean over 2 years for January is 20 and the value is 21 in January 2018, the anomaly would be +1 for January 2018

#Example lines of code I have tried, but have not worked

#ValueError:Unable to coerce to Series, length must be 1: given 12
#anomalies = socal_csv.groupby(socal_csv['Date'].dt.month) - monthly_means

#TypeError: unhashable type: "list"
#anomalies = socal_csv.groupby(socal_csv['Date'].dt.month).transform([np.subtract])

Answer 1

你可以像这样使用pd.merge：

import numpy as np
import pandas as pd

dates = pd.date_range(start='2018-01-01', end='2020-01-01', freq='MS')


values = np.random.randint(0,100,size=25)


df = pd.DataFrame({'Dates': dates, 'Values': values})

monthly_means = df.groupby(df['Dates'].dt.month.mean()


df['month']=df['Dates'].dt.strftime("%m").astype(int)
df=df.merge(monthly_means.rename(columns={'Dates':'month','Values':'Mean'}),on='month',how='left')
df['Diff']=df['Mean']-df['Values']

输出：

 df['Diff']
Out[19]: 
0     33.333333
1     19.500000
2    -29.500000
3    -22.500000
4    -24.000000
5     -3.000000
6     10.000000
7      2.500000
8     14.500000
9    -17.500000
10    44.000000
11    31.000000
12   -11.666667
13   -19.500000
14    29.500000
15    22.500000
16    24.000000
17     3.000000
18   -10.000000
19    -2.500000
20   -14.500000
21    17.500000
22   -44.000000
23   -31.000000
24   -21.666667

如果你想要绝对差异，你可以使用abs()

Answer 2

单行解决方案是：

df = pd.DataFrame({'Values': values}, index=dates)
df.groupby(df.index.month).transform(lambda x: x-x.mean())