如何从熊猫的时间序列数据中获取斜率？答案

【问题标题】：How to get slope from timeseries data in pandas?如何从熊猫的时间序列数据中获取斜率？
【发布时间】：2019-04-05 15:00:12
【问题描述】：

我有一个 pandas 数据框，其中包含 date 和一些类似于下面的值

原始数据：

list = [('2018-10-29', 6.1925), ('2018-10-29', 6.195), ('2018-10-29', 1.95833333333333), 
        ('2018-10-29', 1.785), ('2018-10-29', 3.05), ('2018-10-29', 1.30666666666667), 
        ('2018-10-29', 1.6325), ('2018-10-30', 1.765), ('2018-10-30', 1.265), 
        ('2018-10-30', 2.1125), ('2018-10-30', 2.16714285714286), ('2018-10-30', 1.485), 
        ('2018-10-30', 1.72), ('2018-10-30', 2.754), ('2018-10-30', 1.79666666666667), 
        ('2018-10-30', 1.27833333333333), ('2018-10-30', 3.48), ('2018-10-30', 6.19), 
        ('2018-10-30', 6.235), ('2018-10-30', 6.11857142857143), ('2018-10-30', 6.088), 
        ('2018-10-30', 4.3), ('2018-10-30', 7.80666666666667), 
        ('2018-10-30', 7.78333333333333), ('2018-10-30', 10.9766666666667), 
        ('2018-10-30', 2.19), ('2018-10-30', 1.88)]

加载到 pandas 后

df = pd.DataFrame(list)


             0          1
0   2018-10-29   6.192500
1   2018-10-29   6.195000
2   2018-10-29   1.958333
3   2018-10-29   1.785000
4   2018-10-29   3.050000
5   2018-10-29   1.306667
6   2018-10-29   1.632500
7   2018-10-30   1.765000
8   2018-10-30   1.265000
9   2018-10-30   2.112500
10  2018-10-30   2.167143
11  2018-10-30   1.485000
12  2018-10-30   1.720000
13  2018-10-30   2.754000
14  2018-10-30   1.796667
15  2018-10-30   1.278333
16  2018-10-30   3.480000
17  2018-10-30   6.190000
18  2018-10-30   6.235000
19  2018-10-30   6.118571
20  2018-10-30   6.088000
21  2018-10-30   4.300000
22  2018-10-30   7.806667
23  2018-10-30   7.783333
24  2018-10-30  10.976667
25  2018-10-30   2.190000
26  2018-10-30   1.880000

这就是我加载数据框的方式

df = pd.DataFrame(list)
df[0] = pd.to_datetime(df[0], errors='coerce')
df.set_index(0, inplace=True)

现在我想找到slope。在互联网上进行研究后，我发现这是获得slope所需要做的事情

trend_coord = list(map(list, zip(df.index.strftime('%Y-%m-%d'), sm.tsa.seasonal_decompose(df.iloc[:,0].values).trend.interpolate(method='linear',axis=0).fillna(0).values)))

results = sm.OLS(np.asarray(sm.tsa.seasonal_decompose(df.iloc[:,0].values).trend.interpolate(method='linear', axis=0).fillna(0).values), sm.add_constant(np.array([i for i in range(len(trend_coord))])), missing='drop').fit()

slope = results.params[1]
print(slope)

但我收到以下错误

Traceback (most recent call last):
  File "/home/souvik/Music/UI_Server2/test35.py", line 11, in <module>
    trend_coord = list(map(list, zip(df.index.strftime('%Y-%m-%d'), sm.tsa.seasonal_decompose(df.iloc[:,0].values).trend.interpolate(method='linear',axis=0).fillna(0).values)))
  File "/home/souvik/django_test/webdev/lib/python3.5/site-packages/statsmodels/tsa/seasonal.py", line 127, in seasonal_decompose
    raise ValueError("You must specify a freq or x must be a "
ValueError: You must specify a freq or x must be a pandas object with a timeseries index with a freq not set to None

现在，如果我在seasonal_decompose 方法中添加freq 参数，例如

trend_coord = list(map(list, zip(df.index.strftime('%Y-%m-%d'), sm.tsa.seasonal_decompose(df.iloc[:,0].values, freq=1).trend.interpolate(method='linear',axis=0).fillna(0).values)))

然后我收到类似

的错误

Traceback (most recent call last):
  File "/home/souvik/Music/UI_Server2/test35.py", line 11, in <module>
    trend_coord = list(map(list, zip(df.index.strftime('%Y-%m-%d'), sm.tsa.seasonal_decompose(df.iloc[:,0].values, freq=1).trend.interpolate(method='linear',axis=0).fillna(0).values)))
AttributeError: 'numpy.ndarray' object has no attribute 'interpolate'

但是，如果我摆脱任何细粒度的数据（例如 interpolate 等）并执行以下操作

trend_coord = sm.tsa.seasonal_decompose(df.iloc[:,0].values, freq=1, model='additive').trend

results = sm.OLS(np.asarray(trend_coord),
                 sm.add_constant(np.array([i for i in range(len(trend_coord))])), missing='drop').fit()
slope = results.params[1]
print(">>>>>>>>>>>>>>>> slope", slope)

然后我得到一个slope 值0.13668559218559242。

但我不确定这是否是找出slope 的正确方法，甚至该值是否正确。

有没有更好的方法找到slope？

【问题讨论】：

不要在内置后调用变量 - 你正在隐藏它们。使用list,dict,tuple,set,max,min,abs,sum,... 作为变量名是一个坏习惯。

标签： python python-2.7 pandas

【解决方案1】：

对于这样的简单情况，要获得线性回归线的斜率和截距（y = 截距 + 斜率 * x），您需要使用 numpy polyfit() 方法。我的解释与下面的代码一致。

# You should only need numpy and pandas
import numpy as np
import pandas as pd

# Now your list 
list = [('2018-10-29', 6.1925), ('2018-10-29', 6.195), ('2018-10-29', 1.95833333333333), 
        ('2018-10-29', 1.785), ('2018-10-29', 3.05), ('2018-10-29', 1.30666666666667), 
        ('2018-10-29', 1.6325), ('2018-10-30', 1.765), ('2018-10-30', 1.265), 
        ('2018-10-30', 2.1125), ('2018-10-30', 2.16714285714286), ('2018-10-30', 1.485), 
        ('2018-10-30', 1.72), ('2018-10-30', 2.754), ('2018-10-30', 1.79666666666667), 
        ('2018-10-30', 1.27833333333333), ('2018-10-30', 3.48), ('2018-10-30', 6.19), 
        ('2018-10-30', 6.235), ('2018-10-30', 6.11857142857143), ('2018-10-30', 6.088), 
        ('2018-10-30', 4.3), ('2018-10-30', 7.80666666666667), 
        ('2018-10-30', 7.78333333333333), ('2018-10-30', 10.9766666666667), 
        ('2018-10-30', 2.19), ('2018-10-30', 1.88)]

# Create a single pandas DataFrame
df = pd.DataFrame(list)

# Make it into a Time Series with 'date' and 'value' columns
ts = pd.DataFrame(list, columns=['date', 'value'])

#print it to check it
ts.head(10)

# Now separate it into x and y lists

x = ts['date']
y = ts['value'].astype(float)

# Create a sequance of integers from 0 to x.size to use in np.polyfit() call
x_seq = np.arange(x.size) # should give you [ 0  1  2  3  4 ... 26]

# call numpy polyfit() method with x_seq, y 
fit = np.polyfit(x_seq, y, 1)
fit_fn = np.poly1d(fit)
print('Slope = ', fit[0], ", ","Intercept = ", fit[1])
print(fit_fn)

斜率 = 0.1366855921855925，截距 = 1.9827865961199274

0.1367 x + 1.983

【讨论】：

顺便说一句，以上答案仅适用于 python 3

【解决方案2】：

我会参与 Franco 的回答，但你不需要 sklearn。您可以使用 scipy 轻松完成。

import datetime as dt
from scipy import stats

df = pd.DataFrame(list, columns=['date', 'value'])
df.date =pd.to_datetime(df.date)
df['date_ordinal'] = pd.to_datetime(df['date']).map(dt.datetime.toordinal)
slope, intercept, r_value, p_value, std_err = stats.linregress(df['date_ordinal'], df['value'])

slope
Out[114]: 0.80959404761905

【讨论】：

欢迎来到 SO！当你给出答案时，即使它是对的，试着解释一下。

【解决方案3】：

您可以使用datetime.toordinal 将每个日期映射到一个整数，并使用sklearn.linear_model 对您的数据拟合线性回归模型以获得如下斜率：

import datetime as dt
from sklearn import linear_model

df = pd.DataFrame(list, columns=['date', 'value'])
df['date_ordinal'] = pd.to_datetime(df['date']).map(dt.datetime.toordinal)
reg = linear_model.LinearRegression()
reg.fit(df['date_ordinal'].values.reshape(-1, 1), df['value'].values)

reg.coef_

array([0.80959405])

【讨论】：