Pandas pd.cut on Timestamps -“ValueError：bins必须单调增加”

Question

我正在尝试将时间序列数据拆分为如下标记的段：

import pandas as pd
import numpy as np

# Create example DataFrame of stock values
df = pd.DataFrame({
    'ticker':np.repeat( ['aapl','goog','yhoo','msft'], 25 ),
    'date':np.tile( pd.date_range('1/1/2011', periods=25, freq='D'), 4 ),
    'price':(np.random.randn(100).cumsum() + 10) })

# Cut the date into sections 
today = df['date'].max()
bin_edges = [pd.Timestamp.min, today - pd.Timedelta('14 days'), today - pd.Timedelta('7 days'), pd.Timestamp.max]
df['Time Group'] = pd.cut(df['date'], bins=bin_edges, labels=['history', 'previous week', 'this week'])

但即使bin_edges 似乎确实在单调增加，我也遇到了错误..

Traceback (most recent call last):
  File "C:\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3267, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-42-00524c0a883b>", line 13, in <module>
    df['Time Group'] = pd.cut(df['date'], bins=bin_edges, labels=['history', 'previous week', 'this week'])
  File "C:\Anaconda3\lib\site-packages\pandas\core\reshape\tile.py", line 228, in cut
    raise ValueError('bins must increase monotonically.')
ValueError: bins must increase monotonically.


In[43]: bin_edges
Out[43]: 
[Timestamp('1677-09-21 00:12:43.145225'),
 Timestamp('2011-01-11 00:00:00'),
 Timestamp('2011-01-18 00:00:00'),
 Timestamp('2262-04-11 23:47:16.854775807')]

为什么会这样？

Answer 1

这是熊猫中的一个错误。您的边缘需要转换为数值才能执行cut，并且通过使用pd.Timestamp.min 和pd.Timestamp.max，您实际上是在将边缘设置为可以用64 位整数表示的下限/上限。当尝试比较单调性边缘时，这会导致溢出，这使它看起来不是单调递增的。

溢出演示：

In [2]: bin_edges_numeric = [t.value for t in bin_edges]

In [3]: bin_edges_numeric
Out[3]:
[-9223372036854775000,
 1294704000000000000,
 1295308800000000000,
 9223372036854775807]

In [4]: np.diff(bin_edges_numeric)
Out[4]:
array([-7928668036854776616,      604800000000000,  7928063236854775807],
      dtype=int64)

在解决此问题之前，我的建议是使用更接近实际日期但仍能达到相同最终结果的下限/上限：

first = df['date'].min()
today = df['date'].max()
bin_edges = [first - pd.Timedelta('1000 days'), today - pd.Timedelta('14 days'),
             today - pd.Timedelta('7 days'), today + pd.Timedelta('1000 days')]

我任意选择了 1000 天，您可以根据需要选择不同的值。通过这些修改，cut 应该不会引发错误。