在numpy中聚合时间序列答案

【问题标题】：Aggregate time series in numpy在numpy中聚合时间序列
【发布时间】：2018-05-18 07:00:17
【问题描述】：

我正在尝试将变量聚合到时隙上，但我不确定如何以 Pythonic 方式执行此操作。我的计划是每十秒汇总一次。我的时间数据是第一列，从午夜开始以秒为单位。第二列是我要聚合的变量。大多数条目为零，但其他条目是我想要求和的实际数字。

所以我们的想法是从第 1 列开始每 10 秒对第 2 列求和。矩阵向量的第一列每十秒有一次时间，第二列有一个聚合变量。

我的第一个想法是在第二列上做一个 while 循环，并在我们处于同一时间段时对所有数字求和。并将其放置在一个 for 循环中，每增加 10 秒。这似乎令人难以置信的沉重和缓慢。

非常感谢任何建议。谢谢。

样本数据：

3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009535350000078e+04    0.000000000000000000e+00
3.420009723610000219e+04    0.000000000000000000e+00
3.420009926809999888e+04    0.000000000000000000e+00
3.420019142339999962e+04    0.000000000000000000e+00
3.420025728430000163e+04    0.000000000000000000e+00
3.420025728430000163e+04    0.000000000000000000e+00
3.420025728430000163e+04    0.000000000000000000e+00
3.420025728430000163e+04    0.000000000000000000e+00
3.420025728430000163e+04    0.000000000000000000e+00
3.420025728430000163e+04    0.000000000000000000e+00
3.420025728430000163e+04    0.000000000000000000e+00
3.420025728430000163e+04    0.000000000000000000e+00
3.420025728430000163e+04    0.000000000000000000e+00
3.420025728430000163e+04    0.000000000000000000e+00
3.420025728430000163e+04    0.000000000000000000e+00
3.420025728430000163e+04    0.000000000000000000e+00
3.420025728430000163e+04    0.000000000000000000e+00
3.420025728430000163e+04    0.000000000000000000e+00
3.420025728430000163e+04    0.000000000000000000e+00
3.420030975760000001e+04    0.000000000000000000e+00
3.420031578240000090e+04    0.000000000000000000e+00
3.420033592689999932e+04    0.000000000000000000e+00
3.420046674769999663e+04    0.000000000000000000e+00
3.420046679600000061e+04    0.000000000000000000e+00
3.420074155890000111e+04    3.000000000000000000e+02
3.420074291200000152e+04    0.000000000000000000e+00
3.420076289620000171e+04    -3.000000000000000000e+02
3.420076459099999920e+04    2.000000000000000000e+02
3.420080930030000309e+04    0.000000000000000000e+00
3.420088430179999705e+04    0.000000000000000000e+00
3.420089157010000054e+04    6.000000000000000000e+00
3.420103050939999957e+04    3.000000000000000000e+02
3.420118753419999848e+04    3.000000000000000000e+02
3.420118888490000245e+04    0.000000000000000000e+00
3.420141332990000228e+04    0.000000000000000000e+00
3.420191020060000301e+04    0.000000000000000000e+00
3.420282872450000286e+04    6.000000000000000000e+00
3.420304732930000318e+04    0.000000000000000000e+00
3.420305702279999969e+04    0.000000000000000000e+00
3.420308584690000134e+04    0.000000000000000000e+00
3.420310770000000048e+04    0.000000000000000000e+00
3.420319724010000209e+04    0.000000000000000000e+00
3.420338125820000278e+04    -6.000000000000000000e+00
3.420341430170000240e+04    0.000000000000000000e+00
3.420341743809999753e+04    0.000000000000000000e+00
3.420365752020000218e+04    -1.000000000000000000e+02
3.420377651160000096e+04    0.000000000000000000e+00
3.420377900809999846e+04    0.000000000000000000e+00
3.420394856640000216e+04    0.000000000000000000e+00
3.420477753839999787e+04    0.000000000000000000e+00
3.420477777210000204e+04    0.000000000000000000e+00
3.420509855940000125e+04    0.000000000000000000e+00
3.420509857180000108e+04    0.000000000000000000e+00
3.420509858389999863e+04    0.000000000000000000e+00
3.420509858600000007e+04    0.000000000000000000e+00
3.420510251910000079e+04    3.000000000000000000e+02
3.420540111949999846e+04    0.000000000000000000e+00
3.420560046090000105e+04    0.000000000000000000e+00
3.420650914059999923e+04    3.000000000000000000e+02
3.420677362859999994e+04    0.000000000000000000e+00
3.420677385650000360e+04    0.000000000000000000e+00
3.420680100729999685e+04    0.000000000000000000e+00
3.420680135709999740e+04    0.000000000000000000e+00
3.420740102699999989e+04    0.000000000000000000e+00
3.420760059799999726e+04    0.000000000000000000e+00
3.420827570510000078e+04    0.000000000000000000e+00
3.420827846660000068e+04    -1.000000000000000000e+02
3.420860100809999858e+04    0.000000000000000000e+00
3.420860318459999689e+04    0.000000000000000000e+00
3.420860717129999830e+04    3.000000000000000000e+02
3.420920105229999899e+04    0.000000000000000000e+00
3.420949705469999753e+04    0.000000000000000000e+00
3.420949705469999753e+04    1.000000000000000000e+02
3.420949874789999740e+04    0.000000000000000000e+00
3.420949964329999784e+04    1.000000000000000000e+02
3.420950525599999673e+04    1.000000000000000000e+02
3.420950532879999810e+04    1.000000000000000000e+02
3.420950537430000259e+04    1.000000000000000000e+02
3.420950638910000271e+04    2.000000000000000000e+02
3.420950643239999772e+04    0.000000000000000000e+00
3.420950668009999936e+04    -2.000000000000000000e+02
3.420950796770000306e+04    0.000000000000000000e+00
3.420951590530000249e+04    2.000000000000000000e+02
3.420951602249999996e+04    1.000000000000000000e+02
3.420951673219999793e+04    -2.000000000000000000e+02
3.420951805140000215e+04    2.000000000000000000e+02
3.420951861829999689e+04    -1.000000000000000000e+02
3.420951993930000026e+04    -2.000000000000000000e+02
3.420951995310000348e+04    0.000000000000000000e+00
3.420952149579999968e+04    -1.000000000000000000e+02
3.420953443149999657e+04    0.000000000000000000e+00
3.420955497840000317e+04    -1.000000000000000000e+02
3.420980103039999813e+04    0.000000000000000000e+00
3.420980107300000236e+04    0.000000000000000000e+00
3.420980108509999991e+04    0.000000000000000000e+00
3.420989885620000132e+04    0.000000000000000000e+00
3.420989886560000014e+04    0.000000000000000000e+00
3.420989887540000200e+04    0.000000000000000000e+00
3.421022118190000037e+04    6.000000000000000000e+00
3.421028878499999701e+04    0.000000000000000000e+00
3.421029493570000341e+04    -6.000000000000000000e+00
3.421030143400000088e+04    6.000000000000000000e+00
3.421040105849999964e+04    0.000000000000000000e+00
3.421048329460000241e+04    -6.000000000000000000e+00
3.421048642470000050e+04    6.000000000000000000e+00
3.421085856629999762e+04    0.000000000000000000e+00
3.421091312309999921e+04    2.000000000000000000e+02
3.421091378969999641e+04    -2.000000000000000000e+02
3.421091715290000138e+04    1.000000000000000000e+02
3.421118993179999961e+04    0.000000000000000000e+00
3.421119293469999684e+04    6.000000000000000000e+00
3.421120108150000306e+04    0.000000000000000000e+00
3.421125067030000355e+04    1.000000000000000000e+02
3.421125067030000355e+04    2.000000000000000000e+02
3.421125072869999713e+04    0.000000000000000000e+00
3.421125074440000026e+04    1.000000000000000000e+02
3.421125075079999806e+04    1.000000000000000000e+02
3.421125081070000306e+04    0.000000000000000000e+00
3.421125213139999687e+04    -2.000000000000000000e+02
3.421125223679999908e+04    1.000000000000000000e+02
3.421128031339999870e+04    -1.000000000000000000e+02
3.421128036189999693e+04    -1.000000000000000000e+02
3.421129058039999654e+04    0.000000000000000000e+00
3.421136299729999882e+04    -6.000000000000000000e+00
3.421136675249999826e+04    6.000000000000000000e+00
3.421144104959999822e+04    -6.000000000000000000e+00
3.421144451439999830e+04    6.000000000000000000e+00
3.421180177070000354e+04    1.000000000000000000e+02
3.421180182290000084e+04    1.000000000000000000e+02
3.421180183370000304e+04    1.000000000000000000e+02
3.421180184079999890e+04    0.000000000000000000e+00
3.421180240860000049e+04    -2.000000000000000000e+02
3.421182461470000271e+04    -1.000000000000000000e+02
3.421182466330000170e+04    -1.000000000000000000e+02
3.421183488509999734e+04    0.000000000000000000e+00
3.421209945889999653e+04    7.500000000000000000e+01
3.421213015650000307e+04    1.000000000000000000e+02
3.421250285479999729e+04    2.500000000000000000e+01
3.421250285479999729e+04    3.000000000000000000e+02
3.421250292720000289e+04    1.000000000000000000e+02
3.421250293499999680e+04    1.000000000000000000e+02
3.421250294089999807e+04    0.000000000000000000e+00
3.421250461730000097e+04    2.000000000000000000e+02
3.421250660069999867e+04    2.000000000000000000e+02
3.421250661260000197e+04    0.000000000000000000e+00
3.421250662810000358e+04    -2.000000000000000000e+02
3.421250862849999976e+04    2.000000000000000000e+02
3.421250865109999722e+04    -2.000000000000000000e+02
3.421251071360000060e+04    2.000000000000000000e+02
3.421251073390000238e+04    -2.000000000000000000e+02
3.421251200200000312e+04    2.000000000000000000e+02
3.421251202500000363e+04    -2.000000000000000000e+02
3.421251290870000230e+04    2.000000000000000000e+02
3.421251292259999900e+04    0.000000000000000000e+00
3.421251417010000296e+04    0.000000000000000000e+00
3.421252315950000047e+04    -3.750000000000000000e+02
3.421253324080000311e+04    -1.000000000000000000e+02
3.421253324419999990e+04    -1.000000000000000000e+02
3.421253325169999880e+04    0.000000000000000000e+00
3.421280107870000211e+04    0.000000000000000000e+00
3.421280108050000126e+04    0.000000000000000000e+00
3.421280109909999737e+04    0.000000000000000000e+00
3.421300649369999883e+04    -1.000000000000000000e+02
3.421300649369999883e+04    -5.000000000000000000e+00
3.421301662930000020e+04    0.000000000000000000e+00
3.421301664270000038e+04    0.000000000000000000e+00
3.421308269650000148e+04    -1.000000000000000000e+00
3.421308569990000251e+04    1.000000000000000000e+00
3.421340113990000100e+04    -2.000000000000000000e+02
3.421360080849999940e+04    0.000000000000000000e+00
3.421360081950000313e+04    0.000000000000000000e+00
3.421429604950000066e+04    0.000000000000000000e+00
3.421429604950000066e+04    2.000000000000000000e+02
3.421465075329999672e+04    1.750000000000000000e+02
3.421465535619999719e+04    0.000000000000000000e+00
3.421480113819999679e+04    2.000000000000000000e+02
3.421500744149999809e+04    2.000000000000000000e+02
3.421500745640000241e+04    -2.000000000000000000e+02
3.421501012210000044e+04    0.000000000000000000e+00
3.421501918660000229e+04    2.000000000000000000e+02
3.421506023750000168e+04    2.000000000000000000e+02
3.421506026279999787e+04    0.000000000000000000e+00
3.421506290029999946e+04    0.000000000000000000e+00
3.421529837140000018e+04    -2.000000000000000000e+02
3.421540116690000286e+04    0.000000000000000000e+00
3.421549945029999799e+04    0.000000000000000000e+00
3.421554515879999963e+04    0.000000000000000000e+00
3.421556687119999697e+04    -3.000000000000000000e+02
3.421556832669999858e+04    0.000000000000000000e+00
3.421560137959999702e+04    0.000000000000000000e+00
3.421591907280000305e+04    0.000000000000000000e+00
3.421640112349999981e+04    0.000000000000000000e+00
3.421660092140000052e+04    0.000000000000000000e+00
3.421780116010000347e+04    0.000000000000000000e+00
3.421860098989999824e+04    0.000000000000000000e+00
3.422180125080000289e+04    0.000000000000000000e+00
3.422181661359999998e+04    0.000000000000000000e+00
3.422946705189999921e+04    0.000000000000000000e+00
3.422947095989999798e+04    3.000000000000000000e+02
3.422947237350000069e+04    -2.000000000000000000e+02
3.422980151579999801e+04    0.000000000000000000e+00
3.423008524700000271e+04    0.000000000000000000e+00
3.423035811159999867e+04    0.000000000000000000e+00
3.423036595089999901e+04    0.000000000000000000e+00
3.423315292909999698e+04    -1.000000000000000000e+00
3.423315298620000249e+04    -1.000000000000000000e+02
3.423315299590000359e+04    -1.000000000000000000e+02
3.423315301110000291e+04    0.000000000000000000e+00
3.423315437930000189e+04    0.000000000000000000e+00
3.423315469930000108e+04    -2.000000000000000000e+02
3.423315568000000349e+04    0.000000000000000000e+00
3.423317673620000278e+04    1.000000000000000000e+02
3.423318700569999783e+04    1.000000000000000000e+02

【问题讨论】：

您可能想考虑使用pandas，它内置了吨时间序列功能。 pandas.pydata.org/pandas-docs/stable/timeseries.html
是的。我会使用df.set_index('my_time_column', inplace=True)。然后你可以这样做：df.groupby(df.index.seconds).sum()
你能解释一下你是怎么得到这样一个时间戳的吗？它看起来不像一个标准的 UNIX 纪元（# of (nano-), (micro-), (mili-), seconds）
@python_newbie，从当前午夜开始？
@python_newbie 是的，所以，实际上，它应该是没有“s”的a.index.second，但这只适用于DatetimeIndex。但是，如果您只有浮点值，请尝试使用 a.groupby(df['a'] // 10).sum() 注意，// 只是整数除法，所以 123.456 // 10 == 12.0，所以基本上，同一 10 秒槽内的所有时间都将组合在一起。编辑我看到 MaxU 已经击败了我......

标签： python numpy time-series aggregate

【解决方案1】：

考虑以下方法：

import pandas as pd

df = pd.read_csv(filename, header=None, names=['ts','val'])

In [31]: df.groupby(df['ts']//10*10)['val'].sum()
Out[31]:
ts
34200.0    1806.0
34210.0     907.0
34220.0     100.0
34230.0    -201.0
Name: val, dtype: float64

来源 DF：

In [33]: df
Out[33]:
               ts    val
0    34200.095354    0.0
1    34200.095354    0.0
2    34200.095354    0.0
3    34200.095354    0.0
4    34200.095354    0.0
5    34200.095354    0.0
6    34200.095354    0.0
7    34200.095354    0.0
8    34200.095354    0.0
9    34200.095354    0.0
..            ...    ...
237  34230.365951    0.0
238  34233.152929   -1.0
239  34233.152986 -100.0
240  34233.152996 -100.0
241  34233.153011    0.0
242  34233.154379    0.0
243  34233.154699 -200.0
244  34233.155680    0.0
245  34233.176736  100.0
246  34233.187006  100.0

[247 rows x 2 columns]

df['ts']//10*10 - 将时间戳取整为 10 秒，因此我们可以稍后将其用于分组：

In [34]: df['ts']//10*10
Out[34]:
0      34200.0
1      34200.0
2      34200.0
3      34200.0
4      34200.0
5      34200.0
6      34200.0
7      34200.0
8      34200.0
9      34200.0
        ...
237    34230.0
238    34230.0
239    34230.0
240    34230.0
241    34230.0
242    34230.0
243    34230.0
244    34230.0
245    34230.0
246    34230.0
Name: ts, Length: 247, dtype: float64

【讨论】：

@python_newbie，我在答案中添加了一个简短的解释 - 请检查
知道了！谢谢！！