按不同条件从 DataFrame 中删除行答案

【问题标题】：Dismiss rows from DataFrame by condition in different one按不同条件从 DataFrame 中删除行
【发布时间】：2016-10-25 12:17:18
【问题描述】：

我有两组 csv，它们始终具有不同的时间频率——即每 5 分钟测量一次，然后每小时测量一次，等等。

我想要做的是第二个 csv (column 2)，如果在 hour 的任何地方有一个大于 190 的值，然后摆脱 CSV 一个小时

有没有一种神奇的方式使用 Pandas 来做到这一点？我正在考虑将条件设置为 true 和 false 作为索引，然后以此为第一个 CSV 数据计时。但我认为为此，它们需要完全相同的数据间隔。

CSV1 有数据类型（日期、A、B、C、D、E、F、G、H）：

24-jan-08 23:50,  -8.6,  7.7, 0.0213,  .9820, 0.0213, 1.6316, 1.00,46.810
24-jan-08 23:55,  -6.7,  7.7, 0.0213,  .9824, 0.0213, 1.6321, 1.00,46.802
25-jan-08 00:00,  -1.7,  7.7, 0.0213,  .9828, 0.0213, 1.6328, 1.00,46.799
25-jan-08 00:05,   -32,  7.7, 0.0213,  .9835, 0.0213, 1.6334, 1.00,46.757
25-jan-08 00:10, -11.1,  7.7, 0.0213,  .9842, 0.0213, 1.6342, 1.00,46.742

等等，但如前所述，从 5 分钟到每小时，但 CSV 文件太大，无法在此处发布

CSV2 有数据类型 (Date,A,B)：

2008-01-24 23:50,6.55,186.9
2008-01-24 23:51,6.84,188.6  
2008-01-24 23:52,7.14,188.1
2008-01-24 23:53,7.12,189.9
2008-01-24 23:54,7.45,188.6
2008-01-24 23:55,7.52,190.5
2008-01-24 23:56,7.29,189.5
2008-01-24 23:57,7.07,192.4
2008-01-24 23:58,7.33,193.7
2008-01-24 23:59,7.25,192.6
2008-01-25 00:02,6.52,191
2008-01-25 00:03,6.58,189
2008-01-25 00:04,6.43,190.5
2008-01-25 00:05,6.6,188.3
2008-01-25 00:06,6.52,188.7
2008-01-25 00:07,6.75,188.9
2008-01-25 00:08,6.62,188.9
2008-01-25 00:09,6.26,188.8
2008-01-25 00:10,6.6,193.2

190 完全是任意的，需要选择一个适合完整数据集的数字

【问题讨论】：

所以在第一个数据集中，第二个样本将被删除所有数据，因为在第 23 小时和第 00 小时的值高于 190？
是的，所以如果在第二个样本的每个小时内，有一个结果超过 190，它将删除第一个样本中的那几个小时的数据集

标签： python csv pandas dataframe timestamp

【解决方案1】：

设置双read_csv:

import pandas as pd
import io

temp=u"""24-jan-08 23:50,-8.6,7.7,0.0213,.9820,0.0213,1.6316,1.00,46.810
24-jan-08 23:55,-6.7,7.7,0.0213,.9824,0.0213,1.6321,1.00,46.802
25-jan-08 00:00,-1.7,7.7,0.0213,.9828,0.0213,1.6328,1.00,46.799
25-jan-08 00:05,-32,7.7,0.0213,.9835,0.0213,1.6334,1.00,46.757
25-jan-08 00:10,-11.1,7.7,0.0213,.9842,0.0213,1.6342,1.00,46.742"""
#after testing replace io.StringIO(temp) to filename
df1 = pd.read_csv(io.StringIO(temp), parse_dates=[0], names=['Date','A','B','C','D','E','F','G', 'H'])

temp=u"""
2008-01-24 23:50,6.55,186.9
2008-01-24 23:51,6.84,188.6
2008-01-24 23:52,7.14,188.1
2008-01-24 23:53,7.12,189.9
2008-01-24 23:54,7.45,188.6
2008-01-24 23:55,7.52,190.5
2008-01-24 23:56,7.29,189.5
2008-01-24 23:57,7.07,192.4
2008-01-24 23:58,7.33,193.7
2008-01-24 23:59,7.25,192.6
2008-01-25 00:02,6.52,191
2008-01-25 00:03,6.58,189
2008-01-25 00:04,6.43,190.5
2008-01-25 00:05,6.6,188.3
2008-01-25 00:06,6.52,188.7
2008-01-25 00:07,6.75,188.9
2008-01-25 00:08,6.62,188.9
2008-01-25 00:09,6.26,188.8
2008-01-25 00:10,6.6,193.2"""
#after testing replace io.StringIO(temp) to filename
df2 = pd.read_csv(io.StringIO(temp), parse_dates=[0],names=['Date','A','B'])

print (df1)
                 Date     A    B        C       D       E       F    G       H
0 2008-01-24 23:50:00  -8.6  7.7   0.0213  0.9820  0.0213  1.6316  1.0  46.810
1 2008-01-24 23:55:00  -6.7  7.7   0.0213  0.9824  0.0213  1.6321  1.0  46.802
2 2008-01-25 00:00:00  -1.7  7.7   0.0213  0.9828  0.0213  1.6328  1.0  46.799
3 2008-01-25 00:05:00 -32.0  7.7   0.0213  0.9835  0.0213  1.6334  1.0  46.757
4 2008-01-25 00:10:00 -11.1  7.7   0.0213  0.9842  0.0213  1.6342  1.0  46.742

print (df2)
                  Date     A      B
0  2008-01-24 23:50:00  6.55  186.9
1  2008-01-24 23:51:00  6.84  188.6
2  2008-01-24 23:52:00  7.14  188.1
3  2008-01-24 23:53:00  7.12  189.9
4  2008-01-24 23:54:00  7.45  188.6
5  2008-01-24 23:55:00  7.52  190.5
6  2008-01-24 23:56:00  7.29  189.5
7  2008-01-24 23:57:00  7.07  192.4
8  2008-01-24 23:58:00  7.33  193.7
9  2008-01-24 23:59:00  7.25  192.6
10 2008-01-25 00:02:00  6.52  191.0
11 2008-01-25 00:03:00  6.58  189.0
12 2008-01-25 00:04:00  6.43  190.5
13 2008-01-25 00:05:00  6.60  188.3
14 2008-01-25 00:06:00  6.52  188.7
15 2008-01-25 00:07:00  6.75  188.9
16 2008-01-25 00:08:00  6.62  188.9
17 2008-01-25 00:09:00  6.26  188.8
18 2008-01-25 00:10:00  6.60  193.2

可以先转换列Dateto_period：

df1.index = df1['Date'].dt.to_period('h')
df2['per'] = df2['Date'].dt.to_period('h')

print (df1)

                                Date     A    B        C       D       E  \
Date                                                                       
2008-01-24 23:00 2008-01-24 23:50:00  -8.6  7.7   0.0213  0.9820  0.0213   
2008-01-24 23:00 2008-01-24 23:55:00  -6.7  7.7   0.0213  0.9824  0.0213   
2008-01-25 00:00 2008-01-25 00:00:00  -1.7  7.7   0.0213  0.9828  0.0213   
2008-01-25 00:00 2008-01-25 00:05:00 -32.0  7.7   0.0213  0.9835  0.0213   
2008-01-25 00:00 2008-01-25 00:10:00 -11.1  7.7  ;0.0213  0.9842  0.0213   

                       F    G       H  
Date                                   
2008-01-24 23:00  1.6316  1.0  46.810  
2008-01-24 23:00  1.6321  1.0  46.802  
2008-01-25 00:00  1.6328  1.0  46.799  
2008-01-25 00:00  1.6334  1.0  46.757  
2008-01-25 00:00  1.6342  1.0  46.742

print (df2)
                  Date     A      B              per
0  2008-01-24 23:50:00  6.55  186.9 2008-01-24 23:00
1  2008-01-24 23:51:00  6.84  188.6 2008-01-24 23:00
2  2008-01-24 23:52:00  7.14  188.1 2008-01-24 23:00
3  2008-01-24 23:53:00  7.12  189.9 2008-01-24 23:00
4  2008-01-24 23:54:00  7.45  188.6 2008-01-24 23:00
5  2008-01-24 23:55:00  7.52  190.5 2008-01-24 23:00
6  2008-01-24 23:56:00  7.29  189.5 2008-01-24 23:00
7  2008-01-24 23:57:00  7.07  192.4 2008-01-24 23:00
8  2008-01-24 23:58:00  7.33  193.7 2008-01-24 23:00
9  2008-01-24 23:59:00  7.25  192.6 2008-01-24 23:00
10 2008-01-25 00:02:00  6.52  191.0 2008-01-25 00:00
11 2008-01-25 00:03:00  6.58  189.0 2008-01-25 00:00
12 2008-01-25 00:04:00  6.43  190.5 2008-01-25 00:00
13 2008-01-25 00:05:00  6.60  188.3 2008-01-25 00:00
14 2008-01-25 00:06:00  6.52  188.7 2008-01-25 00:00
15 2008-01-25 00:07:00  6.75  188.9 2008-01-25 00:00
16 2008-01-25 00:08:00  6.62  188.9 2008-01-25 00:00
17 2008-01-25 00:09:00  6.26  188.8 2008-01-25 00:00
18 2008-01-25 00:10:00  6.60  193.2 2008-01-25 00:00

然后通过条件找到uniqueperiods：

pers = df2.loc[df2.B > 190, 'per'].unique()
print (pers)
[Period('2008-01-24 23:00', 'H') Period('2008-01-25 00:00', 'H')]

最后dropdf1中的所有行：

print (df1.drop(pers))
Empty DataFrame
Columns: [Date, A, B, C, D, E, F, G]
Index: []

通过评论编辑：

如果 df1 和 df2 有 DatetimeIndex 使用：

df1.index = df1.index.to_period('h')
df2['per'] = df2.index.to_period('h')

【讨论】：

你是如何得到：df1.index = df1.Date.dt.to_period('h') 工作的？我得到 DataFrame 对象没有属性“日期”它认为日期列不是列，但标题从“A”开始
好的，我发现代码中有错误 - 您需要将列 H 添加到 df1。也请使用read_csv 查看解决方案。
名称是任意的：实际上我应该说：aethalometer=['Date','Conc','Flow','SZ','SB','RZ','RB' ,'Fraction','Attenuation'] df1=pd.read_csv('Output14.csv', index_col=0, names=aethalometer, parse_dates=True) 但是再做 df1.Date 会出现上述错误
是的，我想我明白了——你需要使用[]df1.index = df1['Date'].dt.to_period('h') df2['per'] = df2['Date'].dt.to_period('h')，见编辑。
names=['Date','Wind Speed','Wind Direction'] df2 = pd.read_csv(io.StringIO('all_test.csv'), index_col=0, names=names, parse_dates=True) aethalometer=['Date','Conc','Flow','SZ','SB','RZ','RB','Fraction','Attenuation'] df1=pd.read_csv(io.StringIO('Output14.csv'), index_col=0, names=aethalometer, parse_dates=True) df1.index = df1['Date'].dt.to_period('h') df2['per'] = df2['Date'].dt.to_period('h')