Python CSV：使用时间值条件获取行中的所有值

Question

CSV 数据的示例，条件是我正在尝试获取：

c1,c2,v1,v2,p1,p2,r1,a1,f1,f2,f3,Time_Stamp 

0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:00
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:01
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:02
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:03
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:04
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:05
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:06
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:07
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:08
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:09
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:10
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:11
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:12
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:13
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:14
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:15
415.7,12.5,30.2,154.6,4675.2,1,-1,5199.4,0,50,0,13/06/2017 16:38:16 
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:17
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:18
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:19
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:20
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:21

读取csv的代码：

import plotly
import plotly.plotly as py
import plotly.graph_objs as go
import plotly.figure_factory as FF
import numpy as np
from datetime import date,time,datetime
import pandas as pd
%matplotlib inline
import matplotlib.pyplot as plt

def readcsv(x): #def function to read csv files based on code below*
    Data = pd.read_csv(x, parse_dates=['Time_Stamp'], infer_datetime_format=True)
    Data['Date'] = Data.Time_Stamp.dt.date #date column in DataFrame
    Data['Time'] = Data.Time_Stamp.dt.time #time column in DataFrame

    Data['Time_Stamp'] = pd.to_datetime(Data['Time_Stamp'])
    print(Data[1:6])
    return Data

Data = readcsv('datafile.csv')#*

def getMask(start,end,Data):
    mask = (Data['Time_Stamp'] > start) & (Data['Time_Stamp'] <= end)
    return mask;

start = '2017-06-13 16:00:00'
end = '2017-06-13 16:40:00'
timerange = Data.loc[getMask(start, end, Data)] #* <----  using this Dataframe
#timeR.plot(x='Time_Stamp', y='AC_Input_Current', style='-', color='black')

我想要得到的是：

[例如]执行pspike（代码如下）后，我将得到以下输出：

13/06/2017 16:38:00
13/06/2017 16:38:01
13/06/2017 16:38:02
13/06/2017 16:38:03
13/06/2017 16:38:04
13/06/2017 16:38:05
13/06/2017 16:38:06
13/06/2017 16:38:07
13/06/2017 16:38:08
13/06/2017 16:38:09
13/06/2017 16:38:10
13/06/2017 16:38:11
13/06/2017 16:38:12
13/06/2017 16:38:13
13/06/2017 16:38:14
13/06/2017 16:38:15
13/06/2017 16:38:17
13/06/2017 16:38:18
13/06/2017 16:38:19
13/06/2017 16:38:20
13/06/2017 16:38:21   

*请注意，我正在使用数据帧timerange，其中每秒的Time 值从16:00:00 到16:40:00，以获取pspike，如果c1，它会跳过行值为

[来自输出print(pspike)] 条件：如果打印的行其中Time 的值为16:38:15，并且下一行的Time 值为16:38:17（其中下一行的Time 值跳过了1 秒）.. . 打印被跳过的行（在这种情况下，Time 的值是16:38:16）

pspike = (timerange.loc[timerange['AC_Input_Current'] <= 5.0])    
print(pspike)

with open('welding_data_by_selRange.csv','a', newline='') as duraweld:
    a = csv.writer(duraweld)
    data = [countIC2 ,countIC, Datetime]
    a.writerow(data)

Answer 1

更新：

以下代码将打印丢失的时间戳，无论缺少多少时间戳，因此它比以前的解决方案更健壮。

for i in range(df.shape[0] - 1):
    row1 = df.iloc[i]
    row2 = df.iloc[i+1]
    skipped_ts = (row2[-1] - row1[-1]).seconds
    if  skipped_ts > 1:
    for ts in range(1,skipped_ts):
        print (row1[-1] + pd.Timedelta(ts * '1s'))

Answer 2

非 Pandas 解决方案

从每一行中提取时间戳和其他信息；使用格式字符串将时间戳转换为 datetime.datetime 对象；从前一个时间戳中减去当前时间戳；测试经过的时间和过程（如果适用）。

import datetime, io

#setup
s = '''0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:12
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:13
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:15
0,2.3,0.6,-0.9,-0.5,1,-1,941.0,0,50,0,13/06/2017 16:38:16
'''
#data is a file-like object
data = io.StringIO(s)

fmt = '%d/%m/%Y %H:%M:%S'
previous = None

for row in data:
    *info, timestamp = row.strip().split(',')
    timestamp = datetime.datetime.strptime(timestamp, fmt)
    try:
        dt = timestamp-previous[0]
    except TypeError as e:
        previous = (timestamp, info)
        continue
    if dt.seconds > 1:
        print('!!!\tprevious:{}\n\tcurrent:{}'.format(previous,(timestamp, info)))
    previous = (timestamp, info)

它可以适用于 csv.reader。

---

时间戳最初是通过拆分从行中的最后一列获得的。然后将其制成一个 datetime.datetime 对象，以便轻松计算时间差。

对于磁盘文件，打开它并对其进行迭代...

with open(filepath) as data:
    for row in data:
        *info, timestamp = row.strip().split(',')
        timestamp = datetime.datetime.strptime(timestamp, fmt)
        ....

使用 csv 阅读器：

import csv
with open(filepath) as data:
    rows = csv.reader(data)
    for row in rows:
        *info, timestamp = row
        timestamp = datetime.datetime.strptime(timestamp, fmt)
        ....

如果您可以将整个文件读入数据框，那么您应该可以将其读入变量。

with open(filepath) as f:
    data = f.read()

for row in data:
    *info, timestamp = row.strip().split(',')
    timestamp = datetime.datetime.strptime(timestamp, fmt)
    ....