逐行读取多个文件，如果时间不等，转到一个文件中的下一行（python3）

Question

我想逐行读取多个文件，如果该行的时间戳不适合其他文件的时间戳，我想跳到该文件的下一行，同时将另一个文件保留在同一行，直到它们匹配。

伪

read two files
for file0line, file1line in zip(file0, file1):
    # compare time file0line with time file1line
    if file0line < file1line:
        next file0line
        # compare again while file1line stay the same
    if file1line < file0line:
        next file1line
        # compare again while file0line stay the same
    else:
        # they are synced in time

我的代码：

class MyDialect(csv.Dialect):
    strict = True
    skipinitialspace = True
    quoting = csv.QUOTE_ALL
    delimiter = ','
    quotechar = '"'
    lineterminator = '\n'

files = [open(filename) for filename in accelerometers]
for line0, line1 in zip_longest(*files, fillvalue=''):
    b0 = io.StringIO(line0)
    b1 = io.StringIO(line1)
    r0 = csv.reader(b0, MyDialect())
    r1 = csv.reader(b1, MyDialect())
    for i0, i1 in zip(r0, r1):
        time0 = datetime.strptime(i0[0], "%Y-%m-%d %H:%M:%S.%f")
        time1 = datetime.strptime(i1[0], "%Y-%m-%d %H:%M:%S.%f")
        print(time0, time1)
        if time0 != time1:
            if time0 < time1:
                next(line0)
            if time1 < time0:
                next(line1)
        else:
            print(time0, time1)

文件示例1

Time,Accel-X (g), Accel-Y (g), Accel-Z (g)
2017-03-17 15:41:20.534,0.468750,0.734375,-0.625000
2017-03-17 15:41:20.545,0.218750,0.437500,0.921631
2017-03-17 15:41:20.555,0.281006,0.375000,0.999756
2017-03-17 15:41:20.565,0.343506,0.296875,1.031006
2017-03-17 15:41:20.575,0.328125,0.296875,1.031006
2017-03-17 15:41:20.584,0.328125,0.281250,1.046631
2017-03-17 15:41:20.594,0.296875,0.281006,1.031250
2017-03-17 15:41:20.604,0.281250,0.312256,1.031006
2017-03-17 15:41:20.614,0.250000,0.327881,1.046631
2017-03-17 15:41:20.625,0.249756,0.327881,1.031250
2017-03-17 15:41:20.635,0.250000,0.343506,1.031006
2017-03-17 15:41:20.645,0.249756,0.374756,1.031250
2017-03-17 15:41:20.655,0.265381,0.421631,1.000000
2017-03-17 15:41:20.665,0.281250,0.453125,0.984131
2017-03-17 15:41:20.674,0.296875,0.484375,0.921631
2017-03-17 15:41:20.684,0.312500,0.515625,0.921631
2017-03-17 15:41:20.694,0.328125,0.562500,0.921875
2017-03-17 15:41:20.704,0.359375,0.593750,0.937500
2017-03-17 15:41:20.715,0.375000,0.625000,0.937256
2017-03-17 15:41:20.725,0.406250,0.656250,0.890381

文件示例2

Time,Accel-X (g), Accel-Y (g), Accel-Z (g)
2017-03-17 15:41:16.514,-0.781250,0.250000,0.500000
2017-03-17 15:41:16.524,-0.468750,-0.234131,0.499756
2017-03-17 15:41:16.534,-0.484131,-0.250000,0.515625
2017-03-17 15:41:16.545,-0.499756,-0.218750,0.484375
2017-03-17 15:41:16.555,-0.515381,-0.203125,0.390625
2017-03-17 15:41:16.565,-0.500000,-0.156250,0.437256
2017-03-17 15:41:16.575,-0.468750,-0.171631,0.468506
2017-03-17 15:41:16.584,-0.452881,-0.156250,0.468506
2017-03-17 15:41:16.594,-0.515381,-0.125000,0.515381
2017-03-17 15:41:16.604,-0.546631,-0.124756,0.515381
2017-03-17 15:41:16.614,-0.546875,-0.156006,0.577881
2017-03-17 15:41:16.625,-0.500000,-0.202881,0.656006
2017-03-17 15:41:16.635,-0.468750,-0.265381,0.687256
2017-03-17 15:41:16.645,-0.452881,-0.312500,0.671875
2017-03-17 15:41:16.655,-0.437500,-0.375000,0.703125
2017-03-17 15:41:16.665,-0.452881,-0.437500,0.718750
2017-03-17 15:41:16.674,-0.437500,-0.500000,0.781250
2017-03-17 15:41:16.684,-0.468750,-0.484131,0.796875
2017-03-17 15:41:16.694,-0.500000,-0.468506,0.796875
2017-03-17 15:41:16.704,-0.500000,-0.421631,0.843750

Answer 1

Zip 将始终从每个生成器中选择一个结果 - 无论是文件还是 CSV 阅读器。

因此，对于每组行，检查最大时间戳，如果任何行的日期时间较小，则只读取该行，直到所有行的时间戳相等。

我为 n 个文件编写了代码，其中任何一个都允许领先 - 所以乍一看它比你的版本要复杂一些。但是，将其封装在生成器中可以非常简单地使用：

def getdate(f):
    return datetime.strptime(f, "%Y-%m-%d %H:%M:%S.%f")

def sync_line_reader(*readers, skip_lines=1):
    def get_one_line_of_each(readers):
        lines = []
        for reader in readers:
            lines.append(next(reader))


    try:
        #read first lines, skipping the header:
        for _ in range(skip_lines):
            lines = get_one_line_of_each(*readers)
        while True:
            max_date = max(get_date(f[0]) for f in lines)
            for i, line in enumerate(lines[:]):
                while get_date(line[0]) < max_date:
                    lines[i] = line = next(readers[i])
            yield lines
            lines = get_one_line_of_each(readers)    
    except StopIteration:
        return

您可以将其插入代码中：

...
files = [open(filename) for filename in accelerometers]
for line0, line1 in zip_longest(*files, fillvalue=''):
    b0 = io.StringIO(line0)
    b1 = io.StringIO(line1)
    r0 = csv.reader(b0, MyDialect())
    r1 = csv.reader(b1, MyDialect())
    for i0, i1 in sync_line_reader((r0, r1)):
         print(get_time(i0[0]), get_time(i1[0]))