比较两个文件的代码很慢 - 搜索以改进我的代码

Question

我不是 Python 专家，我编写了一个脚本来比较两个文件（地震数据位置）。我写的很丑而且很慢。 有人有改进我的代码的想法吗？谢谢！

#!/usr/bin/env python
# -*- coding: utf-8 -*-

file_1 = 'Loc_1D.txt'
file_2 = 'Loc_3D.txt'
output_file = 'result_file.txt'

with open(file_1, "r") as f1:
    for line1 in f1:
        yr1, mth1, d1, hr1, m1, s1, lat1, lon1, z1, mag  = line1.split()
        time1 = [yr1, mth1, d1, hr1, m1]
        with open(file_2, "r") as f2:
            for line2 in f2:
                yr2, mth2, d2, hr2, m2, s2, lat2, lon2, z2, *_ = line2.split()
                time2 = [yr2, mth2, d2, hr2, m2]
                with open(output_file, "w") as oup:
                    if time1 == time2 and abs(float(s1)-float(s2)) <= 2:
                        Event = [yr2, mth2, d2, hr2, m2, s2, lat2, lon2, z2, mag]
                        print (Event)
                        oup.write(str(Event))

Answer 1

这类似于马特的评论。假设时间戳在所有情况下都是唯一的，这可能是最有效的解决方案：

file_1 = 'Loc_1D.txt'
file_2 = 'Loc_3D.txt'
output_file = 'result_file.txt'
with open(file_1) as f1:
    f1_data = {}
    for line in f1.read().split("\n"):
        line_data = line.split()
        f1_data["-".join(map(str, line_data[:5]))] = line_data

with open(file_2) as f2:
    f2_data = {}
    for line in f2.read().split("\n"):
        line_data = line.split()
        f2_data["-".join(map(str, line_data[:5]))] = line_data

output_data = []
for data_key in [key for key in f1_data.keys() if key in f2_data.keys()]:
    if abs(float(f1_data[data_key][5])-float(f2_data[data_key][5])) <= 2.0:
        Event = str(f2_data[data_key][5])
        print(Event)
        output_data.append(Event)

with open(output_file, 'w') as f:
    f.write("\n".join(output_data))

原因是如果您考虑使用大 O 表示法的效率，您正在从 O(n^2) 转换为 O(3n) 效率。 简单来说就是迭代次数减少。例如，如果每个文件有 100 行数据，则计算机必须在前一种情况下处理大约 10000 行，在这种情况下处理 300 行。

Answer 2

这是更正，感谢@Samay Gupta

file_1 = 'Loc_1D.txt'
file_2 = 'Loc_3D.txt'
output_file = 'result_file.txt'

with open(file_1) as f1:
    f1_data = {}
    for line in f1.read().split("\n"):
        line_data = line.split()
        f1_data["-".join(map(str, line_data[:5]))] = line_data

with open(file_2) as f2:
    f2_data = {}
    for line in f2.read().split("\n"):
        line_data = line.split()
        f2_data["-".join(map(str, line_data[:5]))] = line_data

output_data = []
for data_key in [key for key in f1_data.keys() if key in f2_data.keys()]:
    if abs(float(f1_data[data_key][5])-float(f2_data[data_key][5])) <= 2.0:
        Event = f2_data[data_key][:9], f1_data[data_key][9]
        print(Event)
        output_data.append(Event)

with open(output_file, 'w') as f:
    f.write("\n".join(str(output_data)))