比较两个文本文件 - 逐行比较（涉及屏蔽） - python

Question

我正在从事一个涉及解析、比较和验证两个长文本的项目——其中一些是数千行文本。这些文件确实有共同的行和模式，但总体上有所不同。我感兴趣的是两个文件中的独特行。以下场景就是一个很好的例子：

文件1 -

- This file is located in 3000.3422.63.34 description "the mother of all files"
- City address of file is "Melbourne"
- Country of file is Australia

文件2-

 -This file is located in 3000.3422.62.89 description "the brother of all good files"
 - City address of file is "Sydney"
 - This file spent sometime in "Gold Coast"
 - Country of file is Australia

任务是使用 file1 作为参考来验证 file2 - 使用模式检查。 我想屏蔽两个文件的共同模式（见下文）并进行比较。

  - This is the first file located in 3000.3422.xxxx.xxxx description "xxxx"
  - City address of file is "xxxx"
  - Country of file is xxxx

使用这个逻辑。第二个文件有一个独特的行，我将其导出到报告函数：

   - This file spent sometime in "Gold Coast"

我怎样才能轻松地[在两个文件上] 进行动态屏蔽 - 感谢您的帮助？

Answer 1

这就是答案——终于自己破解了——:)

import os
import sys
import re
import webbrowser

比较函数 - 逐行：

def CompareFiles(str_file1,str_file2):
    '''
    This function compares two long string texts and returns their 
    differences as two sequences of unique lines, one list for each.
    '''
    #reading from text file and splitting str_file into lines - delimited by "\n"
    file1_lines = str_file1.split("\n")
    file2_lines = str_file2.split("\n")

    #unique lines to each one, store it in their respective lists
    unique_file1 = []
    unique_file2 = []

    #unique lines in str1
    for line1 in file1_lines:
        if line1 !='':
           if line1 not in file2_lines:
              unique_file1.append(line1)

    #unique lines in str2
    for line2 in file2_lines:
        if line2 != '':
           if line2 not in file1_lines:
              unique_file2.append(line2)

    return unique_file1, unique_file2

使用这个函数来屏蔽：

def Masker(pattern_lines, file2mask):
    '''
    This function masks some fields (based on the pattern_lines) with 
    dummy text to simplify the comparison
    '''
    #mask the values of all matches from the pattern_lines by a dummy data - 'xxxxxxxxxx'
    for pattern in pattern_lines:
        temp = pattern.findall(file2mask)
        if len(temp) != 0:
           for value in temp:
               if isinstance(value, str):
                  masked_file = file2mask.replace(str(value),'x'*10)
               elif isinstance(value, tuple):
                    for tup in value:
                        masked_file = file2mask.replace(str(tup),'x'*10)
    return masked_file

打开文件：

f1 = open("file1.txt","r")
data1 = f1.read()
f1.close()

f3 = open("file2.txt","r")
data3 = f3.read()
f3.close()

创建一个文件夹来存储输出文件（可选）：

save_path = os.path.join(os.path.dirname(__file__), 'outputs')
filename = os.path.normpath(os.path.join(save_path,"interim.txt"))

用于掩蔽的图案线：

pattern_lines = [
    re.compile(r'\- This file is located in 3000.3422.(.*) description \"(.*)\"', re.M),
    re.compile(r'\- City address of file is \"(.*)\"',re.M),
    re.compile(r'\- Country of file is (.*)',re.M)
]

屏蔽这两个文件：

data1_masked = Masker(pattern_lines,data1)
data3_masked = Masker(pattern_lines,data3)

比较两个文件并返回两个文件的唯一行

unique1, unique2 = CompareFiles(data1_masked, data3_masked)

报告——你可以把它写成一个函数：

file = open(filename,'w')
file.write("-------------------------\n")
file.write("\nONLY in FILE ONE\n")
file.write("\n-------------------------\n")
file.write(str('\n'.join(unique1)))
file.write("\n-------------------------\n")
file.write("\nONLY in FILE TWO\n")
file.write("\n-------------------------\n")
file.write(str('\n'.join(unique2)))
file.close()

最后打开比较输出文件：

webbrowser.open(filename)