【问题标题】:copy variable number of lines based on start and end triggers根据开始和结束触发器复制可变数量的行
【发布时间】:2018-05-22 02:15:23
【问题描述】:

我有一个脚本,它只收集来自多个日志文件的重要文本行并将它们写入单独的文本文件。只要我提供一个字符串 sn-p 来搜索,它就可以工作。一旦找到,它将复制找到字符串 sn-p 的整行文本。现在我需要它根据已知的开始和结束文本 sn-ps 选择多行文本。开始和结束文本 sn-ps 之间/包括的总行数因我正在阅读的日志文件而异。我已经搜索并尝试了很多东西,但无法让它工作。我正在提供我目前所拥有的。

我不是 python 新手,但远非专家。任何/所有建议都非常感谢。

import os, time, datetime, fnmatch, re
from time import strftime, time

# Open log file to record actions
dirName = "D://GISlogs//" + strftime("%Y%m%d")
global pubfilePrefix
pubfilePrefix = datetime.datetime.now() + datetime.timedelta(minutes=1)
pubfileName = "D://GISlogs//" + strftime("%Y%m%d") + "//" + pubfilePrefix.strftime("%Y%m%d_%H%M%S") + "_SUMMARIZE_MorningActivities.txt"
if os.path.exists(pubfileName) and os.path.isfile(pubfileName) and os.access(pubfileName, os.W_OK):
    with open(pubfileName, "a") as myFile:
        myFile.write(strftime("%Y-%m-%d %H:%M:%S") + "  SUMMARIZING ALL GIS ACTIVITIES1" + "\r\n" + "\r\n")
else:
    try:
        os.makedirs(dirName)
        with open(pubfileName, "a") as myFile:
            myFile.write(strftime("%Y-%m-%d %H:%M:%S") + "  SUMMARIZING ALL GIS ACTIVITIES2" + "\r\n" + "\r\n")
    except OSError:
        if os.path.isdir(dirName):
            with open(pubfileName, "a") as myFile:
                myFile.write(strftime("%Y-%m-%d %H:%M:%S") + "  SUMMARIZING ALL GIS ACTIVITIES3" + "\r\n" + "\r\n")

def Search_LogFile(searchPhrase, titleBlock):
    if searchPhrase == "~-~-~":
        with open(pubfileName, "a") as myFile:
            myFile.write("\r\n" + titleBlock + "\r\n" + "---------------------------------------------------------------------------------------" + "\r\n")

    with open("D://GISlogs//" + strftime("%Y%m%d") + "//" + searchFile, "r") as readFile:
        for readLine in readFile:
            if searchPhrase in readLine:
                if re.compile("^[0-9]{4}-[0-9]{2}-[0-9]{2} ").match(readLine):
                    readLine = readLine[11:]
                with open(pubfileName, "a") as myFile:
                    myFile.write(readLine)

for searchFile in os.listdir(dirName):

    if fnmatch.fnmatch(searchFile, "*COPY_TaxRollData.txt"):
        Search_LogFile("~-~-~", "Acquiring Updated Data from PVA")
        Search_LogFile(" - DELETING PREVIOUSLY DOWNLOADED PVA DATA", "")
        Search_LogFile(" - DOWNLOADING THE LATEST SNAPSHOT OF PUBLIC INQUIRY DATABASE", "")
        Search_LogFile(" - ARCHIVING ALL DOWNLOADED FILES", "")
        Search_LogFile(" - RECONCILING NEW DOWNLOAD WITH PRIOR DATA", "")
        Search_LogFile(" - EXECUTING FME MODEL TO OVERWRITE PVA DATA IN SDE", "")

    if fnmatch.fnmatch(searchFile, "*INTEGRATE_TaxRollData.txt"):
        Search_LogFile("~-~-~", "Integrating Updated Data from PVA into the SDE Production EGDB")
        Search_LogFile("|INFORM|Reading table/feature class '", "")
        Search_LogFile("|INFORM|Opened Shape File '", "")
        Search_LogFile("|INFORM|CSV reader: Opening dataset '", "")
        Search_LogFile("|ERROR |", "")
        Search_LogFile("|INFORM|Translation was SUCCESSFUL", "")
        Search_LogFile("|INFORM|FME Session Duration:", "")
        #THE NEXT 9 LINES IS MY LATEST ATTEMPT. IT WORKS OUTSIDE OF THIS BLOCK OF CODE, BUT NOT HERE
        valid=False
        for line in searchFile:
            if re.compile(".*Features Written Summary.*").match(line):
                valid=True
            if re.compile(".*Total Features Written.*").match(line):
                valid=False
            if valid:
                with open(pubfileName, "a") as myFile:
                    myFile.write(line)
        #Search_LogFile("Features Written Summary", "")
        Search_LogFile("  PVA TAX ROLL DATA HAS BEEN RE-INTEGRATED IN THE SDE PRODUCTION DATABASE", "") 

print('\r\n' +'The Script Completed successfully')   

为了进一步澄清 - 文本“Features Written Summary”在我希望我的捕获开始的行上,而“Total Features Written”在我希望我的捕获结束的行上(我想复制两个 sn-ps 出现的整行以及两者之间的所有内容)。开始和结束的 sn-ps 在日志文件中只出现一次。

@MadPhysicist 当我运行代码时,它会找到第一个日志文件并查找特定的短语。当它找到它们时,它将找到该短语的整行文本写入不同的文本文件。然后它会寻找下一个特定的短语并做同样的事情。在该日志文件中查找所有特定短语后,它将找到下一个日志文件并查找特定于该特定日志文件的一组不同的短语。效果很好。

但我要添加的是我无法弄清楚的一些不同的调整。在第二个日志文件中,我想将在其中找到的多个连续行复制到我的另一个文本文件中。我可以使用上述方法复制起始行和结束行,但问题是我还想复制起始行和结束行之间的所有文本行。开始行和结束行之间的行数也可以变化,因此不能硬编码。

【问题讨论】:

  • 你的缩进有点乱。如果您不想嵌套 with 块(我不想),请执行 with open(...) as myFile, open(...) as readFile: 之类的操作。
  • 不要使用双 forward 斜杠。你只需要双反斜杠
  • strftime 缺少参数。请展示您尝试运行代码时会发生什么。你至少应该展示一些没有明显语法错误的东西。
  • 作为一般建议,请用文字描述您计划使用的算法。这将使您自己编写代码变得更加容易。
  • @MadPhysicist - 抱歉,我是 StackOverflow 的新手,还在学习礼仪(这只是我的第二个问题)。我修复了缩进(我认为)。还包括省略的代码(为了简洁起见,最初省略了)。希望语法错误不像以前那样“明显”。

标签: python-2.7


【解决方案1】:

所以我现在有一个可行的解决方案,但我不确定它是否最有效和/或是否可以设计得更好。我将把它落实到位,但如果有人能想到更好的方法来完成我在这里尝试做的事情,我会全力以赴......

import os, time, datetime, fnmatch, re
from time import strftime, time

# Open log file to record actions
dirName = "D://GISlogs//" + strftime("%Y%m%d")
global pubfilePrefix
pubfilePrefix = datetime.datetime.now() + datetime.timedelta(minutes=1)
pubfileName = "D://GISlogs//" + strftime("%Y%m%d") + "//" + 
pubfilePrefix.strftime("%Y%m%d_%H%M%S") + "_SUMMARIZE_MorningActivities.txt"
if os.path.exists(pubfileName) and os.path.isfile(pubfileName) and os.access(pubfileName, os.W_OK):
    with open(pubfileName, "a") as myFile:
        myFile.write(strftime("%Y-%m-%d %H:%M:%S") + "  SUMMARIZING ALL GIS ACTIVITIES1" + "\r\n" + "\r\n")
else:
    try:
        os.makedirs(dirName)
        with open(pubfileName, "a") as myFile:
            myFile.write(strftime("%Y-%m-%d %H:%M:%S") + "  SUMMARIZING ALL GIS ACTIVITIES2" + "\r\n" + "\r\n")
    except OSError:
        if os.path.isdir(dirName):
            with open(pubfileName, "a") as myFile:
                myFile.write(strftime("%Y-%m-%d %H:%M:%S") + "  SUMMARIZING ALL GIS ACTIVITIES3" + "\r\n" + "\r\n")

def Search_LogFile(searchPhrase, titleBlock):
    if searchPhrase == "~-~-~":
        with open(pubfileName, "a") as myFile:
            myFile.write("\r\n" + titleBlock + "\r\n" + "---------------------------------------------------------------------------------------" + "\r\n")

    with open("D://GISlogs//" + strftime("%Y%m%d") + "//" + searchFile, "r") as readFile:
        for readLine in readFile:
            if searchPhrase in readLine:
                if re.compile("^[0-9]{4}-[0-9]{2}-[0-9]{2} ").match(readLine):
                    readLine = readLine[11:]
                with open(pubfileName, "a") as myFile:
                    myFile.write(readLine)

            if searchPhrase in readLine and searchPhrase == "Features Written Summary":
                valid=False
                for line in readFile:
                    if re.compile(".*|                           Features Written Summary.*").match(line):
                        valid = True
                    if re.compile(".*END - ProcessID.*").match(line):
                        valid = False
                    if valid:
                        if re.compile("^[0-9]{4}-[0-9]{2}-[0-9]{2} ").match(line):
                            line = line[11:]
                            with open(pubfileName, "a") as myFile:
                                myFile.write(line)

for searchFile in os.listdir(dirName):

    if fnmatch.fnmatch(searchFile, "*COPY_TaxRollData.txt"):
        Search_LogFile("~-~-~", "Acquiring Updated Data from PVA")
        Search_LogFile(" - DELETING PREVIOUSLY DOWNLOADED PVA DATA", "")
        Search_LogFile(" - DOWNLOADING THE LATEST SNAPSHOT OF PUBLIC INQUIRY DATABASE", "")
        Search_LogFile(" - ARCHIVING ALL DOWNLOADED FILES", "")
        Search_LogFile(" - RECONCILING NEW DOWNLOAD WITH PRIOR DATA", "")
        Search_LogFile(" - DISCONNECTING W: DRIVE", "")
        Search_LogFile(" - EXECUTING FME MODEL TO OVERWRITE PVA DATA IN SDE", "")

    if fnmatch.fnmatch(searchFile, "*INTEGRATE_TaxRollData.txt"):
        Search_LogFile("~-~-~", "Integrating Updated Data from PVA into the SDE Production EGDB")
        Search_LogFile("Reading table/feature class '", "")
        Search_LogFile("Opened Shape File '", "")
        Search_LogFile("CSV reader: Opening dataset '", "")
        Search_LogFile("|ERROR |", "")
        #Search_LogFile("|WARN  |", "")
        Search_LogFile("Features Written Summary", "")

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 2018-12-20
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2017-08-22
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多