【发布时间】:2017-05-08 20:27:22
【问题描述】:
我正在使用多处理模块并行处理文件,几乎每次都能正常工作。 我也在 try 中写了这个,除了块来捕获任何异常。 我遇到过 except 块没有捕获异常的情况。
由于代码很大,我只是将相关的块放在了给出问题的地方。
def reader(que, ip, start, end, filename):
""" Reader function checks each line of the file
and if the line contains any of the ip addresses which are
being scanned, then it writes to its buffer.
If the line field doesn't match date string it skips the line.
"""
logging.info("Processing : %s" % os.path.basename(filename))
ip_pat = re.compile("(\d+\.\d+\.\d+\.\d+\:\d+)")
chunk = 10000000 # taking chunk of 10MB data
buff = ""
with bz2.BZ2File(filename,"rb", chunk) as fh: # open the compressed file
for line in fh:
output = []
fields = line.split()
try:
ts = fields[1].strip() + "/" +fields[0]+"/"+fields[3].split("-")[0]+" "+fields[2]
times = da.datetime.strptime(ts,"%d/%b/%Y %H:%M:%S")
if times < start:
continue
if times > end:
break
ips = re.findall(ip_pat,line)
if len(ips) < 3:
continue
if ips[0].split(":")[0] == ip:
output.append(times.strftime("%d/%m/%Y %H:%M:%S"))
status = "SESSION_OPEN" if "SESSION_OPEN" in line or "CREATE" in line else "SESSION_CLOSE"
protocol = "TCP" if "TCP" in line else "UDP"
output.append(status)
output.append(protocol)
ips[1], ips[2] = ips[2], ips[1]
output.extend(ips)
res = "|".join(output)
buff += res + "\n"
except IndexError, ValueError:
continue
logging.info("Processed : %s of size [ %d ]" % (os.path.basename(filename), os.path.getsize(filename)))
if buff:
que.put((ip,buff))
return buff
这就是作为错误接收的内容。
文件“/usr/lib64/python2.7/multiprocessing/pool.py”,第 554 行,在 get 提高自我价值 ValueError: 时间数据 '2/Dec/20 10:59:59' 与格式 '%d/%b/%Y %H:%M:%S' 不匹配
我不明白为什么没有捕获到异常,我在except块中提到了ValueError。
解决这个问题的最佳方法是什么。
【问题讨论】:
标签: python exception multiprocessing