stderr 和 stdout python 日志记录中的消息重复

Question

我想将信息和警告级别记录到 stdout 和错误级别及以上到 stderr 我该怎么做？

代码：

import os
import subprocess
from typing import Union
import logging
import yaml
import logging.config

with open("logging.conf", "r") as f:
    config = yaml.safe_load(f.read())
    logging.config.dictConfig(config)


logger = logging.getLogger(__name__)    


def run(
    cmd: Union[str, list[str]],
    *,
    cwd: Union[str, os.PathLike[str], None] = None,
    shell: bool = False,
    check: bool = True,
    capture_output: bool = True,
    text: bool = True,
) -> subprocess.CompletedProcess[str]:
    logger.info("start run!!")
    return subprocess.run(cmd, check=True, capture_output=capture_output, text=text)


cmd = ["lxs"]


def main():
    try:
        output = run(cmd)
    except Exception as e:
        logger.error(e, exc_info=False)


main()

日志记录.conf

version: 1
formatters:
  simple:
    format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
handlers:
  outconsole:
    class: logging.StreamHandler
    level: INFO
    formatter: simple
    stream: ext://sys.stdout
  errconsole:
    class: logging.StreamHandler
    level: WARNING
    formatter: simple
    stream: ext://sys.stderr

  file_handler:
    class: logging.FileHandler
    level: INFO
    formatter: simple
    filename: info.log
    encoding: utf8
    mode: w

loggers:
  sampleLogger:
    level: DEBUG
    handlers: [file_handler]
    propagate: no
root:
  level: INFO
  handlers: [errconsole, outconsole, file_handler]
  propagate: no

并在python3 -m tests 1> output.log 2> err.log 之后输出：

错误日志

2022-10-14 14:34:02,354 - __main__ - ERROR - [Errno 2] No such file or directory: 'lxs'

输出日志

2022-10-14 14:34:02,353 - __main__ - INFO - start run!!
2022-10-14 14:34:02,354 - __main__ - ERROR - [Errno 2] No such file or directory: 'lxs'

因此，控制台上的输出为：

2022-10-14 14:48:33,178 - __main__ - INFO - start run!!
2022-10-14 14:48:33,179 - __main__ - ERROR - [Errno 2] No such file or directory: 'lxs'
2022-10-14 14:48:33,179 - __main__ - ERROR - [Errno 2] No such file or directory: 'lxs'

我的问题是如何使信息不重复本身？我想要 strerr 上的 ERROR 和 stdout 上的 INFO 。

Answer 1

由于级别在logging 中的工作方式，如果要将stdout 限制为WARNING 及以下并将stderr 限制为ERROR 及以上，则需要在解决方案中使用过滤器。一种方法如下。

首先，添加一个与formatters 和handlers 平行的filters 条目，以定义几个过滤器：

filters:
  warnings_and_below:
    "()": __main__.filter_maker
    level: WARNING
    sense: below
  errors_and_above:
    "()": __main__.filter_maker
    level: ERROR
    sense: above

并在处理程序配置中引用它们：

filters: [warnings_and_below]  # for outconsole

和

filters: [errors_and_above]  # for errconsole

过滤器只是函数，我们将定义filter_maker 如下：

def filter_maker(level, sense):
    level = getattr(logging, level)  # get the actual numeric value from the string
    if sense == 'below':  # return a function which only passes if level is at or below threshold
        def filter(record):
            return record.levelno <= level
    else:  # return a function which only passes if level is at or above threshold
        def filter(record):
            return record.levelno >= level
    return filter

我正在使用一个小脚本并直接运行它，因此filter_maker 将位于__main__ 模块中（如上面的“()”行所示）。您可能需要更改它以适应您的具体情况。

现在，如果我添加一些行来记录内容：

logging.config.dictConfig(...)
logging.debug('A DEBUG message')
logging.info('An INFO message')
logging.warning('A WARNING message')
logging.error('An ERROR message')
logging.critical('A CRITICAL message')

假设这一切都在main.py，那么你可以运行

python main.py 2>stderr.log >stdout.log

然后你应该看到预期的结果：

$ more stdout.log 
2022-10-14 16:41:09,867 - root - INFO - An INFO message
2022-10-14 16:41:09,867 - root - WARNING - A WARNING message

$ more stderr.log
2022-10-14 16:41:09,868 - root - ERROR - An ERROR message
2022-10-14 16:41:09,868 - root - CRITICAL - A CRITICAL message

$ more info.log
2022-10-14 16:41:09,867 - root - INFO - An INFO message
2022-10-14 16:41:09,867 - root - WARNING - A WARNING message
2022-10-14 16:41:09,868 - root - ERROR - An ERROR message
2022-10-14 16:41:09,868 - root - CRITICAL - A CRITICAL message