【问题标题】:Multiple requests Flask, Gunicorn , Nginx logging not working多个请求 Flask、Gunicorn、Nginx 日志记录不起作用
【发布时间】:2016-05-18 08:35:38
【问题描述】:

我正在我的 Flask 应用程序中进行性能测试。

  ngnix -> gunicorn -> flask

日志文件:

  • access.log 好的
  • gunicorn.log 好的
  • api.log 只显示 1 个请求

我正在使用 ab 测试工具。我的请求被正确处理并正确记录在我的开发环境(MacOS 10.10.5)中的 Flask 日志文件中。

ab -c 10 -n 10 -A username:password https://1.1.1.1:8443/api/1.0/echo/

当我开始生产时,(Ubuntu)我看不到所有 10 个请求都记录在 Flask 日志文件中。我只在 api 日志文件中看到 1 。 ab 工具报告所有请求都得到了正确处理。

我确实在 Ngnix、Gunicorn 中看到了 10 个请求,但在 Flask 中没有。

gunicorn 启动脚本(Supervisord)

GUNICORN_LOGFILE=/usr/local/src/imbue/application/imbue/log/gunicorn.log
NUM_WORKERS=8
TIMEOUT=60
WORKER_CONNECTIONS=2000
BACKLOG=1000
exec gunicorn api:api_app --bind 0.0.0.0:8081 --log-level=DEBUG --log-file=$GUNICORN_LOGFILE --workers $NUM_WORKERS --worker-connections=$WORKER_CONNECTIONS --backlog=$BACKLOG --timeout $TIMEOUT

api 文件:

from application.app import api_app

if __name__ == "__main__":
    api_app.run(debug=False, threaded=True)

api_app Flask 应用程序

# =========================================================
# API Logs
# =========================================================

log = logging_conf.LoggerManager().getLogger("___app___", logging_file=settings.api_logfile)

# =========================================================
# Flask Application imports and database
# =========================================================
from flask import Flask
from flask.ext import restful
from werkzeug.contrib.fixers import ProxyFix

# =========================================================
# Flask Main Application
# =========================================================

api_app = Flask(__name__)  # Flask Application
api_app.config.from_pyfile("../../../conf/settings.py")  # Flask configuration

imbue_api = restful.Api(api_app)  # Define API
db = Model.db.init_app(api_app)  # Initialize database

# =========================================================
# API Definition
# =========================================================

imbue_api.add_resource(ApiBase, settings.BASE_API_URL)
imbue_api.add_resource(Echo, '/api/1.0/echo/')
# =========================================================
# API Proxy WSGi for gunicorn
# =========================================================

api_app.wsgi_app = ProxyFix(api_app.wsgi_app)
log.info('Initializing API >>>')

api_main

import logging

# =========================================================
# IMBUE imports
# =========================================================

from flask import current_app
from flask import jsonify, request, Response, url_for
from flask.ext import restful
from flask.ext.restful import Resource

# =========================================================
# API Controller
# =========================================================

api = restful.Api

# =========================================================
# API Logs
# =========================================================

log = logging_conf.LoggerManager().getLogger("___app___", logging_file=settings.api_logfile)

# =========================================================
# API Version information
# =========================================================

class Echo(Resource):
    @authenticator.requires_auth
    def get(self):
        """

        :return:
        """
        try:
            # =========================================================
            # GET API
            # =========================================================
            log.info(request.remote_addr + ' ' + request.__repr__())
            if request.headers['Content-Type'] == 'application/json':
                # =========================================================
                # Send API version information
                # =========================================================
                log.info('api() | GET | Version' + settings.api_version)
                response = json.dumps('version: ' + settings.api_version)
                resp = Response(response, status=200, mimetype='application/json')
                return resp

版本

gunicorn==19.3.0
Flask==0.10.1
Flask-HTTPAuth==2.7.0
Flask-Limiter==0.9.3
Flask-Login==0.3.2
Flask-Mail==0.9.1
Flask-Migrate==1.6.0
Flask-OAuthlib==0.9.1
Flask-Principal==0.4.0
Flask-RateLimiter==0.2.0
Flask-RESTful==0.3.5
Flask-Restless==0.17.0
Flask-Script==2.0.5
Flask-Security==1.7.5
Flask-SQLAlchemy==2.1
Flask-WTF==0.12

logging_conf

class Singleton(type):
    """

    """
    _instances = {}

    def __call__(cls, *args, **kwargs):
        if cls not in cls._instances.keys():
            cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)
        return cls._instances[cls]


class LoggerManager(object):
    """

    """
    __metaclass__ = Singleton

    _loggers = {}

    def __init__(self, *args, **kwargs):
        pass

    @staticmethod
    def getLogger(name='___app___', logging_file='imbued.log', **kwargs):
        """

        :param name:
        :param logging_file:
        :param kwargs:
        :return:
        """
        # Define timezone
        logging.basicConfig(filename=logging_file,
                            filemode='w+',
                            level=logging.INFO,
                            format='%(asctime)s.%(msecs).03d %(levelname)s %(message)s',
                            datefmt='%m/%d/%Y %H:%M:%S')

        if not name:
            return logging.getLogger()
        elif name not in LoggerManager._loggers.keys():
            LoggerManager._loggers[name] = logging.getLogger(str(name))
        return LoggerManager._loggers[name]

【问题讨论】:

  • logging_conf 来哪儿了?
  • 刚刚更新了原来的问题

标签: python logging flask gunicorn flask-restful


【解决方案1】:

当您声明 log 变量时会出现问题。 gunicorn 的多个线程调用getLogger 的多个实例,然后覆盖您的日志文件。当您使用带有threaded 参数app.run(threaded=True) 的Werkzeug 服务器时,不会发生这种情况。

如果您只需要记录请求,您可以在 gunicorn 启动脚本上使用参数 --access-logfile REQUEST_LOG_FILE

GUNICORN_LOGFILE=/usr/local/src/imbue/application/imbue/log/gunicorn.log
NUM_WORKERS=8
TIMEOUT=60
WORKER_CONNECTIONS=2000
BACKLOG=1000
REQUEST_LOG_FILE=./request-log-file.log
exec gunicorn api:api_app --bind 0.0.0.0:8081 --log-level=DEBUG --log-file=$GUNICORN_LOGFILE --workers $NUM_WORKERS --worker-connections=$WORKER_CONNECTIONS --backlog=$BACKLOG --timeout $TIMEOUT --access-logfile REQUEST_LOG_FILE

documentation上查看有关登录的更多信息。

警告:可能不安全

作为一种简单的解决方法,您可以将 getLogger 中的 basicConfig 方法更改为追加到现有文件,而不是覆盖文件。这是不安全的,多个线程更改同一个文件可能会导致意外结果。

logging.basicConfig(filename=logging_file,
                        filemode='w+', # Open, read and truncate the file.
                        level=logging.INFO,
                        format='%(asctime)s.%(msecs).03d %(levelname)s %(message)s',
                        datefmt='%m/%d/%Y %H:%M:%S')

logging.basicConfig(filename=logging_file,
                        filemode='a+', # Open, read and append to a file.
                        level=logging.INFO,
                        format='%(asctime)s.%(msecs).03d %(levelname)s %(message)s',
                        datefmt='%m/%d/%Y %H:%M:%S')

编辑 1

显然是logging is thread safe,但我不确定这是否会适用于 gunicorn 工人...

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2012-12-19
    • 2021-11-19
    • 2021-02-07
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多