最新消息:20181230 VPS服务器已从Linode换到腾讯云香港,主题仍用朋友推荐的大前端D8

【已解决】把Flask中的app的logger改造成单例以避免循环引用和多次初始化Flask的实例

Flask crifan 118浏览 0评论

折腾:

【未解决】Flask部署到线上生产环境后多实例多线程中无法共享全局变量

期间,对于此处,之前在把Flask改为工厂模式去初始化的时候,

别的模块想要调用flask的app,其中主要考虑就是:想要用到app.logger

导致别的模块依赖app

导致了循环调用,递归引用的问题

而此处,从:

Creating a singleton in Python – Stack Overflow

注意到:

其实也是可以把logger弄成singleton的

从而避免了Flask中其他子模块依赖factory.py或app.py中的app

避免了递归调用,避免了多次初始化Flask的app

所以后续也要去:

把之前Flask的中的app中的logger也弄成单例

之前Flask中初始化log的方式是:

app.py

<code>from factory import create_app
app = create_app(settings)
app.app_context().push()
# register_extensions(app)
log = app.logger
log.debug("app=%s", app)

log.debug("settings.FLASK_ENV=%s", settings.FLASK_ENV)
if __name__ == "__main__":
    app.run(
        host=settings.FLASK_HOST,
        port=settings.FLASK_PORT,
        debug=settings.DEBUG,
        use_reloader=False
    )
</code>

factory.py

<code>import os
from flask import Flask
import logging
from logging.handlers import RotatingFileHandler
...
from conf.app import settings

...
from flask import g

################################################################################
# Global Function
################################################################################

def create_app(config_object, init_extensions=True):
    # global log

    # app = Flask(__name__) #&lt;Flask 'factory'&gt;
    app = Flask(config_object.FLASK_APP_NAME) #&lt;Flask 'RobotQA'&gt;
    ...
    app.config.from_object(config_object)
    with app.app_context():
        g.app = app

        log = create_log(app)
        g.log = log
        ...

    return app

...

def create_log(app):
    print("create_log: before init log: app.logger=%s" % app.logger)
    logFormatterStr = app.config["LOG_FORMAT"]
    logFormatter = logging.Formatter(logFormatterStr)

    fileHandler = RotatingFileHandler(
        app.config['LOG_FILE_FILENAME'],
        maxBytes=app.config["LOG_FILE_MAX_BYTES"],
        backupCount=app.config["LOG_FILE_BACKUP_COUNT"],
        encoding="UTF-8")
    fileHandler.setLevel(logging.DEBUG)
    fileHandler.setFormatter(logFormatter)
    app.logger.addHandler(fileHandler)
    # Note: should NOT set StreamHandler here, otherwise will duplicate debug log
    app.logger.setLevel(logging.DEBUG)  # set root log level

    log = app.logger
    log.info("app=%s", app)
    # log.debug("app.config=%s", app.config)
    print("create_log: after init log: app.logger=%s" % app.logger)

    return log
...
</code>

然后其他模块,包括celery的task中,去导入flask的g中的log:

resources/tts.py

<code>from flask import g

app = g.app
log = g.log
def createAudioTempFolder():
    log.info("createAudioTempFolder")
</code>

这就导致了:

不同的模块,都要依赖于Flask的app,以及Flask的g

才能获取g.log,这个全局的logger

也就导致了,容易产生互相的递归引用

以及:

对于特殊的:

celery的task:

resources/extensions_celery.py

<code># from flask_celery import Celery
from conf.app import settings
from celery import Celery
from celery.utils.log import get_task_logger

# celery = Celery()
celery = Celery(settings.FLASK_APP_NAME, broker=settings.CELERY_BROKER_URL)
celery_logger = get_task_logger(__name__)
print("in extensions_celery: celery=%s" % celery)
</code>

resources/tasks.py

<code>from resources.extensions_celery import celery, celery_logger as log
log.info("create_celery_app return: celery=%s, log=%s", celery, log)
</code>

此处的log,实际上在最开始初始化时:

只是获取了celery的logger,而不是Flask的app的logger,导致了:

  • 没有达到希望:统一使用Flask的logger,不要用不同的logger

  • 最开始初始化调试时,此处的log输出看不到

    • 因为之后后续终端中通过

    • celery worker -A resources.tasks.celery –loglevel=DEBUG

    • 才能正常的在终端中显示log

其中之前输出的log的信息是:

<code>[2018-08-29 13:43:21,759 DEBUG app.py:32 &lt;module&gt;] app=&lt;Flask 'RobotQA'&gt;
[2018-08-29 13:43:21,762 DEBUG app.py:34 &lt;module&gt;] log=&lt;Logger flask.app (DEBUG)&gt;
</code>

所以为了避免这些问题:

现在要去改为多线程安全的单例:

期间,出错:

【已解决】Flask中logging的单例初始化出错:AttributeError: ‘Formatter’ object has no attribute find

然后log的单例就弄好了。

【总结】

目前,至少实现了多线程thread(暂时不支持多进程process)的logging/logger的单例:

代码如下:

common/FlaskLogSingleton.py

<code>import logging
from logging.handlers import RotatingFileHandler
from conf.app import settings
from common.ThreadSafeSingleton import ThreadSafeSingleton
# from sys import stdout

def init_logger(flask_settings, enableConsole=True):
    print("init_logger")
    flaskAppLogger = logging.getLogger(flask_settings.FLASK_APP_NAME) # &lt;Logger RobotQA (WARNING)&gt;
    print("flaskAppLogger=%s" % flaskAppLogger)
    flaskAppLogger.setLevel(flask_settings.LOG_LEVEL_FILE)

    logFormatter = logging.Formatter(flask_settings.LOG_FORMAT)

    fileHandler = RotatingFileHandler(
        flask_settings.LOG_FILE_FILENAME,
        maxBytes=flask_settings.LOG_FILE_MAX_BYTES,
        backupCount=flask_settings.LOG_FILE_BACKUP_COUNT,
        encoding="UTF-8")
    fileHandler.setLevel(flask_settings.LOG_LEVEL_FILE)
    fileHandler.setFormatter(logFormatter)
    flaskAppLogger.addHandler(fileHandler)

    if enableConsole :
        # define a Handler which writes INFO messages or higher to the sys.stderr
        console = logging.StreamHandler()
        # console = logging.StreamHandler(stdout)
        console.setLevel(flask_settings.LOG_LEVEL_CONSOLE)
        # set a format which is simpler for console use
        formatter = logging.Formatter(
            # fmt=logFormatter)
            # fmt=logFormatter,
            fmt=flask_settings.LOG_FORMAT,
            datefmt=flask_settings.LOG_CONSOLE_DATA_FORMAT)
        # tell the handler to use this format
        console.setFormatter(formatter)
        flaskAppLogger.addHandler(console)

    print("init_logger: after init flaskAppLogger%s" % flaskAppLogger)

    return flaskAppLogger

class LoggerSingleton(metaclass=ThreadSafeSingleton):
    curLog = ""

    def __init__(self):
        self.curLog = init_logger(settings)
        # Note: during __init__, AVOID use log, otherwise will deadlock
        # log.info("LoggerSingleton __init__: curLog=%s", self.curLog)
        print("LoggerSingleton __init__: curLog=%s" % self.curLog)

logSingleton = LoggerSingleton()
log = logSingleton.curLog
log.info("LoggerSingleton inited, logSingleton=%s", logSingleton) # &lt;factory.LoggerSingleton object at 0x10cbcafd0&gt;
log.info("log=%s", log) # &lt;Logger RobotQA (DEBUG)&gt;

# # debug for singleton log
# log2 = LoggerSingleton()
# print("log2=%s" % log2)
</code>

其中的配置是:

conf/app/settings.py

<code>FLASK_APP_NAME = "RobotQA"

# Log File
LOG_LEVEL_FILE = logging.DEBUG
LOG_FILE_FILENAME = "logs/" + FLASK_APP_NAME + ".log"
LOG_FORMAT = "[%(asctime)s %(levelname)s %(filename)s:%(lineno)d %(funcName)s] %(message)s"
LOG_FILE_MAX_BYTES = 2 * 1024 * 1024
LOG_FILE_BACKUP_COUNT = 10
# Log Console
LOG_LEVEL_CONSOLE = logging.INFO
LOG_CONSOLE_DATA_FORMAT = '%Y%m%d %I:%M:%S'
</code>

转载请注明:在路上 » 【已解决】把Flask中的app的logger改造成单例以避免循环引用和多次初始化Flask的实例

发表我的评论
取消评论

表情

Hi,您需要填写昵称和邮箱!

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址
63 queries in 0.110 seconds, using 10.04MB memory