最新消息:20210816 当前crifan.com域名已被污染,为防止失联,请关注(页面右下角的)公众号

【已解决】把Celery+Redis集成到在线Flask中且用supervisor去管理后台服务

celery crifan 2826浏览 0评论

折腾:

【已解决】Flask中如何保存临时文件且可以指定有效期

期间,需要去:

把之前mac本地测试逻辑上通过的celery+redis去实现的延迟执行任务的功能

集成到Flask中,且由于celery和redis都有后台sever要运行,

所以还要用supervisor去管理celery和redis的任务

确保每次正常能启动后台服务

celery redis supervisor

Setting up Nginx, Gunicorn, Celery, Redis, Supervisor, and Postgres with Django to run your Python…

好像就是去supervisor中加上program的部分的配置就可以了?

【已解决】supervisor管理celery时是否需要设置workdir参数

服务器使用Supervisor后台运行Celery – 杨仕航的博客

解释的不错

Monitoring and Management Guide — Celery 4.1.0 documentation

python celery异步任务队列(redis + supervisor)例子_壹聚教程网

django使用supervisor管理celery – 简书

“如果用户是非root用户,而supervisor是 ROOT 用户进行的,所以要在tasks.py中加上

from celery import Celery,platforms

platforms.C_FORCE_ROOT = True”

【已解决】celery beat是什么

然后去supervisor中配置celery的worker和redis

然后先去:把本地的pipfile上传到在线环境:

然后pipenv去install,把celery,redis等更新进去

[root@xxx-general-01 ~]# cd xxx/server/robotDemo/

[root@xx-general-01 robotDemo]# ll

total 60

drwxr-xr-x 5 root root  4096 May 10 14:57 ai

-rw-r–r– 1 root root 12740 May 10 14:56 app.py

-rw-r–r– 1 root root  1130 May  3 09:37 config.py

-rw-r–r– 1 root root  2005 Apr 24 16:36 gunicorn_config.py

drwxr-xr-x 2 root root  4096 May 10 14:57 logs

-rw-r–r– 1 root root   303 May 11 17:22 Pipfile

-rw-r–r– 1 root root 12454 May 11 17:24 Pipfile.lock

drwxr-xr-x 2 root root  4096 May 10 14:56 __pycache__

-rw-r–r– 1 root root  2102 May  2 16:10 README.md

[root@xx-general-01 robotDemo]# pipenv shell

Warning: Your Pipfile requires python_version 3.6, but you are using 3.4.5 (/root/.local/share/v/r/bin/python).

  $ pipenv check will surely fail.

Spawning environment shell (/bin/bash). Use ‘exit’ to leave.

[root@xx-general-01 robotDemo]# . /root/.local/share/virtualenvs/robotDemo-dwdcgdaG/bin/activate

(robotDemo-dwdcgdaG) [root@xx-x-01 robotDemo]# pipenv graph

Flask-Cors==3.0.4

  – Flask [required: >=0.9, installed: 1.0.1]

    – click [required: >=5.1, installed: 6.7]

    – itsdangerous [required: >=0.24, installed: 0.24]

    – Jinja2 [required: >=2.10, installed: 2.10]

      – MarkupSafe [required: >=0.23, installed: 1.0]

    – Werkzeug [required: >=0.14, installed: 0.14.1]

  – Six [required: Any, installed: 1.11.0]

Flask-PyMongo==0.5.1

  – Flask [required: >=0.8, installed: 1.0.1]

    – click [required: >=5.1, installed: 6.7]

    – itsdangerous [required: >=0.24, installed: 0.24]

    – Jinja2 [required: >=2.10, installed: 2.10]

      – MarkupSafe [required: >=0.23, installed: 1.0]

    – Werkzeug [required: >=0.14, installed: 0.14.1]

  – PyMongo [required: >=2.5, installed: 3.6.1]

Flask-RESTful==0.3.6

  – aniso8601 [required: >=0.82, installed: 3.0.0]

  – Flask [required: >=0.8, installed: 1.0.1]

    – click [required: >=5.1, installed: 6.7]

    – itsdangerous [required: >=0.24, installed: 0.24]

    – Jinja2 [required: >=2.10, installed: 2.10]

      – MarkupSafe [required: >=0.23, installed: 1.0]

    – Werkzeug [required: >=0.14, installed: 0.14.1]

  – pytz [required: Any, installed: 2018.4]

  – six [required: >=1.3.0, installed: 1.11.0]

gunicorn==19.8.1

openpyxl==2.5.3

  – et-xmlfile [required: Any, installed: 1.0.1]

  – jdcal [required: Any, installed: 1.4]

PyMySQL==0.8.0

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]# pipenv install

Warning: Your Pipfile requires python_version 3.6, but you are using 3.4.5 (/root/.local/share/v/r/bin/python).

  $ pipenv check will surely fail.

Installing dependencies from Pipfile.lock (132fcc)…

   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 29/29 — 00:00:08

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]# pipenv graph  

celery==4.1.0

  – billiard [required: <3.6.0,>=3.5.0.2, installed: 3.5.0.3]

  – kombu [required: <5.0,>=4.0.2, installed: 4.1.0]

    – amqp [required: >=2.1.4,<3.0, installed: 2.2.2]

      – vine [required: >=1.1.3, installed: 1.1.4]

  – pytz [required: >dev, installed: 2018.4]

Flask-Cors==3.0.4

  – Flask [required: >=0.9, installed: 1.0.2]

    – click [required: >=5.1, installed: 6.7]

    – itsdangerous [required: >=0.24, installed: 0.24]

    – Jinja2 [required: >=2.10, installed: 2.10]

      – MarkupSafe [required: >=0.23, installed: 1.0]

    – Werkzeug [required: >=0.14, installed: 0.14.1]

  – Six [required: Any, installed: 1.11.0]

Flask-PyMongo==0.5.1

  – Flask [required: >=0.8, installed: 1.0.2]

    – click [required: >=5.1, installed: 6.7]

    – itsdangerous [required: >=0.24, installed: 0.24]

    – Jinja2 [required: >=2.10, installed: 2.10]

      – MarkupSafe [required: >=0.23, installed: 1.0]

    – Werkzeug [required: >=0.14, installed: 0.14.1]

  – PyMongo [required: >=2.5, installed: 3.6.1]

Flask-RESTful==0.3.6

  – aniso8601 [required: >=0.82, installed: 3.0.0]

  – Flask [required: >=0.8, installed: 1.0.2]

    – click [required: >=5.1, installed: 6.7]

    – itsdangerous [required: >=0.24, installed: 0.24]

    – Jinja2 [required: >=2.10, installed: 2.10]

      – MarkupSafe [required: >=0.23, installed: 1.0]

    – Werkzeug [required: >=0.14, installed: 0.14.1]

  – pytz [required: Any, installed: 2018.4]

  – six [required: >=1.3.0, installed: 1.11.0]

gunicorn==19.8.1

openpyxl==2.5.3

  – et-xmlfile [required: Any, installed: 1.0.1]

  – jdcal [required: Any, installed: 1.4]

PyMySQL==0.8.1

redis==2.10.6

requests==2.18.4

  – certifi [required: >=2017.4.17, installed: 2018.4.16]

  – chardet [required: >=3.0.2,<3.1.0, installed: 3.0.4]

  – idna [required: >=2.5,<2.7, installed: 2.6]

  – urllib3 [required: <1.23,>=1.21.1, installed: 1.22]

然后还要在CentOS服务器中,确保已经安装了redis,否则还要安装redis

【已解决】CentOS中安装和运行redis

其中supervisor中配置celery的worker中的celery的路径,通过:

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]# pwd

/xx/robotDemo

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]# which celery

/root/.local/share/virtualenvs/robotDemo-dwdcgdaG/bin/celery

而找到;

/root/.local/share/virtualenvs/robotDemo-dwdcgdaG/bin/celery

然后接着去配置

supervisord_server.conf

[program:redis]

directory=/root/xx/server/robotDemo

command=/usr/bin/redis-server

autostart=true

autorestart=true

stdout_logfile=/root/xx/robotDemo/logs/redis-%(program_name)s-stdout.log

stdout_logfile_maxbytes=2MB

stdout_logfile_backups=10

stderr_logfile=/root/xx/robotDemo/logs/redis-%(program_name)s-stderr.log

stderr_logfile_maxbytes=2MB

stderr_logfile_backups=10

[program:robotDemo_CeleryWorker]

command=/root/.local/share/virtualenvs/robotDemo-dwdcgdaG/bin/celery worker -A app.celeryApp –logfile

directory=/root/xx/robotDemo

autostart=true

autorestart=true

stdout_logfile=/root/xx/robotDemo/logs/celery-worker-%(program_name)s-stdout.log

stdout_logfile_maxbytes=2MB

stdout_logfile_backups=10

stderr_logfile=/root/xx/robotDemo/logs/celery-worker-%(program_name)s-stderr.log

stderr_logfile_maxbytes=2MB

stderr_logfile_backups=10

[program:robotDemo]

command=/root/.local/share/virtualenvs/robotDemo-dwdcgdaG/bin/gunicorn -c gunicorn_config.py app:app

directory=/root/xx/robotDemo

startsecs=0

stopwaitsecs=0

autostart=true

autorestart=true

killasgroup=true

stopasgroup=true

stdout_logfile=/root/xx/robotDemo/logs/supervisord-%(program_name)s-stdout.log

stdout_logfile_maxbytes=2MB

stdout_logfile_backups=10

stderr_logfile=/root/xx/robotDemo/logs/supervisord-%(program_name)s-stderr.log

stderr_logfile_maxbytes=2MB

stderr_logfile_backups=10

然后去试试效果

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]# supervisorctl stop all

robotDemo: stopped

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]# supervisorctl help

default commands (type help <topic>):

=====================================

add    clear  fg        open  quit    remove  restart   start   stop  update

avail  exit   maintail  pid   reload  reread  shutdown  status  tail  version

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]# supervisorctl reload

Restarted supervisord

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]# supervisorctl status all

redis                            FATAL     Exited too quickly (process log may have details)

robotDemo                        RUNNING   pid 11128, uptime 0:00:09

robotDemo_CeleryWorker           STARTING  

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]# supervisorctl stop all

robotDemo: stopped

robotDemo_CeleryWorker: stopped

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]#

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]#

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]# supervisorctl restart all

robotDemo: started

robotDemo_CeleryWorker: started

redis: ERROR (abnormal termination)

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]#

去看看日志:

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]# cat logs/

celery-worker-robotDemo_CeleryWorker-stderr.log  gunicorn_error.log                               RobotQA.log

celery-worker-robotDemo_CeleryWorker-stdout.log  redis-redis-stderr.log                           supervisord-robotDemo-stderr.log

gunicorn_access.log                              redis-redis-stdout.log                           supervisord-robotDemo-stdout.log

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]# cat logs/redis-redis-stderr.log

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]# cat logs/redis-redis-stdout.log

11130:C 14 May 16:20:16.048 # Warning: no config file specified, using the default config. In order to specify a config file use /usr/bin/redis-server /path/to/redis.conf

11130:M 14 May 16:20:16.049 * Increased maximum number of open files to 10032 (it was originally set to 1024).

11130:M 14 May 16:20:16.050 # Creating Server TCP listening socket *:6379: bind: Address already in use

11161:C 14 May 16:20:17.112 # Warning: no config file specified, using the default config. In order to specify a config file use /usr/bin/redis-server /path/to/redis.conf

11161:M 14 May 16:20:17.113 * Increased maximum number of open files to 10032 (it was originally set to 1024).

11161:M 14 May 16:20:17.114 # Creating Server TCP listening socket *:6379: bind: Address already in use

11198:C 14 May 16:20:19.376 # Warning: no config file specified, using the default config. In order to specify a config file use /usr/bin/redis-server /path/to/redis.conf

11198:M 14 May 16:20:19.377 * Increased maximum number of open files to 10032 (it was originally set to 1024).

11198:M 14 May 16:20:19.377 # Creating Server TCP listening socket *:6379: bind: Address already in use

11209:C 14 May 16:20:22.622 # Warning: no config file specified, using the default config. In order to specify a config file use /usr/bin/redis-server /path/to/redis.conf

11209:M 14 May 16:20:22.622 * Increased maximum number of open files to 10032 (it was originally set to 1024).

11209:M 14 May 16:20:22.623 # Creating Server TCP listening socket *:6379: bind: Address already in use

11298:C 14 May 16:20:54.314 # Warning: no config file specified, using the default config. In order to specify a config file use /usr/bin/redis-server /path/to/redis.conf

11298:M 14 May 16:20:54.315 * Increased maximum number of open files to 10032 (it was originally set to 1024).

11298:M 14 May 16:20:54.316 # Creating Server TCP listening socket *:6379: bind: Address already in use

11308:C 14 May 16:20:55.417 # Warning: no config file specified, using the default config. In order to specify a config file use /usr/bin/redis-server /path/to/redis.conf

11308:M 14 May 16:20:55.418 * Increased maximum number of open files to 10032 (it was originally set to 1024).

11308:M 14 May 16:20:55.418 # Creating Server TCP listening socket *:6379: bind: Address already in use

11337:C 14 May 16:20:57.876 # Warning: no config file specified, using the default config. In order to specify a config file use /usr/bin/redis-server /path/to/redis.conf

11337:M 14 May 16:20:57.877 * Increased maximum number of open files to 10032 (it was originally set to 1024).

11337:M 14 May 16:20:57.878 # Creating Server TCP listening socket *:6379: bind: Address already in use

11348:C 14 May 16:21:01.179 # Warning: no config file specified, using the default config. In order to specify a config file use /usr/bin/redis-server /path/to/redis.conf

11348:M 14 May 16:21:01.180 * Increased maximum number of open files to 10032 (it was originally set to 1024).

11348:M 14 May 16:21:01.180 # Creating Server TCP listening socket *:6379: bind: Address already in use

估计是之前运行的redis没有被干掉,导致无法重启redis?

去看看

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]# supervisorctl restart all       

robotDemo: stopped

robotDemo_CeleryWorker: stopped

robotDemo: started

robotDemo_CeleryWorker: started

redis: started

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]# cat logs/redis-redis-stdout.log

11130:C 14 May 16:20:16.048 # Warning: no config file specified, using the default config. In order to specify a config file use /usr/bin/redis-server /path/to/redis.conf

11130:M 14 May 16:20:16.049 * Increased maximum number of open files to 10032 (it was originally set to 1024).

11748:M 14 May 16:23:10.516 * Increased maximum number of open files to 10032 (it was originally set to 1024).

                _._                                                  

           _.-“__ ”-._                                             

      _.-“    `.  `_.  ”-._           Redis 3.2.10 (00000000/0) 64 bit

  .-“ .-“`.  “`\/    _.,_ ”-._                                   

(    ‘      ,       .-`  | `,    )     Running in standalone mode

|`-._`-…-` __…-.“-._|’` _.-‘|     Port: 6379

|    `-._   `._    /     _.-‘    |     PID: 11748

  `-._    `-._  `-./  _.-‘    _.-‘                                   

|`-._`-._    `-.__.-‘    _.-‘_.-‘|                                  

|    `-._`-._        _.-‘_.-‘    |           http://redis.io

  `-._    `-._`-.__.-‘_.-‘    _.-‘                                   

|`-._`-._    `-.__.-‘    _.-‘_.-‘|                                  

|    `-._`-._        _.-‘_.-‘    |                                  

  `-._    `-._`-.__.-‘_.-‘    _.-‘                                   

      `-._    `-.__.-‘    _.-‘                                       

          `-._        _.-‘                                           

              `-.__.-‘                                               

11748:M 14 May 16:23:10.517 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.

11748:M 14 May 16:23:10.517 # Server started, Redis version 3.2.10

11748:M 14 May 16:23:10.517 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add ‘vm.overcommit_memory = 1’ to /etc/sysctl.conf and then reboot or run the command ‘sysctl vm.overcommit_memory=1’ for this to take effect.

11748:M 14 May 16:23:10.517 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command ‘echo never > /sys/kernel/mm/transparent_hugepage/enabled’ as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.

11748:M 14 May 16:23:10.517 * The server is now ready to accept connections on port 6379

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]#

貌似可以了。

然后去试试flask的rest的api

然后就可以了:

然后会生成对应临时mp3的音频文件:

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]# ll tmp/audio/

total 12

-rw-r–r– 1 root root 10368 May 14 16:29 eb0be3a0-c2a2-412e-a6c7-ed899dd58b4d.mp3

然后等会,celery的work就工作了,就可以去删除文件了

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]# tail logs/celery-worker-robotDemo_CeleryWorker-stderr.log

[2018-05-14 16:29:09,630] INFO in app: resp=<Response [200]>

[2018-05-14 16:29:09,631] INFO in app: respJson={‘access_token’: ‘24.5f268f4de2e09aff6da8c8e0d21dadcd.2592000.1528878549.282335-11192483’, ‘scope’: ‘public audio_voice_assistant_get audio_tts_post wise_adapt lebo_resource_base lightservice_public hetu_basic lightcms_map_poi kaidian_kaidian ApsMisTest_Test权限 vis-classify_flower lpq_开放 cop_helloScope ApsMis_fangdi_permission smartapp_snsapi_base’, ‘expires_in’: 2592000, ‘refresh_token’: ‘25.3c96fde3397a273edd8389868a3a5d84.315360000.1841646549.282335-11192483’, ‘session_secret’: ‘813fa177bb66b96ba5e672490f5718da’, ‘session_key’: ‘9mzdDtR8dgk+GKd0vEW2L0TI45a57qtjZ3xIDkayWQPBTaoAjvLTSAVehu79MRMPJ6B41dsz0M7Q+kFWhUSIgdiimce2uw==’}

[2018-05-14 16:29:09,632] INFO in app: get baidu token resp: {‘access_token’: ‘24.5f268f4de2e09aff6da8c8e0d21dadcd.2592000.1528878549.282335-11192483’, ‘scope’: ‘public audio_voice_assistant_get audio_tts_post wise_adapt lebo_resource_base lightservice_public hetu_basic lightcms_map_poi kaidian_kaidian ApsMisTest_Test权限 vis-classify_flower lpq_开放 cop_helloScope ApsMis_fangdi_permission smartapp_snsapi_base’, ‘expires_in’: 2592000, ‘refresh_token’: ‘25.3c96fde3397a273edd8389868a3a5d84.315360000.1841646549.282335-11192483’, ‘session_secret’: ‘813fa177bb66b96ba5e672490f5718da’, ‘session_key’: ‘9mzdDtR8dgk+GKd0vEW2L0TI45a57qtjZ3xIDkayWQPBTaoAjvLTSAVehu79MRMPJ6B41dsz0M7Q+kFWhUSIgdiimce2uw==’}

[2018-05-14 16:29:09,632] INFO in app: audioTmpFolder=tmp/audio

[2018-05-14 16:29:09,632] INFO in app: curFolderAbsPath=/xx/robotDemo

[2018-05-14 16:29:09,633] INFO in app: audioTmpFolderFullPath=/root/xx/robotDemo/tmp/audio

[2018-05-14 16:29:09,633] INFO in app: gTempAudioFolder=/xx/robotDemo/tmp/audio

[2018-05-14 16:29:09,712] INFO in app: purePymongo=MongoClient(host=[‘localhost:port’], document_class=dict, tz_aware=False, connect=True, authsource=’gridfs’)

[2018-05-14 16:29:09,714] INFO in app: gridfsDb=Database(MongoClient(host=[‘localhost:port’], document_class=dict, tz_aware=False, connect=True, authsource=’gridfs’), ‘gridfs’)

[2018-05-14 16:29:09,715] INFO in app: fsCollection=<gridfs.GridFS object at 0x7f112cbb61d0>

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]# tail logs/celery-worker-robotDemo_CeleryWorker-stderr.log

[2018-05-14 16:29:55,455: WARNING/ForkPoolWorker-3] [2018-05-14 16:29:55,454] INFO in app: deleteTmpAudioFile: filename=eb0be3a0-c2a2-412e-a6c7-ed899dd58b4d.mp3

[2018-05-14 16:29:55,454: INFO/ForkPoolWorker-3] deleteTmpAudioFile: filename=eb0be3a0-c2a2-412e-a6c7-ed899dd58b4d.mp3

[2018-05-14 16:29:55,455: WARNING/ForkPoolWorker-3] [2018-05-14 16:29:55,455] INFO in app: audioTmpFolder=tmp/audio

[2018-05-14 16:29:55,455: INFO/ForkPoolWorker-3] audioTmpFolder=tmp/audio

[2018-05-14 16:29:55,456: WARNING/ForkPoolWorker-3] [2018-05-14 16:29:55,456] INFO in app: curFolderAbsPath=/rootxxx/robotDemo

[2018-05-14 16:29:55,456: INFO/ForkPoolWorker-3] curFolderAbsPath=/root/xx/robotDemo

[2018-05-14 16:29:55,456: WARNING/ForkPoolWorker-3] [2018-05-14 16:29:55,456] INFO in app: audioTmpFolderFullPath=/root/xx/robotDemo/tmp/audio

[2018-05-14 16:29:55,456: INFO/ForkPoolWorker-3] audioTmpFolderFullPath=/root/xx/robotDemo/tmp/audio

[2018-05-14 16:29:55,456: WARNING/ForkPoolWorker-3] [2018-05-14 16:29:55,456] INFO in app: Ok to delete file /root/xx/robotDemo/tmp/audio/eb0be3a0-c2a2-412e-a6c7-ed899dd58b4d.mp3

[2018-05-14 16:29:55,456: INFO/ForkPoolWorker-3] Ok to delete file /root/xx/robotDemo/tmp/audio/eb0be3a0-c2a2-412e-a6c7-ed899dd58b4d.mp3

(robotDemo-dwdcgdaG) [root@xx-general-01 robotDemo]# ll tmp/audio/

total 0

如此,就可以正常的实现了,把本地的celery+redis,部署到centos在线服务器了。

然后重启了下服务器,也是可以正常通过supervisor去启动对应program的:

[root@xx-general-01 ~]# supervisorctl status all

redis                            RUNNING   pid 682, uptime 0:04:36

robotDemo                        RUNNING   pid 1018, uptime 0:04:32

robotDemo_CeleryWorker           RUNNING   pid 728, uptime 0:04:33

所以至此就完成了。

【总结】

此处把本地mac可以工作的celery+redis部署到在线CentOS中的步骤是:

1.CentOS中安装redis

yum -y install redis

2.更新supervisord的配置

/etc/supervisord.d/supervisord_server.conf

[program:redis]

directory=/xx/robotDemo

command=/usr/bin/redis-server

autostart=true

autorestart=true

stdout_logfile=/xx/robotDemo/logs/redis-%(program_name)s-stdout.log

stdout_logfile_maxbytes=2MB

stdout_logfile_backups=10

stderr_logfile=/xx/robotDemo/logs/redis-%(program_name)s-stderr.log

stderr_logfile_maxbytes=2MB

stderr_logfile_backups=10

[program:robotDemo_CeleryWorker]

command=/root/.local/share/virtualenvs/robotDemo-dwdcgdaG/bin/celery worker -A app.celeryApp

directory=/xx/robotDemo

autostart=true

autorestart=true

stdout_logfile=/xx/robotDemo/logs/celery-worker-%(program_name)s-stdout.log

stdout_logfile_maxbytes=2MB

stdout_logfile_backups=10

stderr_logfile=/xx/robotDemo/logs/celery-worker-%(program_name)s-stderr.log

stderr_logfile_maxbytes=2MB

stderr_logfile_backups=10

[program:robotDemo]

command=/root/.local/share/virtualenvs/robotDemo-dwdcgdaG/bin/gunicorn -c gunicorn_config.py app:app

directory=/xx/robotDemo

startsecs=0

stopwaitsecs=0

autostart=true

autorestart=true

killasgroup=true

stopasgroup=true

stdout_logfile=/xx/robotDemo/logs/supervisord-%(program_name)s-stdout.log

stdout_logfile_maxbytes=2MB

stdout_logfile_backups=10

stderr_logfile=/xx/robotDemo/logs/supervisord-%(program_name)s-stderr.log

stderr_logfile_maxbytes=2MB

stderr_logfile_backups=10

即可。

注:

其中本地此处之前已经配置好了supervisord的配置,默认是加载:

/etc/supervisord.conf

其中最后是有:

[include]

;files = supervisord.d/*.ini

files = /etc/supervisord.d/*.conf

所以保证了/etc/supervisord.d/

中的supervisord_server.conf能执行到。

注意:

supervisord_server.conf

中的配置顺序是:

  • redis

  • celery worker

    • 其中用到redis

  • flask的app:robotDemo

    • 用到celery worker + redis

转载请注明:在路上 » 【已解决】把Celery+Redis集成到在线Flask中且用supervisor去管理后台服务

发表我的评论
取消评论

表情

Hi,您需要填写昵称和邮箱!

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址
81 queries in 0.167 seconds, using 22.23MB memory