最新消息:20190717 VPS服务器:Vultr新加坡,WordPress主题:大前端D8,统一介绍入口:关于

【临时解决】Flask出错:NoRedisConnectionException Could not resolve a Redis connection

Flask crifan 1192浏览 0评论

折腾:

【记录】再次给Flask的app添加后台任务

期间,在Flask的app中,写了对应的代码:

/Users/crifan/dev/dev_root/daryun/Projects/RunningFast/sourcecode/RunningFast-Server/runningfast/app.py

from runningfast.resources.BackgroundWorker import *

/Users/crifan/dev/dev_root/daryun/Projects/RunningFast/sourcecode/RunningFast-Server/runningfast/resources/BackgroundWorker.py

############################################################
# Current Project
############################################################
from runningfast.app import gLog
from runningfast.app import db
from runningfast.app import rq
############################################################
# Flask-RQ2 related
############################################################
@rq.job
def updateUserStatus():
    global gOnlineUserIdList
    curDatetime = datetime.now()
。。。
    return
# schedule periodic job to update user status
updateUserStatusJob = updateUserStatus.schedule(
    timedelta(seconds=BACKGROUND_UPDATE_USER_STATUS_INTERVAL_SECONDS), repeat=0)
gLog.debug("updateUserStatusJob=%s", updateUserStatusJob)

结果出错了:

对应的log是:

[2016-11-06 14:39:09 +0000] [21725] [INFO] Listening at: http://0.0.0.0:21085 (21725)
[2016-11-06 14:39:09 +0000] [21725] [INFO] Using worker: flask_sockets.worker
[2016-11-06 14:39:09 +0000] [21736] [INFO] Booting worker with pid: 21736

<div–<——————————————————————————

DEBUG in app [/root/RunningFast/staging/runningfast/app.py:90]:
app=<Flask ‘runningfast.app’>, server_port=21085, api=<flask_restful.Api object at 0x7fb8b33aedd0>, redis_store=<flask_redis.FlaskRedis object at 0x7fb8b45a4090>, db=<SQLAlchemy engine=’mysql://runningfast:Jiandao123@localhost/runningfast_dev’>, server_mode=staging, server_type=develop, rq=<flask_rq2.app.RQ object at 0x7fb8b33ca090>, sockets=<flask_sockets.Sockets object at 0x7fb8b33ca210>

<div–<——————————————————————————

[2016-11-06 14:39:09 +0000] [21736] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/root/Envs/RunningFast/lib/python2.7/site-packages/gunicorn/arbiter.py", line 557, in spawn_worker
    worker.init_process()
  File "/root/Envs/RunningFast/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 190, in init_process
    super(GeventWorker, self).init_process()
  File "/root/Envs/RunningFast/lib/python2.7/site-packages/gunicorn/workers/base.py", line 126, in init_process
    self.load_wsgi()
  File "/root/Envs/RunningFast/lib/python2.7/site-packages/gunicorn/workers/base.py", line 136, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/root/Envs/RunningFast/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/root/Envs/RunningFast/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 65, in load
    return self.load_wsgiapp()
  File "/root/Envs/RunningFast/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/root/Envs/RunningFast/lib/python2.7/site-packages/gunicorn/util.py", line 357, in import_app
    __import__(module)
  File "/root/Envs/RunningFast/lib/python2.7/site-packages/gevent/builtins.py", line 93, in __import__
    result = _import(*args, **kwargs)
  File "/root/RunningFast/staging/run.py", line 4, in <module>
    from runningfast.app import app
  File "/root/Envs/RunningFast/lib/python2.7/site-packages/gevent/builtins.py", line 93, in __import__
    result = _import(*args, **kwargs)
  File "/root/RunningFast/staging/runningfast/app.py", line 102, in <module>
    from runningfast.resources.BackgroundWorker import *
  File "/root/Envs/RunningFast/lib/python2.7/site-packages/gevent/builtins.py", line 93, in __import__
    result = _import(*args, **kwargs)
  File "/root/RunningFast/staging/runningfast/resources/BackgroundWorker.py", line 81, in <module>
    timedelta(seconds=BACKGROUND_UPDATE_USER_STATUS_INTERVAL_SECONDS), repeat=0)
  File "/root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/helpers.py", line 81, in schedule
    return self.rq.get_scheduler().schedule(
  File "/root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/app.py", line 294, in get_scheduler
    connection=self.connection,
  File "/root/Envs/RunningFast/lib/python2.7/site-packages/rq_scheduler/scheduler.py", line 25, in __init__
    self.connection = resolve_connection(connection)
  File "/root/Envs/RunningFast/lib/python2.7/site-packages/rq/connections.py", line 70, in resolve_connection
    raise NoRedisConnectionException(‘Could not resolve a Redis connection’)
NoRedisConnectionException: Could not resolve a Redis connection

很明显,是缺少了后台的redis

注:

此处,没有配置redis的配置

-》因为之前另外一个Flask的app,也没有添加redis的配置,都是用的默认的,是可以正常工作的。

所以这里是去模仿之前的操作,去再次利用Falsk-RQ2去实现后台任务。

然后去参考自己之前的:

[已解决]为Flask的后台任务而在CentOS中安装Redis

先去确认的确还存在对应的redis-server和配置文件:

(RunningFast) ➜  staging which redis-server       
/usr/local/bin/redis-server
(RunningFast) ➜  staging ls /etc/redis/redis.conf
/etc/redis/redis.conf

去运行redis的sever:

(RunningFast) ➜  staging nohup redis-server /etc/redis/redis.conf >> logs/redis.log 2>&1 &
[1] 21944
(RunningFast) ➜  staging
[1]  + 21944 exit 1     nohup redis-server /etc/redis/redis.conf >> logs/redis.log 2>&1

同时,参考:

[已解决]Flask-RQ2+redis的后台进程不工作

再去运行对应的:

rqscheduler

和:

worker

(RunningFast) ➜  staging which rqscheduler
/root/Envs/RunningFast/bin/rqscheduler
(RunningFast) ➜  staging which rq
/root/Envs/RunningFast/bin/rq
(RunningFast) ➜  staging nohup rqscheduler -i 4 > logs/rqscheduler.log 2>&1 &
[1] 22176
(RunningFast) ➜  staging
[1]  + 22176 exit 1     nohup rqscheduler -i 4 > logs/rqscheduler.log 2>&1
(RunningFast) ➜  staging nohup rq worker > logs/rq_worker.log 2>&1 &
[1] 22190

然后可以看到对应的logs:

(RunningFast) ➜  staging ll logs
total 728K
-rw-r–r– 1 root root 479K Nov  6 14:39 gunicorn.err
-rw-r–r– 1 root root    0 Nov  5 16:37 gunicorn.log
-rw-r–r– 1 root root  134 Nov  6 14:54 redis.log
-rw-r–r– 1 root root  689 Nov  6 15:06 rqscheduler.log
-rw-r–r– 1 root root  200 Nov  6 15:06 rq_worker.log
-rw-r–r– 1 root root 235K Nov  6 14:39 RunningFast.log

再去运行,结果出错:

(RunningFast) ➜  staging supervisorctl -c supervisor.conf restart runningfast_staging
runningfast_staging: ERROR (not running)
runningfast_staging: started

去看log

看到:

/Users/crifan/dev/dev_root/daryun/Projects/RunningFast/sourcecode/RunningFast-Server/logs/redis.log

nohup: ignoring input
21944:M 06 Nov 14:54:46.246 # Creating Server TCP listening socket 127.0.0.1:6379: bind: Address already in use

/Users/crifan/dev/dev_root/daryun/Projects/RunningFast/sourcecode/RunningFast-Server/logs/rq_worker.log

nohup: ignoring input
15:06:49 RQ worker u’rq:worker:AY140128113754462e2eZ.22190′ started, version 0.6.0
15:06:49 Cleaning registries for queue: default
15:06:49
15:06:49 *** Listening on default…

/Users/crifan/dev/dev_root/daryun/Projects/RunningFast/sourcecode/RunningFast-Server/logs/rqscheduler.log

nohup: ignoring input
15:06:43 Running RQ scheduler…
Traceback (most recent call last):
  File "/root/Envs/RunningFast/bin/rqscheduler", line 11, in <module>
    sys.exit(main())
  File "/root/Envs/RunningFast/lib/python2.7/site-packages/rq_scheduler/scripts/rqscheduler.py", line 53, in main
    scheduler.run(burst=args.burst)
  File "/root/Envs/RunningFast/lib/python2.7/site-packages/rq_scheduler/scheduler.py", line 336, in run
    self.register_birth()
  File "/root/Envs/RunningFast/lib/python2.7/site-packages/rq_scheduler/scheduler.py", line 33, in register_birth
    raise ValueError("There’s already an active RQ scheduler")
ValueError: There’s already an active RQ scheduler

但是错误依旧:

去看后台的redis:

(RunningFast) ➜  staging netstat -tupln
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name   
tcp        0      0 0.0.0.0:8998                0.0.0.0:*                   LISTEN      32463/java          
tcp        0      0 0.0.0.0:10022               0.0.0.0:*                   LISTEN      17286/docker-proxy  
tcp        0      0 0.0.0.0:8999                0.0.0.0:*                   LISTEN      32463/java          
tcp        0      0 0.0.0.0:9000                0.0.0.0:*                   LISTEN      19706/docker-proxy  
tcp        0      0 0.0.0.0:3306                0.0.0.0:*                   LISTEN      17851/mysqld        
tcp        0      0 127.0.0.1:6379              0.0.0.0:*                   LISTEN      19659/redis-server  
tcp        0      0 0.0.0.0:9003                0.0.0.0:*                   LISTEN      19669/docker-proxy  
tcp        0      0 0.0.0.0:80                  0.0.0.0:*                   LISTEN      2222/nginx          
tcp        0      0 127.0.0.1:8080              0.0.0.0:*                   LISTEN      24318/python        
tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      1335/sshd           
tcp        0      0 0.0.0.0:3000                0.0.0.0:*                   LISTEN      17240/docker-proxy  
tcp        0      0 0.0.0.0:8088                0.0.0.0:*                   LISTEN      29962/docker-proxy  
tcp        0      0 0.0.0.0:5432                0.0.0.0:*                   LISTEN      18421/docker-proxy  
tcp        0      0 0.0.0.0:8090                0.0.0.0:*                   LISTEN      6675/java           
tcp        0      0 0.0.0.0:8091                0.0.0.0:*                   LISTEN      6675/java           
tcp        0      0 0.0.0.0:21084               0.0.0.0:*                   LISTEN      7846/python         
tcp        0      0 0.0.0.0:8093                0.0.0.0:*                   LISTEN      28556/java          
tcp        0      0 0.0.0.0:8094                0.0.0.0:*                   LISTEN      28556/java          
tcp        0      0 0.0.0.0:8000                0.0.0.0:*                   LISTEN      19505/docker-proxy  
udp        0      0 192.168.42.1:123            0.0.0.0:*                               1346/ntpd           
udp        0      0 115.29.173.126:123          0.0.0.0:*                               1346/ntpd           
udp        0      0 10.161.170.247:123          0.0.0.0:*                               1346/ntpd           
udp        0      0 127.0.0.1:123               0.0.0.0:*                               1346/ntpd           
udp        0      0 0.0.0.0:123                 0.0.0.0:*                               1346/ntpd  

是有的:

tcp        0      0 127.0.0.1:6379              0.0.0.0:*                   LISTEN      19659/redis-server

搜:

python rq NoRedisConnectionException Could not resolve a Redis connection

Could not resolve a Redis connection · Issue #164 · nvie/rq

get_current_job() throws NoRedisConnectionException in 0.5.0 · Issue #479 · nvie/rq

rq/connections.py at master · nvie/rq

redis – How to get Job by id in RQ python? – Stack Overflow

难道是:

当前的服务器的另外一个Flask的app已经连接了已有的redis的server

所以另外的flask的app,无法连接redis了?

NoRedisConnectionException Could not resolve a Redis connection

get_current_job() with async=False should return None instead of throwing an exception · Issue #195 · nvie/rq

此处确认redis是正常工作的:

(RunningFast) ➜  staging redis-cli
127.0.0.1:6379> ping
PONG
127.0.0.1:6379> quit

-》

估计是:

rq worker

rqscheduler

没有正常工作?

当前好像是有rqworker工作的:

(RunningFast) ➜  staging ps aux | grep rq
root         3  0.0  0.0      0     0 ?        S    Jun05   0:59 [ksoftirqd/0]
root        43  0.0  0.0      0     0 ?        S    Jun05   0:33 [ksoftirqd/1]
root     22190  0.0  0.3 201516 12540 pts/2    SN   15:06   0:00 /root/Envs/RunningFast/bin/python /root/Envs/RunningFast/bin/rq worker
root     22472  0.0  0.0 103368   820 pts/2    S+   15:28   0:00 grep –color=auto –exclude-dir=.bzr –exclude-dir=CVS –exclude-dir=.git –exclude-dir=.hg –exclude-dir=.svn rq
root     26804  0.0  0.3 194612 12264 ?        SN   Sep04   2:35 /root/Envs/SIPEvents/bin/python /root/Envs/SIPEvents/bin/rqscheduler -i 30
root     26813  0.0  0.3 199684 12796 ?        SN   Sep04   0:08 /root/Envs/SIPEvents/bin/python /root/Envs/SIPEvents/bin/rq worker

不过好像rqscheduler只有一个。

再去看rqscheduler的log:

There’s already an active RQ scheduler

-》

估计同时只能运行一个rqshecduler

Flask RQ2 NoRedisConnectionException

此处看了看安装的库,好像也是对的:

Flask-Redis==0.3.0
Flask-RQ2==16.1.1
redis==2.10.5
rq==0.6.0
rq-scheduler==0.7.0

NoRedisConnectionException on starting app · Issue #2 · jezdez/Flask-RQ2

貌似和我类似的问题?

jezdez/Flask-RQ2: A Flask extension for RQ.

  • worker — Starts an RQ worker (required to run jobs).
  • scheduler — Starts an RQ Scheduler (optional for scheduled jobs).

-》

才明白:

worker是针对于(无需调度,单次执行的)job的

scheduler:针对于需要调度,在不同时间,运行(多次)的任务的

搜:

Creating Server TCP listening socket 127.0.0.1:6379: bind: Address already in use

redis Creating Server TCP listening socket 127.0.0.1:6379: bind: Address already in use

redis-server in ubuntu14.04: Bind address already in use – Stack Overflow

去杀掉redis,重启

(RunningFast) ➜  staging netstat -tupln | grep redis
tcp        0      0 127.0.0.1:6379              0.0.0.0:*                   LISTEN      19659/redis-server  
(RunningFast) ➜  staging kill -9 19659
(RunningFast) ➜  staging
[1]  + 22190 exit 1     nohup rq worker > logs/rq_worker.log 2>&1
(RunningFast) ➜  staging netstat -tupln | grep redis
(RunningFast) ➜  staging nohup redis-server /etc/redis/redis.conf >> logs/redis.log 2>&1 &
[1] 22761

去看log:

/Users/crifan/dev/dev_root/daryun/Projects/RunningFast/sourcecode/RunningFast-Server/logs/redis.log

nohup: ignoring input
21944:M 06 Nov 14:54:46.246 # Creating Server TCP listening socket 127.0.0.1:6379: bind: Address already in use
nohup: ignoring input
                _._                                                 
           _.-“__ ”-._                                            
      _.-“    `.  `_.  ”-._           Redis 3.2.3 (00000000/0) 64 bit
  .-“ .-“`.  “`\/    _.,_ ”-._                                  
(    ‘      ,       .-`  | `,    )     Running in standalone mode
|`-._`-…-` __…-.“-._|’` _.-‘|     Port: 6379
|    `-._   `._    /     _.-‘    |     PID: 22761
  `-._    `-._  `-./  _.-‘    _.-‘                                  
|`-._`-._    `-.__.-‘    _.-‘_.-‘|                                 
|    `-._`-._        _.-‘_.-‘    |           http://redis.io       
  `-._    `-._`-.__.-‘_.-‘    _.-‘                                  
|`-._`-._    `-.__.-‘    _.-‘_.-‘|                                 
|    `-._`-._        _.-‘_.-‘    |                                 
  `-._    `-._`-.__.-‘_.-‘    _.-‘                                  
      `-._    `-.__.-‘    _.-‘                                      
          `-._        _.-‘                                          
              `-.__.-‘                                              
22761:M 06 Nov 15:50:25.575 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
22761:M 06 Nov 15:50:25.575 # Server started, Redis version 3.2.3
22761:M 06 Nov 15:50:25.575 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add ‘vm.overcommit_memory = 1’ to /etc/sysctl.conf and then reboot or run the command ‘sysctl vm.overcommit_memory=1’ for this to take effect.
22761:M 06 Nov 15:50:25.575 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command ‘echo never > /sys/kernel/mm/transparent_hugepage/enabled’ as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
22761:M 06 Nov 15:50:25.575 * The server is now ready to accept connections on port 6379

然后再去运行程序看看。

错误依旧

NoRedisConnectionException: Could not resolve a Redis connection

再去重启:

(RunningFast) ➜  staging nohup rqscheduler -i 4 > logs/rqscheduler.log 2>&1 &
[2] 22839
(RunningFast) ➜  staging nohup rq worker > logs/rq_worker.log 2>&1 &
[3] 22850

看log,好像rq worker和rqscheduler没问题:

/Users/crifan/dev/dev_root/daryun/Projects/RunningFast/sourcecode/RunningFast-Server/logs/rq_worker.log

nohup: ignoring input
15:52:47 RQ worker u’rq:worker:AY140128113754462e2eZ.22850′ started, version 0.6.0
15:52:47 Cleaning registries for queue: default
15:52:47
15:52:47 *** Listening on default…

/Users/crifan/dev/dev_root/daryun/Projects/RunningFast/sourcecode/RunningFast-Server/logs/rqscheduler.log

nohup: ignoring input
15:52:40 Running RQ scheduler…
15:52:40 Checking for scheduled jobs…
15:52:44 Checking for scheduled jobs…
15:52:48 Checking for scheduled jobs…
15:52:52 Checking for scheduled jobs…
15:52:56 Checking for scheduled jobs…

参考了之前,可以工作的时候的库的版本,发现是:

Flask-Redis==0.3.0
Flask-RQ2==16.0.2
redis==2.10.5
rq==0.6.0
rq-scheduler==0.7.0

唯一不一样的只是:

之前工作的是:

Flask-RQ2==16.0.2

现在不工作的是:

Flask-RQ2==16.1.1

要不去把Flask-RQ2降版本到16.0.2

看看效果?

实在不行,再去试试这个办法。

搜:

self.rq.get_scheduler().schedule

参考:

Flask-RQ2/helpers.py at master · jezdez/Flask-RQ2

    def schedule(self, time_or_delta, *args, **kwargs):
        """
        A function to schedule running a RQ job at a given time
        or after a given timespan::
            @rq.job
            def add(x, y):
                return x + y
            add.schedule(timedelta(hours=2), 1, 2)
            add.schedule(datetime(2016, 12, 31, 23, 59, 59), 1, 2)
            add.schedule(timedelta(days=14), 1, 2, repeat=1)
        """
        repeat = kwargs.pop(‘repeat’, None)
        interval = kwargs.pop(‘interval’, None)
        description = kwargs.pop(‘description’, None)
        job_id = kwargs.pop(‘job_id’, None)
        if isinstance(time_or_delta, timedelta):
            time = datetime.utcnow() + time_or_delta
        else:
            time = time_or_delta
        return self.rq.get_scheduler().schedule(
            time,
            self.wrapped,
            args=args,
            kwargs=kwargs,
            interval=interval,
            repeat=repeat,
            result_ttl=self.result_ttl,
            ttl=self.ttl,
            timeout=self.timeout,
            id=job_id,
            description=description,
            queue_name=self.queue_name,
        )

->

https://github.com/jezdez/Flask-RQ2/blob/master/src/flask_rq2/app.py

    def get_scheduler(self, interval=None):
        """
        When installed returns a “rq_scheduler.Scheduler“ instance to
        schedule job execution, e.g.::
            scheduler = rq.get_scheduler(interval=10)
        :param interval: Time in seconds of the periodic check for scheduled
                         jobs.
        :type interval: int
        """
        if self.scheduler_cls is None:
            raise RuntimeError(‘Cannot import rq-scheduler. Is it installed?’)
        if interval is None:
            interval = self.scheduler_interval
        return self.scheduler_cls(
            queue_name=self.scheduler_queue,
            interval=interval,
            connection=self.connection,
        )

->

ui/rq-scheduler: A light library that adds job scheduling capabilities to RQ (Redis Queue)

-》

rq-scheduler/scheduler.py at master · ui/rq-scheduler

class Scheduler(object):
    scheduler_key = ‘rq:scheduler’
    scheduled_jobs_key = ‘rq:scheduler:scheduled_jobs’
    def __init__(self, queue_name=’default’, interval=60, connection=None):
        from rq.connections import resolve_connection
        self.connection = resolve_connection(connection)
        self.queue_name = queue_name
        self._interval = interval
        self.log = logger

-》

nvie/rq: Simple job queues for Python

-》

rq/connections.py at master · nvie/rq

def push_connection(redis):
    """Pushes the given connection on the stack."""
    _connection_stack.push(patch_connection(redis))
def pop_connection():
    """Pops the topmost connection from the stack."""
    return _connection_stack.pop()
def use_connection(redis=None):
    """Clears the stack and uses the given connection.  Protects against mixed
    use of use_connection() and stacked connection contexts.
    """
    assert len(_connection_stack) <= 1, \
        ‘You should not mix Connection contexts with use_connection()’
    release_local(_connection_stack)
    if redis is None:
        redis = StrictRedis()
    push_connection(redis)
def get_current_connection():
    """Returns the current Redis connection (i.e. the topmost on the
    connection stack).
    """
    return _connection_stack.top
def resolve_connection(connection=None):
    """Convenience function to resolve the given or the current connection.
    Raises an exception if it cannot resolve a connection now.
    """
    if connection is not None:
        return patch_connection(connection)
    connection = get_current_connection()
    if connection is None:
        raise NoRedisConnectionException(‘Could not resolve a Redis connection’)
    return connection

然后这里也有类似问题:

get_current_job() throws NoRedisConnectionException in 0.5.0 · Issue #479 · nvie/rq

对于rq,之前0.4.6也出现过类似bug,不过在0.5.1就解决了。

不过,难道是我此处的:

两个Flask的app,同时连接同一个本地的redis

redis://localhost:6379/0

所以连不上?

redis multiple connection NoRedisConnectionException

RQ: Connections

->

use_connection,用于实现单连接,(虽然很简单,但是会污染全局作用域namespace)所以已经废弃,不推荐使用了。

“Be aware of the fact that use_connection pollutes the global namespace. It also implies that you can only ever use a single connection.”

此处,有两个连接,所以出错了,应该就是这个原因。

不过好像又不是:

其rq已经自己实现了?

push_connection

pop_connection

了?

不是,上面是rq的自己的内部实现。

不是别人调用这两个函数。

Creating and managing multiple connections in Redis Python – Stack Overflow

提到了:

redis的另外的database,1(默认时0)

-》去搞清楚,如何创建redis的database 1

Redis Clients Handling – Redis

flask redis database 1

flask redis db

flask second redis db

Creating and managing multiple connections in Redis Python – Stack Overflow

算了,去参考:

jezdez/Flask-RQ2: A Flask extension for RQ.

“RQ_REDIS_URL

The URL to pass to redis-py’s Redis.from_url classmethod to create a Redis connetion pool. Defaults to connecting to the local Redis instance, database 0.

app.config[‘RQ_REDIS_URL’] = ‘redis://localhost:6379/0’”

去配置为1:

结果无论是:

/Users/crifan/dev/dev_root/daryun/Projects/RunningFast/sourcecode/RunningFast-Server/config.py

############################################################
# Flask-RQ2
############################################################
RQ_REDIS_URL = ‘redis://localhost:6379/1’

还是:

/Users/crifan/dev/dev_root/daryun/Projects/RunningFast/sourcecode/RunningFast-Server/runningfast/app.py

app.config[‘RQ_REDIS_URL’] = ‘redis://localhost:6379/1’
app.config[‘RQ_QUEUES’] = [‘default’]
app.config[‘RQ_ASYNC’] = True
app.config[‘RQ_SCHEDULER_QUEUE’] = ‘default’
app.config[‘RQ_SCHEDULER_INTERVAL’] = 3

都是错误依旧。

去降级:

Flask-RQ2降版本到16.0.2

试试

(RunningFast) ➜  staging pip uninstall flask-rq2
Uninstalling Flask-RQ2-16.1.1:
  /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.1.1.dist-info/DESCRIPTION.rst
  /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.1.1.dist-info/INSTALLER
  /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.1.1.dist-info/METADATA
  /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.1.1.dist-info/RECORD
  /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.1.1.dist-info/WHEEL
  /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.1.1.dist-info/metadata.json
  /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.1.1.dist-info/top_level.txt
  /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/__init__.py
  /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/__init__.pyc
  /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/app.py
  /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/app.pyc
  /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/cli.py
  /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/cli.pyc
  /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/helpers.py
  /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/helpers.pyc
  /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/script.py
  /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/script.pyc
Proceed (y/n)? y
  Successfully uninstalled Flask-RQ2-16.1.1
You are using pip version 8.1.2, however version 9.0.0 is available.
You should consider upgrading via the ‘pip install –upgrade pip’ command.
(RunningFast) ➜  staging pip install flask-rq2==16.0.2
Collecting flask-rq2==16.0.2
  Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by ‘NewConnectionError(‘<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f7fcf5047d0>: Failed to establish a new connection: [Errno 97] Address family not supported by protocol’,)’: /simple/flask-rq2/
  Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connection broken by ‘ReadTimeoutError("HTTPSConnectionPool(host=’pypi.python.org’, port=443): Read timed out. (read timeout=15)",)’: /simple/flask-rq2/
  Using cached Flask_RQ2-16.0.2-py2.py3-none-any.whl
Requirement already satisfied (use –upgrade to upgrade): rq-scheduler>=0.6.1 in /root/Envs/RunningFast/lib/python2.7/site-packages (from flask-rq2==16.0.2)
Requirement already satisfied (use –upgrade to upgrade): Flask>=0.10 in /root/Envs/RunningFast/lib/python2.7/site-packages (from flask-rq2==16.0.2)
Requirement already satisfied (use –upgrade to upgrade): rq>=0.6.0 in /root/Envs/RunningFast/lib/python2.7/site-packages (from flask-rq2==16.0.2)
Requirement already satisfied (use –upgrade to upgrade): croniter>=0.3.9 in /root/Envs/RunningFast/lib/python2.7/site-packages (from rq-scheduler>=0.6.1->flask-rq2==16.0.2)
Requirement already satisfied (use –upgrade to upgrade): itsdangerous>=0.21 in /root/Envs/RunningFast/lib/python2.7/site-packages (from Flask>=0.10->flask-rq2==16.0.2)
Requirement already satisfied (use –upgrade to upgrade): Jinja2>=2.4 in /root/Envs/RunningFast/lib/python2.7/site-packages (from Flask>=0.10->flask-rq2==16.0.2)
Requirement already satisfied (use –upgrade to upgrade): Werkzeug>=0.7 in /root/Envs/RunningFast/lib/python2.7/site-packages (from Flask>=0.10->flask-rq2==16.0.2)
Requirement already satisfied (use –upgrade to upgrade): click>=2.0 in /root/Envs/RunningFast/lib/python2.7/site-packages (from Flask>=0.10->flask-rq2==16.0.2)
Requirement already satisfied (use –upgrade to upgrade): redis>=2.7.0 in /root/Envs/RunningFast/lib/python2.7/site-packages (from rq>=0.6.0->flask-rq2==16.0.2)
Requirement already satisfied (use –upgrade to upgrade): python-dateutil in /root/Envs/RunningFast/lib/python2.7/site-packages (from croniter>=0.3.9->rq-scheduler>=0.6.1->flask-rq2==16.0.2)
Requirement already satisfied (use –upgrade to upgrade): setuptools in /root/Envs/RunningFast/lib/python2.7/site-packages (from croniter>=0.3.9->rq-scheduler>=0.6.1->flask-rq2==16.0.2)
Requirement already satisfied (use –upgrade to upgrade): MarkupSafe in /root/Envs/RunningFast/lib/python2.7/site-packages (from Jinja2>=2.4->Flask>=0.10->flask-rq2==16.0.2)
Requirement already satisfied (use –upgrade to upgrade): six>=1.5 in /root/Envs/RunningFast/lib/python2.7/site-packages (from python-dateutil->croniter>=0.3.9->rq-scheduler>=0.6.1->flask-rq2==16.0.2)
Installing collected packages: flask-rq2
Successfully installed flask-rq2-16.0.2
You are using pip version 8.1.2, however version 9.0.0 is available.
You should consider upgrading via the ‘pip install –upgrade pip’ command.

然后再去试试。

结果问题依旧。

再换回最新的Flask-RQ2:

(RunningFast) ➜  staging pip uninstall flask-rq2
Uninstalling Flask-RQ2-16.0.2:
  /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.0.2.dist-info/DESCRIPTION.rst
  /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.0.2.dist-info/INSTALLER
  /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.0.2.dist-info/METADATA
  /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.0.2.dist-info/RECORD
  /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.0.2.dist-info/WHEEL
  /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.0.2.dist-info/metadata.json
  /root/Envs/RunningFast/lib/python2.7/site-packages/Flask_RQ2-16.0.2.dist-info/top_level.txt
  /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/__init__.py
  /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/__init__.pyc
  /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/app.py
  /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/app.pyc
  /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/cli.py
  /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/cli.pyc
  /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/helpers.py
  /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/helpers.pyc
  /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/script.py
  /root/Envs/RunningFast/lib/python2.7/site-packages/flask_rq2/script.pyc
Proceed (y/n)? y
  Successfully uninstalled Flask-RQ2-16.0.2
You are using pip version 8.1.2, however version 9.0.0 is available.
You should consider upgrading via the ‘pip install –upgrade pip’ command.
(RunningFast) ➜  staging pip install flask-rq2
Collecting flask-rq2
  Using cached Flask_RQ2-16.1.1-py2.py3-none-any.whl
Requirement already satisfied (use –upgrade to upgrade): rq-scheduler>=0.6.1 in /root/Envs/RunningFast/lib/python2.7/site-packages (from flask-rq2)
Requirement already satisfied (use –upgrade to upgrade): Flask>=0.10 in /root/Envs/RunningFast/lib/python2.7/site-packages (from flask-rq2)
Requirement already satisfied (use –upgrade to upgrade): rq>=0.6.0 in /root/Envs/RunningFast/lib/python2.7/site-packages (from flask-rq2)
Requirement already satisfied (use –upgrade to upgrade): croniter>=0.3.9 in /root/Envs/RunningFast/lib/python2.7/site-packages (from rq-scheduler>=0.6.1->flask-rq2)
Requirement already satisfied (use –upgrade to upgrade): itsdangerous>=0.21 in /root/Envs/RunningFast/lib/python2.7/site-packages (from Flask>=0.10->flask-rq2)
Requirement already satisfied (use –upgrade to upgrade): Jinja2>=2.4 in /root/Envs/RunningFast/lib/python2.7/site-packages (from Flask>=0.10->flask-rq2)
Requirement already satisfied (use –upgrade to upgrade): Werkzeug>=0.7 in /root/Envs/RunningFast/lib/python2.7/site-packages (from Flask>=0.10->flask-rq2)
Requirement already satisfied (use –upgrade to upgrade): click>=2.0 in /root/Envs/RunningFast/lib/python2.7/site-packages (from Flask>=0.10->flask-rq2)
Requirement already satisfied (use –upgrade to upgrade): redis>=2.7.0 in /root/Envs/RunningFast/lib/python2.7/site-packages (from rq>=0.6.0->flask-rq2)
Requirement already satisfied (use –upgrade to upgrade): python-dateutil in /root/Envs/RunningFast/lib/python2.7/site-packages (from croniter>=0.3.9->rq-scheduler>=0.6.1->flask-rq2)
Requirement already satisfied (use –upgrade to upgrade): setuptools in /root/Envs/RunningFast/lib/python2.7/site-packages (from croniter>=0.3.9->rq-scheduler>=0.6.1->flask-rq2)
Requirement already satisfied (use –upgrade to upgrade): MarkupSafe in /root/Envs/RunningFast/lib/python2.7/site-packages (from Jinja2>=2.4->Flask>=0.10->flask-rq2)
Requirement already satisfied (use –upgrade to upgrade): six>=1.5 in /root/Envs/RunningFast/lib/python2.7/site-packages (from python-dateutil->croniter>=0.3.9->rq-scheduler>=0.6.1->flask-rq2)
Installing collected packages: flask-rq2
Successfully installed flask-rq2-16.1.1
You are using pip version 8.1.2, however version 9.0.0 is available.
You should consider upgrading via the ‘pip install –upgrade pip’ command.

看了半天,感觉:

get_current_job() throws NoRedisConnectionException in 0.5.0 · Issue #479 · nvie/rq

就是个bug,到目前的最新的0.6.0,还是没解决。。。

那人给出的解决办法:

diff –git a/rq/cli/cli.py b/rq/cli/cli.py
index 6b52bd7..97c5e4d 100755

<div–<- a/rq/cli/cli.py

+++ b/rq/cli/cli.py
@@ -13,6 +13,7 @@ from redis import StrictRedis
from redis.exceptions import ConnectionError
from rq import Connection, get_failed_queue, Queue
+from rq.connections import push_connection
from rq.contrib.legacy import cleanup_ghosts
from rq.exceptions import InvalidJobOperationError
from rq.utils import import_attribute
@@ -171,6 +172,7 @@ def worker(url, config, burst, name, worker_class, job_class, queue_class, path,
     setup_loghandlers_from_args(verbose, quiet)
     conn = connect(url, config)
+    push_connection(conn)
     cleanup_ghosts(conn)
     worker_class = import_attribute(worker_class)
     queue_class = import_attribute(queue_class)

目前0.6.0的代码,还是没有修复:

rq/cli.py at master · nvie/rq

def worker(url, config, burst, name, worker_class, job_class, queue_class, connection_class, 。。。
    setup_loghandlers_from_args(verbose, quiet)
    connection_class = import_attribute(connection_class)
    conn = connect(url, config, connection_class)
    cleanup_ghosts(conn)
    worker_class = import_attribute(worker_class)
    queue_class = import_attribute(queue_class)

去看看自己此处的代码:

同样的:

(RunningFast) ➜  staging cat /root/Envs/RunningFast/lib/python2.7/site-packages/rq/cli/cli.py
def worker(url, config, burst, name, worker_class, job_class, queue_class, path, results_ttl, 。。。
    setup_loghandlers_from_args(verbose, quiet)
    conn = connect(url, config)
    cleanup_ghosts(conn)
    worker_class = import_attribute(worker_class)
    queue_class = import_attribute(queue_class)
    exception_handlers = []

去手动加上试试:

vim /root/Envs/RunningFast/lib/python2.7/site-packages/rq/cli/cli.py

加上:

from rq.connections import push_connection

和:

push_connection(conn)

然后重新启动那个rq worker:

(RunningFast) ➜  staging nohup rq worker > logs/rq_worker.log 2>&1 &
[2] 24162

log中也是对应的24162:

nohup: ignoring input
17:18:32 RQ worker u’rq:worker:AY140128113754462e2eZ.24162′ started, version 0.6.0
17:18:32 Cleaning registries for queue: default
17:18:32
17:18:32 *** Listening on default…

然后重启Flask的app试试。

问题依旧。

重启redis,rq worker,rqscheduler,结果还是不行:

(RunningFast) ➜  staging ps aux | grep rq
root         3  0.0  0.0      0     0 ?        S    Jun05   0:59 [ksoftirqd/0]
root        43  0.0  0.0      0     0 ?        S    Jun05   0:33 [ksoftirqd/1]
root     22839  0.0  0.3 196428 12136 pts/2    SN   15:52   0:01 /root/Envs/RunningFast/bin/python /root/Envs/RunningFast/bin/rqscheduler -i 4
root     22850  0.0  0.3 201516 12536 pts/2    SN   15:52   0:00 /root/Envs/RunningFast/bin/python /root/Envs/RunningFast/bin/rq worker
root     24147  0.0  0.3 201772 12556 pts/5    SN   17:17   0:00 /root/Envs/RunningFast/bin/python /root/Envs/RunningFast/bin/rq worker
root     24162  0.0  0.3 201516 12540 pts/5    SN   17:18   0:00 /root/Envs/RunningFast/bin/python /root/Envs/RunningFast/bin/rq worker
root     24303  0.0  0.0 103368   816 pts/5    S+   17:24   0:00 grep –color=auto –exclude-dir=.bzr –exclude-dir=CVS –exclude-dir=.git –exclude-dir=.hg –exclude-dir=.svn rq
(RunningFast) ➜  staging netstat -tupln
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name   
tcp        0      0 0.0.0.0:8998                0.0.0.0:*                   LISTEN      32463/java          
tcp        0      0 0.0.0.0:10022               0.0.0.0:*                   LISTEN      17286/docker-proxy  
tcp        0      0 0.0.0.0:8999                0.0.0.0:*                   LISTEN      32463/java          
tcp        0      0 0.0.0.0:9000                0.0.0.0:*                   LISTEN      19706/docker-proxy  
tcp        0      0 0.0.0.0:3306                0.0.0.0:*                   LISTEN      17851/mysqld        
tcp        0      0 127.0.0.1:6379              0.0.0.0:*                   LISTEN      22761/redis-server  
tcp        0      0 0.0.0.0:9003                0.0.0.0:*                   LISTEN      19669/docker-proxy  
tcp        0      0 0.0.0.0:80                  0.0.0.0:*                   LISTEN      2222/nginx          
tcp        0      0 127.0.0.1:8080              0.0.0.0:*                   LISTEN      24318/python        
tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      1335/sshd           
tcp        0      0 0.0.0.0:3000                0.0.0.0:*                   LISTEN      17240/docker-proxy  
tcp        0      0 0.0.0.0:8088                0.0.0.0:*                   LISTEN      29962/docker-proxy  
tcp        0      0 0.0.0.0:5432                0.0.0.0:*                   LISTEN      18421/docker-proxy  
tcp        0      0 0.0.0.0:8090                0.0.0.0:*                   LISTEN      6675/java           
tcp        0      0 0.0.0.0:8091                0.0.0.0:*                   LISTEN      6675/java           
tcp        0      0 0.0.0.0:21084               0.0.0.0:*                   LISTEN      7846/python         
tcp        0      0 0.0.0.0:8093                0.0.0.0:*                   LISTEN      28556/java          
tcp        0      0 0.0.0.0:8094                0.0.0.0:*                   LISTEN      28556/java          
tcp        0      0 0.0.0.0:8000                0.0.0.0:*                   LISTEN      19505/docker-proxy  
udp        0      0 192.168.42.1:123            0.0.0.0:*                               1346/ntpd           
udp        0      0 115.29.173.126:123          0.0.0.0:*                               1346/ntpd           
udp        0      0 10.161.170.247:123          0.0.0.0:*                               1346/ntpd           
udp        0      0 127.0.0.1:123               0.0.0.0:*                               1346/ntpd           
udp        0      0 0.0.0.0:123                 0.0.0.0:*                               1346/ntpd           
(RunningFast) ➜  staging kill -9 22761
(RunningFast) ➜  staging
[2]  + 24162 exit 1     nohup rq worker > logs/rq_worker.log 2>&1
(RunningFast) ➜  staging
[1]  + 24147 exit 1     nohup rq worker > logs/rq_worker.log 2>&1
(RunningFast) ➜  staging nohup redis-server /etc/redis/redis.conf >> logs/redis.log 2>&1 &
[1] 24395
(RunningFast) ➜  staging nohup rqscheduler -i 4 > logs/rqscheduler.log 2>&1 &
[2] 24403
(RunningFast) ➜  staging nohup rq worker > logs/rq_worker.log 2>&1 &
[3] 24412
(RunningFast) ➜  staging supervisorctl -c supervisor.conf restart runningfast_staging
runningfast_staging: ERROR (not running)
runningfast_staging: started
(RunningFast) ➜  staging supervisorctl -c supervisor.conf restart runningfast_staging
runningfast_staging: ERROR (not running)
runningfast_staging: started
(RunningFast) ➜  staging supervisorctl -c supervisor.conf restart runningfast_staging
runningfast_staging: ERROR (not running)
runningfast_staging: started

【总结】

此处,貌似是rq的bug,导致后续的别处调用rq的get_current_job却获取不到对应的redis的连接。

暂时是通过自己手动去添加:

push_connection

使得此处有了redis的连接,然后后续就可以获得到当前的连接了:

from redis import Redis
from rq import Queue, Worker
from rq.connections import push_connection
conn = Redis.from_url(u’redis://localhost:6379/0′)
queue = Queue(connection=conn)
pushConnResult = push_connection(conn)
gLog.debug("conn=%s, queue=%s, pushConnResult=%s", conn, queue, pushConnResult)

至少暂时可以正常运行了:

输出了log:

conn=Redis<ConnectionPool<Connection<host=localhost,port=6379,db=0>>>, queue=<Queue u’default’>, pushConnResult=None

此办法,很明显只是临时的办法,需要待官方有更新版本出来解决了此问题,再去更新版本。

转载请注明:在路上 » 【临时解决】Flask出错:NoRedisConnectionException Could not resolve a Redis connection

发表我的评论
取消评论

表情

Hi,您需要填写昵称和邮箱!

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址
61 queries in 0.248 seconds, using 19.05MB memory