Windows 上的 celery beat + django：[WinError 10061] 无法建立连接，因为目标机器主动拒绝它

【问题标题】：celery beat + django on Windows: [WinError 10061] No connection could be made because the target machine actively refused itWindows 上的 celery beat + django：[WinError 10061] 无法建立连接，因为目标机器主动拒绝它
【发布时间】：2020-06-26 16:35:44
【问题描述】：

我已经在网站上尝试过其他解决方案，但它们都不起作用，此外，许多问题都引用了现已过时的 django-celery-beat 包。我在 Django 3 和 Celery 4 上，这应该意味着，根据后者的文档，我不需要任何额外的东西来用 Django 运行 Celery。

我的项目结构如下：

proj/
|- app1/
|- app2/
|- proj/
   |- settings.py
   |- ...
   |- celery.py
   |- tasks.py

（我还尝试将tasks.py 放入其中一个应用程序中。）

celery.py：

import os
from celery import Celery
from celery.schedules import crontab


os.environ.setdefault("DJANGO_SETTINGS_MODULE", "proj.settings")
app = Celery("proj")

app.config_from_object("django.conf:settings", namespace="CELERY")
app.autodiscover_tasks()


@app.on_after_configure.connect
def setup_periodic_tasks(sender, **_):
    from .tasks import debug_task
    sender.add_periodic_task(crontab(), debug_task.apply_async())

tasks.py：

from celery import shared_task


@shared_task
def debug_task():
    print("Task executed.")

我尝试运行该项目的方法是：

  1. py manage.py runserver,
  2. celery -A proj.celery beat,
  3. celery -A proj.celery worker.

在 Ubuntu 上，我会使用分离的 screen 会话运行所有三个命令，以在同一个终端上运行所有内容，以防万一这很重要，但我找不到任何适用于 Windows 的可行替代方案，因此所有三个命令都在单独的终端中执行在前一个命令已经运行后打开。我试着把“beat”和“worker”放在“celery”之后，以防万一。
我还通过 PyCharm 的运行功能（对于 Django 的服务器）和其他两个命令的两个单独的终端实例进行了相同的尝试。如果有更好的程序可以遵循，请随时提出建议。

芹菜节拍的输出：

celery beat v4.4.0 (cliffs) is starting.
__    -    ... __   -        _
LocalTime -> 2020-03-14 16:48:32
Configuration ->
    . broker -> amqp://guest:**@localhost:5672//
    . loader -> celery.loaders.app.AppLoader
    . scheduler -> celery.beat.PersistentScheduler
    . db -> celerybeat-schedule
    . logfile -> [stderr]@%DEBUG
    . maxinterval -> 5.00 minutes (300s)
[2020-03-14 16:48:32,042: DEBUG/MainProcess] Setting default socket timeout to 30
[2020-03-14 16:48:32,044: INFO/MainProcess] beat: Starting...
[2020-03-14 16:48:32,288: DEBUG/MainProcess] Current schedule:
<ScheduleEntry: celery.backend_cleanup celery.backend_cleanup() <crontab: 0 4 * * * (m/h/d/dM/MY)>
[2020-03-14 16:48:32,288: DEBUG/MainProcess] beat: Ticking with max interval->5.00 minutes
[2020-03-14 16:48:32,292: DEBUG/MainProcess] beat: Waking up in 5.00 minutes.

有错误的 Celery worker 的输出：

[2020-03-14 16:49:50,319: DEBUG/MainProcess] | Worker: Preparing bootsteps.
[2020-03-14 16:49:50,326: DEBUG/MainProcess] | Worker: Building graph...
[2020-03-14 16:49:50,327: DEBUG/MainProcess] | Worker: New boot order: {Timer, Hub, Pool, Autoscaler, Beat, StateDB, Consumer}
[2020-03-14 16:49:50,472: DEBUG/MainProcess] | Consumer: Preparing bootsteps.
[2020-03-14 16:49:50,473: DEBUG/MainProcess] | Consumer: Building graph...
[2020-03-14 16:49:50,593: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Agent, Events, Heart, Mingle, Tasks, Control, Gossip, event loop}

 -------------- celery@MY-LATPTOPS-NAME v4.4.0 (cliffs)
--- ***** -----
-- ******* ---- Windows-10-10.0.18362-SP0 2020-03-14 16:49:50
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app:         proj:0x3f91ef8
- ** ---------- .> transport:   amqp://guest:**@localhost:5672//
- ** ---------- .> results:     disabled://
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery


[tasks]
  . celery.accumulate
  . celery.backend_cleanup
  . celery.chain
  . celery.chord
  . celery.chord_unlock
  . celery.chunks
  . celery.group
  . celery.map
  . celery.starmap
  . proj.tasks.debug_task

[2020-03-14 16:49:50,600: DEBUG/MainProcess] | Worker: Starting Pool
[2020-03-14 16:49:50,614: WARNING/MainProcess] e:\path_to\my_project\venv38\lib\site-packages\billiard\pool.py:1022: UserWarning: Soft timeouts are not supported: on this platform: It does not have the SIGUSR1 signal.
  warnings.warn(UserWarning(
[2020-03-14 16:49:50,736: DEBUG/MainProcess] ^-- substep ok
[2020-03-14 16:49:50,737: DEBUG/MainProcess] | Worker: Starting Consumer
[2020-03-14 16:49:50,738: DEBUG/MainProcess] | Consumer: Starting Connection
[2020-03-14 16:49:51,250: DEBUG/MainProcess] recreated blocker with handle 24
[2020-03-14 16:49:51,251: DEBUG/MainProcess] recreated blocker with handle 32
[2020-03-14 16:49:51,253: DEBUG/MainProcess] recreated blocker with handle 20
[2020-03-14 16:49:51,256: DEBUG/MainProcess] recreated blocker with handle 24
[2020-03-14 16:49:51,267: DEBUG/MainProcess] recreated blocker with handle 20
[2020-03-14 16:49:51,268: DEBUG/MainProcess] recreated blocker with handle 24
[2020-03-14 16:49:51,305: INFO/SpawnPoolWorker-1] child process 3316 calling self.run()
[2020-03-14 16:49:51,309: DEBUG/MainProcess] recreated blocker with handle 24
[2020-03-14 16:49:51,310: DEBUG/MainProcess] recreated blocker with handle 28
[2020-03-14 16:49:51,315: INFO/SpawnPoolWorker-3] child process 12052 calling self.run()
[2020-03-14 16:49:51,318: INFO/SpawnPoolWorker-4] child process 13968 calling self.run()
[2020-03-14 16:49:51,360: INFO/SpawnPoolWorker-2] child process 6336 calling self.run()
[2020-03-14 16:49:52,776: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [WinError 10061] No connection could be made because the target machine actively refused it.
Trying again in 2.00 seconds...

正如您所见，工作人员正在看到任务，但从未执行过。

我认为这可能与防火墙有关，所以我在完全禁用它后重试，但没有任何改变。

该错误显然与 Windows 相关，但我需要它也可以在 Ubuntu-18.04 实例上运行，因此如果我需要在两个系统之间进行更改，请随时提及。

更新

很尴尬，我好像忘记在这台机器上设置 RabbitMQ。这样做消除了错误消息，但任务似乎仍然没有按预期运行。工作进程现在大约每秒滴答一次，初始输出（类似于上面显示的）正确显示了调试任务，但它的输出从未在任何终端（Django 的服务器、celery beat 和 celery worker）上显示，即使经过几次活动分钟数。
如何确认任务是否正在运行？

这里是初始输出celery beat -A proj.celery -l DEBUG：

celery beat v4.4.0 (cliffs) is starting.
__    -    ... __   -        _
LocalTime -> 2020-03-19 10:08:34
Configuration ->
    . broker -> amqp://guest:**@localhost:5672//
    . loader -> celery.loaders.app.AppLoader
    . scheduler -> celery.beat.PersistentScheduler
    . db -> celerybeat-schedule
    . logfile -> [stderr]@%DEBUG
    . maxinterval -> 5.00 minutes (300s)
[2020-03-19 10:08:34,789: DEBUG/MainProcess] Setting default socket timeout to 30
[2020-03-19 10:08:34,790: INFO/MainProcess] beat: Starting...
[2020-03-19 10:08:35,109: DEBUG/MainProcess] Current schedule:
<ScheduleEntry: celery.backend_cleanup celery.backend_cleanup() <crontab: 0 4 * * * (m/h/d/dM/MY)>
[2020-03-19 10:08:35,109: DEBUG/MainProcess] beat: Ticking with max interval->5.00 minutes
[2020-03-19 10:08:35,111: DEBUG/MainProcess] beat: Waking up in 5.00 minutes.

这是celery worker -A proj.celery -l DEBUG的输出：

[2020-03-19 10:08:11,288: DEBUG/MainProcess] | Worker: Preparing bootsteps.
[2020-03-19 10:08:11,296: DEBUG/MainProcess] | Worker: Building graph...
[2020-03-19 10:08:11,297: DEBUG/MainProcess] | Worker: New boot order: {StateDB, Beat, Timer, Hub, Pool, Autoscaler, Consumer}
[2020-03-19 10:08:11,475: DEBUG/MainProcess] | Consumer: Preparing bootsteps.
[2020-03-19 10:08:11,475: DEBUG/MainProcess] | Consumer: Building graph...
[2020-03-19 10:08:11,613: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Events, Heart, Mingle, Gossip, Agent, Tasks, Control, event loop}

 -------------- celery@MY-LAPTOPS-NAME v4.4.0 (cliffs)
--- ***** -----
-- ******* ---- Windows-10-10.0.18362-SP0 2020-03-19 10:08:11
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app:         proj:0x4261e50
- ** ---------- .> transport:   amqp://guest:**@localhost:5672//
- ** ---------- .> results:     disabled://
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery


[tasks]
  . celery.accumulate
  . celery.backend_cleanup
  . celery.chain
  . celery.chord
  . celery.chord_unlock
  . celery.chunks
  . celery.group
  . celery.map
  . celery.starmap
  . proj.tasks.debug_task

[2020-03-19 10:08:11,646: DEBUG/MainProcess] | Worker: Starting Pool
[2020-03-19 10:08:11,773: DEBUG/MainProcess] ^-- substep ok
[2020-03-19 10:08:11,773: DEBUG/MainProcess] | Worker: Starting Consumer
[2020-03-19 10:08:11,774: DEBUG/MainProcess] | Consumer: Starting Connection
[2020-03-19 10:08:11,842: DEBUG/MainProcess] Start from server, version: 0.9, properties: {'capabilities': {'publisher_confirms': True, 'exchange_exchange_bindings': True, 'bas
ic.nack': True, 'consumer_cancel_notify': True, 'connection.blocked': True, 'consumer_priorities': True, 'authentication_failure_close': True, 'per_consumer_qos': True, 'direct
_reply_to': True}, 'cluster_name': 'rabbit@MY-LAPTOPS-NAME', 'copyright': 'Copyright (c) 2007-2020 Pivotal Software, Inc.', 'information': 'Licensed under the MPL 1.1. Website:
 https://rabbitmq.com', 'platform': 'Erlang/OTP 22.3', 'product': 'RabbitMQ', 'version': '3.8.3'}, mechanisms: [b'AMQPLAIN', b'PLAIN'], locales: ['en_US']
[2020-03-19 10:08:11,850: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2020-03-19 10:08:11,855: DEBUG/MainProcess] ^-- substep ok
[2020-03-19 10:08:11,856: DEBUG/MainProcess] | Consumer: Starting Events
[2020-03-19 10:08:11,911: DEBUG/MainProcess] Start from server, version: 0.9, properties: {'capabilities': {'publisher_confirms': True, 'exchange_exchange_bindings': True, 'bas
ic.nack': True, 'consumer_cancel_notify': True, 'connection.blocked': True, 'consumer_priorities': True, 'authentication_failure_close': True, 'per_consumer_qos': True, 'direct
_reply_to': True}, 'cluster_name': 'rabbit@MY-LAPTOPS-NAME', 'copyright': 'Copyright (c) 2007-2020 Pivotal Software, Inc.', 'information': 'Licensed under the MPL 1.1. Website:
 https://rabbitmq.com', 'platform': 'Erlang/OTP 22.3', 'product': 'RabbitMQ', 'version': '3.8.3'}, mechanisms: [b'AMQPLAIN', b'PLAIN'], locales: ['en_US']
[2020-03-19 10:08:11,924: DEBUG/MainProcess] ^-- substep ok
[2020-03-19 10:08:11,945: DEBUG/MainProcess] | Consumer: Starting Heart
[2020-03-19 10:08:11,946: DEBUG/MainProcess] using channel_id: 1
[2020-03-19 10:08:11,961: DEBUG/MainProcess] Channel open
[2020-03-19 10:08:12,021: DEBUG/MainProcess] ^-- substep ok
[2020-03-19 10:08:12,021: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2020-03-19 10:08:12,029: DEBUG/MainProcess] | Consumer: Starting Mingle
[2020-03-19 10:08:12,030: INFO/MainProcess] mingle: searching for neighbors
[2020-03-19 10:08:12,030: DEBUG/MainProcess] using channel_id: 1
[2020-03-19 10:08:12,074: DEBUG/MainProcess] Channel open
[2020-03-19 10:08:12,141: DEBUG/MainProcess] Start from server, version: 0.9, properties: {'capabilities': {'publisher_confirms': True, 'exchange_exchange_bindings': True, 'bas
ic.nack': True, 'consumer_cancel_notify': True, 'connection.blocked': True, 'consumer_priorities': True, 'authentication_failure_close': True, 'per_consumer_qos': True, 'direct
_reply_to': True}, 'cluster_name': 'rabbit@MY-LAPTOPS-NAME', 'copyright': 'Copyright (c) 2007-2020 Pivotal Software, Inc.', 'information': 'Licensed under the MPL 1.1. Website:
 https://rabbitmq.com', 'platform': 'Erlang/OTP 22.3', 'product': 'RabbitMQ', 'version': '3.8.3'}, mechanisms: [b'AMQPLAIN', b'PLAIN'], locales: ['en_US']
[2020-03-19 10:08:12,148: DEBUG/MainProcess] using channel_id: 1
[2020-03-19 10:08:12,157: DEBUG/MainProcess] Channel open
[2020-03-19 10:08:12,461: DEBUG/MainProcess] recreated blocker with handle 292
[2020-03-19 10:08:12,462: DEBUG/MainProcess] recreated blocker with handle 296
[2020-03-19 10:08:12,497: DEBUG/MainProcess] recreated blocker with handle 20
[2020-03-19 10:08:12,498: DEBUG/MainProcess] recreated blocker with handle 24
[2020-03-19 10:08:12,504: DEBUG/MainProcess] recreated blocker with handle 20
[2020-03-19 10:08:12,505: DEBUG/MainProcess] recreated blocker with handle 24
[2020-03-19 10:08:12,514: INFO/SpawnPoolWorker-1] child process 9188 calling self.run()
[2020-03-19 10:08:12,554: INFO/SpawnPoolWorker-4] child process 9072 calling self.run()
[2020-03-19 10:08:12,562: INFO/SpawnPoolWorker-2] child process 1284 calling self.run()
[2020-03-19 10:08:12,602: DEBUG/MainProcess] recreated blocker with handle 20
[2020-03-19 10:08:12,606: DEBUG/MainProcess] recreated blocker with handle 24
[2020-03-19 10:08:12,665: INFO/SpawnPoolWorker-3] child process 13412 calling self.run()
[2020-03-19 10:08:13,030: DEBUG/MainProcess] Timer wake-up! Next ETA 0.9840000000003783 secs.
[2020-03-19 10:08:13,244: INFO/MainProcess] mingle: all alone
[2020-03-19 10:08:13,245: DEBUG/MainProcess] ^-- substep ok
[2020-03-19 10:08:13,246: DEBUG/MainProcess] | Consumer: Starting Gossip
[2020-03-19 10:08:13,246: DEBUG/MainProcess] using channel_id: 2
[2020-03-19 10:08:13,248: DEBUG/MainProcess] Channel open
[2020-03-19 10:08:13,267: DEBUG/MainProcess] ^-- substep ok
[2020-03-19 10:08:13,267: DEBUG/MainProcess] | Consumer: Starting Tasks
[2020-03-19 10:08:13,486: DEBUG/MainProcess] ^-- substep ok
[2020-03-19 10:08:13,487: DEBUG/MainProcess] | Consumer: Starting Control
[2020-03-19 10:08:13,488: DEBUG/MainProcess] using channel_id: 3
[2020-03-19 10:08:13,495: DEBUG/MainProcess] Channel open
[2020-03-19 10:08:13,518: DEBUG/MainProcess] ^-- substep ok
[2020-03-19 10:08:13,519: DEBUG/MainProcess] | Consumer: Starting event loop
[2020-03-19 10:08:13,522: INFO/MainProcess] celery@MY-LAPTOPS-NAME ready.
[2020-03-19 10:08:13,522: DEBUG/MainProcess] basic.qos: prefetch_count->16
[2020-03-19 10:08:14,023: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2020-03-19 10:08:15,027: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2020-03-19 10:08:16,028: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
...

【问题讨论】：

  • 可能是 Windows 防火墙不允许本地主机连接？
  • 我尝试禁用它，所以我怀疑就是这样，除非在设置中以某种方式禁用 Windows 防火墙是不够的。
  • 尝试手动连接 amqp://guest:**@127.0.0.1:5672 并确认 amqp 已安装
  • 感谢您的回答。请检查问题底部的更新。 @kkiat
  • 请提供您的日志记录设置。

标签： django windows-10 celery ubuntu-18.04 celerybeat


【解决方案1】：

打印功能将根据 django 默认设置输出到控制台。要输出到调试屏幕，请使用日志记录而不是打印功能。所以，tasks.py 应该如下：

from celery.utils.log import get_task_logger
from celery import shared_task

logger = get_task_logger(__name__)


@shared_task
def debug_task():
     logger.info("Task executed.")

参考： https://docs.celeryproject.org/en/latest/userguide/tasks.html#logging

【讨论】：

  • 更改此代码并让 debug_task 创建文件后，我仍然在任何终端上都看不到任何输出，并且文件没有被创建，因此很可能任务没有被执行全部。鉴于 celery 工人确实“看到”了任务，是什么原因造成的？
  • 可以用芹菜花监控任务
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2022-12-10
  • 1970-01-01
  • 2020-07-07
  • 2021-01-04
  • 2021-02-08
相关资源
最近更新 更多
热门标签
Java Python linux javascript Mysql C# Docker 算法 前端 SpringBoot Redis Vue spring 设计模式 .net core .net kubernetes c++ 数据库 数据结构 大数据 js 机器学习 微服务 Android Go 程序员 面试 JVM ASP.net core 云原生 人工智能 后端 PHP git CSS golang k8s Nginx Django mybatis 深度学习 多线程 React 架构 devops 爬虫 云计算 Spring Boot LeetCode