强制停止芹菜工人作为系统服务运行

Question

如何在重启服务器时杀死workers并获得与以下语句相同的效果：

pkill -9 -f 'celery worker'

来自 celery 文档：

如果worker在经过考虑的时间后仍然没有关闭，因为陷入无限循环或类似情况，您可以使用KILL信号强制终止worker：

但我是作为 systemd 服务启动的，并且具有以下配置以使用以下 systemd 单元文件启动它：

[Unit]
Description=Celery Service
After=network.target

[Service]
Type=forking
User=dsangvikar
Group=www-data
EnvironmentFile=-/etc/default/celery
WorkingDirectory=/home/dsangvikar/apps/msbot/
ExecStart=/home/dsangvikar/apps/msbot/msbotenv/bin/celery multi start \
-A microsoftbotframework.runcelery.celery chatbotworker --concurrency=4 \ 
--workdir=/home/dsangvikar/apps/msbot/ --logfile=/var/log/celery/%n.log 
--pidfile=/var/run/celery/%n.pid
ExecStop=/home/dsangvikar/apps/msbot/msbotenv/bin/celery multi stopwait
RuntimeDirectory=celery

[Install]
WantedBy=multi-user.target

当我执行sudo systemctl status celery 时，我会得到状态和 pid。我用它来杀死它。但是工作进程没有退出。我想强行杀死他们。我的程序逻辑会在每次服务器重新启动时重新创建它们。但是由于即使在系统重新启动时它们也没有被杀死，所以各种不同的问题都出现了。

Answer 1

Celery multi 不应该用于生产。

这是我正在使用的：

它启动 10 个主进程，每个进程有 2 个工作人员。所以总共有 20 个工作进程。

[program:celery_worker]
numprocs=10
process_name=%(program_name)s-%(process_num)s
directory=/opt/worker/main
environment=PATH="/opt/worker/main/bin:%(ENV_PATH)s"
command=/opt/worker/main/bin/celery worker -n worker%(process_num)s.%%h --app=python --time-limit=3600 -c 2 -Ofair -l debug --config=celery_config -E
stdout_logfile=/var/log/celery/%(program_name)s-%(process_num)s.log
user=username
autostart=true
autorestart=true
startretries=99999
startsecs=10
stopsignal=TERM
stopasgroup=false
stopwaitsecs=7200
killasgroup=true

如果您有正在运行的作业，您不希望向 PoolWorker 进程发送 TERM 信号，因为这会导致作业提前中止。您真正想要做的是将 TERM 信号发送到 MainProcess ，它将等待作业结束然后关闭。

所以你想停止主要进程，如果它被杀死，那么你想作为一个组来杀死。

使用此命令启动工作器关闭。如果工作人员未能在主管配置中的 stopwaitsecs 时间之前退出，则将发送一个终止信号，这将终止一切，因为 killasgroup 设置为 true。

sudo supervisorctl stop celery_worker:*

上面的 supervisord 配置启动示例。

username       1659  1.1  0.2 119796 45632 ?        S    10:45   0:06 [celeryd: celery@worker7.hostname:MainProcess] -active- (worker -n worker7.%h --app=python --time-limit=3600 -c 2 -Ofair -l debug --config=celery_config -E)
username       1662  1.1  0.2 119804 45716 ?        S    10:45   0:06 [celeryd: celery@worker6.hostname:MainProcess] -active- (worker -n worker6.%h --app=python --time-limit=3600 -c 2 -Ofair -l debug --config=celery_config -E)
username       1663  1.2  0.2 119724 45412 ?        S    10:45   0:06 [celeryd: celery@worker5.hostname:MainProcess] -active- (worker -n worker5.%h --app=python --time-limit=3600 -c 2 -Ofair -l debug --config=celery_config -E)
username       1666  1.1  0.2 119732 45524 ?        S    10:45   0:05 [celeryd: celery@worker4.hostname:MainProcess] -active- (worker -n worker4.%h --app=python --time-limit=3600 -c 2 -Ofair -l debug --config=celery_config -E)
username       1671  1.2  0.2 119792 45724 ?        S    10:45   0:06 [celeryd: celery@worker3.hostname:MainProcess] -active- (worker -n worker3.%h --app=python --time-limit=3600 -c 2 -Ofair -l debug --config=celery_config -E)
username       1674  1.2  0.2 119792 45420 ?        S    10:45   0:06 [celeryd: celery@worker2.hostname:MainProcess] -active- (worker -n worker2.%h --app=python --time-limit=3600 -c 2 -Ofair -l debug --config=celery_config -E)
username       1678  1.1  0.2 119712 45708 ?        S    10:45   0:05 [celeryd: celery@worker1.hostname:MainProcess] -active- (worker -n worker1.%h --app=python --time-limit=3600 -c 2 -Ofair -l debug --config=celery_config -E)
username       1679  1.2  0.2 119808 45476 ?        S    10:45   0:06 [celeryd: celery@worker0.hostname:MainProcess] -active- (worker -n worker0.%h --app=python --time-limit=3600 -c 2 -Ofair -l debug --config=celery_config -E)
username       1680  1.1  0.2 119796 45512 ?        S    10:45   0:05 [celeryd: celery@worker9.hostname:MainProcess] -active- (worker -n worker9.%h --app=python --time-limit=3600 -c 2 -Ofair -l debug --config=celery_config -E)
username       1681  1.1  0.2 119720 45736 ?        S    10:45   0:06 [celeryd: celery@worker8.hostname:MainProcess] -active- (worker -n worker8.%h --app=python --time-limit=3600 -c 2 -Ofair -l debug --config=celery_config -E)
username       1796  0.0  0.2 118160 39660 ?        S    10:45   0:00 [celeryd: celery@worker9.hostname:PoolWorker-1]
username       1797  0.0  0.2 118232 39548 ?        S    10:45   0:00 [celeryd: celery@worker8.hostname:PoolWorker-1]
username       1798  0.0  0.2 118152 39532 ?        S    10:45   0:00 [celeryd: celery@worker3.hostname:PoolWorker-1]
username       1799  0.0  0.2 118156 39652 ?        S    10:45   0:00 [celeryd: celery@worker2.hostname:PoolWorker-1]
username       1800  0.0  0.2 118168 39748 ?        S    10:45   0:00 [celeryd: celery@worker7.hostname:PoolWorker-1]
username       1801  0.0  0.2 118164 39608 ?        S    10:45   0:00 [celeryd: celery@worker6.hostname:PoolWorker-1]
username       1802  0.0  0.2 118192 39768 ?        S    10:45   0:00 [celeryd: celery@worker1.hostname:PoolWorker-1]
username       1803  0.0  0.2 118200 39728 ?        S    10:45   0:00 [celeryd: celery@worker5.hostname:PoolWorker-1]
username       1804  0.0  0.2 118168 39756 ?        S    10:45   0:00 [celeryd: celery@worker0.hostname:PoolWorker-1]
username       1805  0.0  0.2 118188 39692 ?        S    10:45   0:00 [celeryd: celery@worker4.hostname:PoolWorker-1]
username       1806  0.0  0.2 118152 39536 ?        S    10:45   0:00 [celeryd: celery@worker3.hostname:PoolWorker-2]
username       1807  0.0  0.2 118232 39544 ?        S    10:45   0:00 [celeryd: celery@worker8.hostname:PoolWorker-2]
username       1808  0.0  0.2 118164 39608 ?        S    10:45   0:00 [celeryd: celery@worker6.hostname:PoolWorker-2]
username       1809  0.0  0.2 118200 39732 ?        S    10:45   0:00 [celeryd: celery@worker5.hostname:PoolWorker-2]

如果您想立即停止，请将stopwaitsecs 设置为 1。

lpiner@hostname:~$ sudo supervisorctl status
celery_worker:celery_worker-0    RUNNING   pid 2488, uptime 0:00:48
celery_worker:celery_worker-1    RUNNING   pid 2487, uptime 0:00:48
celery_worker:celery_worker-2    RUNNING   pid 2486, uptime 0:00:48
celery_worker:celery_worker-3    RUNNING   pid 2485, uptime 0:00:48
celery_worker:celery_worker-4    RUNNING   pid 2484, uptime 0:00:48
celery_worker:celery_worker-5    RUNNING   pid 2483, uptime 0:00:48
celery_worker:celery_worker-6    RUNNING   pid 2482, uptime 0:00:48
celery_worker:celery_worker-7    RUNNING   pid 2481, uptime 0:00:48
celery_worker:celery_worker-8    RUNNING   pid 2490, uptime 0:00:48
celery_worker:celery_worker-9    RUNNING   pid 2489, uptime 0:00:48
lpiner@hostname:~$ sudo supervisorctl stop celery_worker:*
celery_worker:celery_worker-7: stopped
celery_worker:celery_worker-6: stopped
celery_worker:celery_worker-5: stopped
celery_worker:celery_worker-4: stopped
celery_worker:celery_worker-3: stopped
celery_worker:celery_worker-2: stopped
celery_worker:celery_worker-1: stopped
celery_worker:celery_worker-0: stopped
celery_worker:celery_worker-9: stopped
celery_worker:celery_worker-8: stopped
lpiner@hostname:~$ sudo supervisorctl status
celery_worker:celery_worker-0    STOPPED   Aug 02 11:17 AM
celery_worker:celery_worker-1    STOPPED   Aug 02 11:17 AM
celery_worker:celery_worker-2    STOPPED   Aug 02 11:17 AM
celery_worker:celery_worker-3    STOPPED   Aug 02 11:17 AM
celery_worker:celery_worker-4    STOPPED   Aug 02 11:17 AM
celery_worker:celery_worker-5    STOPPED   Aug 02 11:17 AM
celery_worker:celery_worker-6    STOPPED   Aug 02 11:17 AM
celery_worker:celery_worker-7    STOPPED   Aug 02 11:17 AM
celery_worker:celery_worker-8    STOPPED   Aug 02 11:17 AM
celery_worker:celery_worker-9    STOPPED   Aug 02 11:17 AM