Multiprocessing + Flask OSError: [Errno 48] Address already in use / OSError: [Errno 38] 非套接字上的套接字操作答案

【问题标题】：Multiprocessing + Flask OSError: [Errno 48] Address already in use / OSError: [Errno 38] Socket operation on non-socketMultiprocessing + Flask OSError: [Errno 48] Address already in use / OSError: [Errno 38] 非套接字上的套接字操作
【发布时间】：2022-01-16 03:50:10
【问题描述】：

我正在尝试构建一个 Flask 应用程序，该应用程序运行一个最终利用多处理库的函数。

import time
from flask import Flask, request

import multiprocessing as mp

app = Flask(__name__)

def proc():
    time.sleep(10)

@app.route('/greeting')
def greeting():
    p = mp.Process(target=proc, args=())
    p.start()
    return ('Hello there')

app.run(port=8000)

当我转到http://localhost:8000/greeting 并尝试调用p.start() 时出现以下错误。

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 125, in _main
    prepare(preparation_data)
  File "/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 268, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/michaelequi/Github/drlm/apps/mp_flask.py", line 18, in <module>
    app.run(port=8000)
  File "/Users/michaelequi/Github/drlm/venv/drlm/lib/python3.9/site-packages/flask/app.py", line 920, in run
    run_simple(t.cast(str, host), port, self, **options)
  File "/Users/michaelequi/Github/drlm/venv/drlm/lib/python3.9/site-packages/werkzeug/serving.py", line 1010, in run_simple
    inner()
  File "/Users/michaelequi/Github/drlm/venv/drlm/lib/python3.9/site-packages/werkzeug/serving.py", line 950, in inner
    srv = make_server(
  File "/Users/michaelequi/Github/drlm/venv/drlm/lib/python3.9/site-packages/werkzeug/serving.py", line 782, in make_server
    return ThreadedWSGIServer(
  File "/Users/michaelequi/Github/drlm/venv/drlm/lib/python3.9/site-packages/werkzeug/serving.py", line 688, in __init__
    super().__init__(server_address, handler)  # type: ignore
  File "/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socketserver.py", line 452, in __init__
    self.server_bind()
  File "/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/server.py", line 138, in server_bind
    socketserver.TCPServer.server_bind(self)
  File "/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socketserver.py", line 466, in server_bind
    self.socket.bind(self.server_address)
OSError: [Errno 48] Address already in use

如果我将代码更改为 debug=True (app.run(port=8000, debug=True))，则会出现不同的错误！

Traceback (most recent call last):
  File "/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 973, in _bootstrap_inner
    self.run()
  File "/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 910, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/michaelequi/Github/drlm/venv/drlm/lib/python3.9/site-packages/werkzeug/serving.py", line 950, in inner
    srv = make_server(
  File "/Users/michaelequi/Github/drlm/venv/drlm/lib/python3.9/site-packages/werkzeug/serving.py", line 782, in make_server
    return ThreadedWSGIServer(
  File "/Users/michaelequi/Github/drlm/venv/drlm/lib/python3.9/site-packages/werkzeug/serving.py", line 676, in __init__
    real_sock = socket.fromfd(fd, self.address_family, socket.SOCK_STREAM)
  File "/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py", line 545, in fromfd
    return socket(family, type, proto, nfd)
  File "/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py", line 232, in __init__
    _socket.socket.__init__(self, family, type, proto, fileno)
OSError: [Errno 38] Socket operation on non-socket

无论我在哪个端口上运行烧瓶服务器，都会发生这种情况。任何人都知道这可能是什么原因以及如何解决它？我在 MacOS Big Sur 上使用 Python 3.9.9。

【问题讨论】：

标签： python python-3.x sockets flask python-multiprocessing

【解决方案1】：

替换

app.run(port=8000)

与

if __name__ == '__main__':
    app.run(port=8000)

当使用多处理时，任何“顶级”的东西都会在每个线程中运行。所以你的主线程和你正在创建的线程都尝试打开端口 8000

【讨论】：

嗯，很有趣。这似乎可以解决它。谢谢！任何机会你都可以解释一下为什么会这样。我知道顶层的内容会在每个线程中运行，但为什么 app.run 即使多处理内容发生在完全不同的文件中也会被调用？
当启动一个新进程时，它必须重新读取你所有的代码。在这里，它必须重新读取您的文件才能知道您要求它调用的proc() 的定义。它看到app.run() 并假设它需要调用它。对于多处理，重要的是分离出只需要由主线程运行的代码。

【解决方案2】：

错误告诉我们：

OSError: [Errno 48] Address already in use

这可能是因为其他东西正在使用同一个端口，甚至是同一个程序的一个实例。

【讨论】：

这一切似乎都在程序中进行（如果杀死任何其他 python 程序并运行此脚本，问题仍然存在）。我的猜测是多处理模块正在尝试使用与烧瓶相同的端口，无论我更改烧瓶使用的端口，多处理模块都会尝试使用相同的端口。这只是一个猜测，我不确定如何修复它，但我 100% 确定这不是因为两个不同的 python 脚本正在运行并使用相同的端口。
如果你删除多处理行，它会运行吗，只留下return ('Hello there') ？
是的，这个错误特别是当我用 p.start() 行开始进程时