分叉时的 multiprocessing.Queue 行为

Question

经过几次os.fork()，我正在尝试与孩子们交换数据。为此，我使用multiprocessing.Queue 实例。当父母放置和孩子获得时，队列正常工作；但不是相反。

我的示例代码：

import os
import multiprocessing as mp
from queue import Empty

if __name__ == '__main__':

    n_workers = 5

    forward_queue = mp.Queue()
    pids_queue = mp.Queue()

    for n in range(n_workers):
        forward_queue.put(n)

    for n in range(n_workers):
        child = os.fork()
        if child:
            pass
        else:
            my_number = forward_queue.get()
            print('pid={} here, my number is {}'.format(os.getpid(), my_number))
            pids_queue.put(os.getpid())
            os._exit(0)  # correct way to exit a fork according to docs

    while True:
        try:
            pid_of_child = pids_queue.get(timeout=5)
        except Empty:
            print('no more pids')
            break
        else:
            print('one of my children had this pid={}'.format(pid_of_child))

我得到的输出：

pid=19715 here, my number is 0
pid=19716 here, my number is 1
pid=19717 here, my number is 2
pid=19721 here, my number is 3
pid=19718 here, my number is 4
no more pids

我期望的输出：

pid=19715 here, my number is 0
pid=19716 here, my number is 1
pid=19717 here, my number is 2
pid=19721 here, my number is 3
pid=19718 here, my number is 4
one of my children had this pid=19715
one of my children had this pid=19716
one of my children had this pid=19717
one of my children had this pid=19721
one of my children had this pid=19718
no more pids

谁能解释为什么会这样？

Answer 1

在你退出分叉之前试试这个：

pids_queue.close()
pids_queue.join_thread()

问题是，队列是如何工作的。将值放入队列后，将启动后台线程将项目传输到管道中。当你立即调用 os._exit 时，线程将被关闭。针对此类问题，开发了 .close 和 .join_thread 方法。