为什么 python 为共享锁抛出“multiprocessing.managers.RemoteError”？答案

【问题标题】：Why python throws "multiprocessing.managers.RemoteError" for shared lock?为什么 python 为共享锁抛出“multiprocessing.managers.RemoteError”？
【发布时间】：2019-12-09 12:17:42
【问题描述】：

我在 Ubuntu 18.04 中使用 python 3.6.7

运行以下脚本后，每个进程都有自己的共享锁：

from multiprocessing import Process, Manager


def foo(l1):
    with l1:
        print('lol')


if __name__ == '__main__':
    processes = []
    with Manager() as manager:
        for cluster in range(10):
            lock1 = manager.Lock()
            calc_args = (lock1, )
            processes.append(Process(target=foo,
                                     args=calc_args))
        for p in processes:
            p.start()

        for p in processes:
            p.join()

我有一个奇怪的例外：

Process Process-2:
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "temp.py", line 5, in foo
    with l1:
  File "/usr/lib/python3.6/multiprocessing/managers.py", line 991, in __enter__
    return self._callmethod('acquire')
  File "/usr/lib/python3.6/multiprocessing/managers.py", line 772, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError: 
---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/managers.py", line 235, in serve_client
    self.id_to_local_proxy_obj[ident]
KeyError: '7f49974624e0'

任何想法或建议如何解决这个问题？

谢谢

【问题讨论】：

那么，你解决了吗？

标签： python-3.x python-multiprocessing shared-objects locks

【解决方案1】：

出于某种原因，您必须保留对从 SyncManager 获得的任何内容的原始引用，请参见下文：

from multiprocessing import Manager, Process, current_process
from multiprocessing.managers import AcquirerProxy, SyncManager

def foo(lock: AcquirerProxy):
    lock.acquire()
    print('pid={}'.format(current_process().pid))

if __name__ == '__main__':
    manager: SyncManager = Manager()

    locks = []
    for i in range(3):
        # Always keep the reference in some variable
        locks.append(manager.Lock())

    processes = []
    for i in range(3):
        p = Process(target=foo, args=[locks[i]])
        processes.append(p)

    # If you clear the list, which you lose the reference, it won't work
    # locks.clear()

    for p in processes:
        p.start()
    for p in processes:
        p.join()

很抱歉来晚了，但我希望它有所帮助！

--------- 原始回复如下：----------

嘿，这个问题在 Python 3.8.2 中仍然存在

我设法重现了错误：

from multiprocessing import Process, Manager, current_process
from multiprocessing.managers import AcquirerProxy

def foo(lock: AcquirerProxy):
    lock.acquire()
    print('pid={}'.format(current_process().pid))

if __name__ == '__main__':
    manager = Manager()

    process1 = Process(target=foo, args=[manager.Lock()])

    process1.start()
    process1.join()

但如果我取出manager.Lock()，它就可以正常工作！

from multiprocessing import Process, Manager, current_process
from multiprocessing.managers import AcquirerProxy

def foo(lock: AcquirerProxy):
    lock.acquire()
    print('pid={}'.format(current_process().pid))

if __name__ == '__main__':
    manager = Manager()

    lock1 = manager.Lock()           # Here
    process1 = Process(target=foo, args=[lock1])

    process1.start()
    process1.join()

真的很困惑为什么取出锁会有所作为

【讨论】：