键盘中断与python的多处理

Question

我在使用 python 的多处理优雅地处理键盘中断时遇到问题

（是的，我知道 Ctr-C 不应该保证正常关闭 - 但让我们将讨论留到另一个线程）

考虑下面的代码，我是用户 multiprocessing.Manager#list()，它是我理解的 ListProxy，它处理对列表的多进程访问。

当我 Ctr-C 退出时 -- 我在尝试访问 ListProxy 时收到 socket.error: [Errno 2] No such file or directory

我希望共享列表不会在 Ctr-C 上被破坏。这可能吗？！

注意：我想在不使用池和队列的情况下解决这个问题。

from multiprocessing import Process, Manager
from time import sleep

def f(process_number, shared_array):
    try:
        print "starting thread: ", process_number
        shared_array.append(process_number)
        sleep(3)
        shared_array.append(process_number)
    except KeyboardInterrupt:
        print "Keyboard interrupt in process: ", process_number
    finally:
        print "cleaning up thread", process_number

if __name__ == '__main__':

    processes = []

    manager = Manager()
    shared_array = manager.list()

    for i in xrange(4):
        p = Process(target=f, args=(i, shared_array))
        p.start()
        processes.append(p)

    try:
        for process in processes:
            process.join()
    except KeyboardInterrupt:
        print "Keyboard interrupt in main"

    for item in shared_array:
        # raises "socket.error: [Errno 2] No such file or directory"
        print item

如果你运行它然后点击 Ctr-C，我们会得到以下结果：

starting thread:  0
starting thread:  1
starting thread:  3
starting thread:  2
^CKeyboard interupt in process:  3
Keyboard interupt in process:  0
cleaning up thread 3
cleaning up thread 0
Keyboard interupt in process:  1
Keyboard interupt in process:  2
cleaning up thread 1
cleaning up thread 2
Keyboard interupt in main
Traceback (most recent call last):
  File "multi.py", line 33, in <module>
    for item in shared_array:
  File "<string>", line 2, in __getitem__
  File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/managers.py", line 755, in _callmethod
    self._connect()
  File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/managers.py", line 742, in _connect
    conn = self._Client(self._token.address, authkey=self._authkey)
  File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/connection.py", line 169, in Client
    c = SocketClient(address)
  File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/connection.py", line 293, in SocketClient
    s.connect(address)
  File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
socket.error: [Errno 2] No such file or directory

（这是另一种使用multiprocessing.Lock 的方法，具有类似的效果...gist） 类似问题：

• Catch Keyboard Interrupt to stop Python multiprocessing worker from working on queue
• Keyboard Interrupts with python's multiprocessing Pool
• Shared variable in python's multiprocessing

Answer 1

multiprocessing.Manager() 启动一个负责处理您的共享列表代理的子进程。

运行时的netstat输出：

unix 2 [ ACC ] 流式监听 3921657 8457/python /tmp/pymp-B9dcij/listener-X423Ml

由 multiprocessing.Manager() 创建的这个子进程正在捕获您的 SIGINT 并退出，导致与它相关的任何内容都被取消引用，因此您的“没有此类文件”错误（我还收到了其他几个错误，具体取决于我何时决定发送 SIGINT )。

要解决这个问题，您可以直接声明一个 SyncManager 对象（而不是让 Manager() 为您做这件事）。这将要求您使用 start() 方法来实际启动子进程。 start() 方法将初始化函数作为其第一个参数（您可以在此处为管理器覆盖 SIGINT）。

下面的代码，试试这个：

from multiprocessing import Process, Manager
from multiprocessing.managers import BaseManager, SyncManager
from time import sleep
import signal

#handle SIGINT from SyncManager object
def mgr_sig_handler(signal, frame):
    print 'not closing the mgr'

#initilizer for SyncManager
def mgr_init():
    signal.signal(signal.SIGINT, mgr_sig_handler)
    #signal.signal(signal.SIGINT, signal.SIG_IGN) # <- OR do this to just ignore the signal
    print 'initialized mananger'

def f(process_number, shared_array):
    try:
        print "starting thread: ", process_number
        shared_array.append(process_number)
        sleep(3)
        shared_array.append(process_number)
    except KeyboardInterrupt:
        print "Keyboard interrupt in process: ", process_number
    finally:
        print "cleaning up thread", process_number

if __name__ == '__main__':

    processes = []

     #using syncmanager directly instead of letting Manager() do it for me
    manager = SyncManager()
    manager.start(mgr_init)  #fire up the child manager process
    shared_array = manager.list()

    for i in xrange(4):
        p = Process(target=f, args=(i, shared_array))
        p.start()
        processes.append(p)

    try:
        for process in processes:
            process.join()
    except KeyboardInterrupt:
        print "Keyboard interrupt in main"

    for item in shared_array:
        print item

Answer 2

正如我对类似问题的回答 (duplicate)：

最简单的解决方案 - 启动管理器

manager.start(signal.signal, (signal.SIGINT, signal.SIG_IGN))

而不是 manager.start()。 并检查信号模块是否在您的导入中（导入信号）。

这会在管理器进程中捕获并忽略 SIGINT (Ctrl-C)。