Python 2.7 多处理 - 无法通过 Manager 字典传递套接字对象

Question

我有一个简单的程序，可以搜索本地网络中的开放端口，并将连接的套接字及其本地地址存储在字典中。现在，我使用 Manager 共享字典来存储这些条目，但它只接受简单对象而不接受套接字实例。代码如下：

from multiprocessing import Process, Manager
import socket

manager = Manager()
# Store connected sockets
sockets = manager.dict()


def ping_addr(addr=None, port=None, timeout=None):
    """
    Create a socket and try to establish a connection to a specific address. If a connection is established, append
    the socket to the sockets dictionary.
    :param addr: The address.
    :param port: The port number.
    :param timeout: How many seconds to wait until its decided that the connection has been refused.
    :return: None
    """
    global sockets
    # Setup the client socket
    csocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    csocket.settimeout(timeout)
    # Try to connect
    try:
        csocket.connect((addr, port))
        print 'connected to {}:{}'.format(addr, port)
        # This works
        sockets.update({addr: 0})
        # This doesnt work
        sockets.update({addr: csocket})
    except socket.error:
        pass


for i in range(256):
    proc = Process(target=ping_addr, kwargs={'addr': '192.168.1.{}'.format(i), 'port': 14540, 'timeout': 0.5})
    proc.start()

我得到的错误是：

Process Process-4:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/alex/PycharmProjects/proj/test.py", line 29, in ping_addr
    sockets.update({addr: csocket})
  File "<string>", line 2, in update
  File "/usr/lib/python2.7/multiprocessing/managers.py", line 758, in _callmethod
    conn.send((self._id, methodname, args, kwds))
TypeError: expected string or Unicode object, NoneType found

我对代理进行了简短的研究，但找不到将它们实现到我的代码中的方法，因此我寻求帮助。如何使套接字实例与 Manager 字典兼容？

Answer 1

将套接字从父进程传递给子进程是一种非常常见的做法。从python3.4开始，你可以直接通过multiprocessing.Manager共享一个socket对象，但是在python2.7中，你必须自己连接它（see full gist here）：

import socket
import copy_reg
from multiprocessing.reduction import rebuild_socket, reduce_socket

copy_reg.pickle(socket.socket, reduce_socket, rebuild_socket)

Answer 2

套接字不能通过 Manager.dict 传递给另一个进程。每个 python 套接字都有一个操作系统级别的套接字。操作系统知道套接字和进程之间的绑定，并维护套接字所有权。套接字只能在创建它的进程中使用。或者可以在类unix系统的子进程中使用。

另见Passing Python object to another Python process