不能在python函数中使用多处理

Question

我是python的初学者，我正在尝试将多处理放入一个函数中，但是python给了我一个错误。

请参考原代码如下：

from multiprocessing import Process
import time
def func1():
    print('test1')
    time.sleep(10)  
def func2():
    print('test2')
    time.sleep(5)
if __name__ == '__main__':
    p_func1 = Process(target=func1)
    p_func2 = Process(target=func2)
    p_func1.start()
    p_func2.start()
    p_func1.join()
    p_func2.join()  
    print('done')

它运行良好并给出了我需要的正确结果。

但是，当我尝试将多处理代码放入函数时：

from multiprocessing import Process
import time

def test_multiprocessing():
    def func1():
        print('test1')
        time.sleep(10)  
    def func2():
        print('test2')
        time.sleep(5)
    if __name__ == '__main__':
        p_func1 = Process(target=func1)
        p_func2 = Process(target=func2)
        p_func1.start()
        p_func2.start()
        p_func1.join()
        p_func2.join()  
        print('done')

test_multiprocessing()

以下是我遇到的错误，我可以知道如何解决这个问题吗？我想将多处理放入一个功能中的原因是因为那里有一个现有的代码，我不想对代码进行重大更改以支持多处理。

Traceback (most recent call last):
  File "multipleprocessing.py", line 20, in <module>
    test_multiprocessing()
  File "multipleprocessing.py", line 14, in test_multiprocessing
    p_func1.start()
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\multiprocessing
\process.py", line 105, in start
    self._popen = self._Popen(self)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\multiprocessing
\context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\multiprocessing
\context.py", line 322, in _Popen
    return Popen(process_obj)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\multiprocessing
\popen_spawn_win32.py", line 65, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\multiprocessing
\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'test_multiprocessing.<locals>.func1'

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\multiprocessing
\spawn.py", line 99, in spawn_main
    new_handle = reduction.steal_handle(parent_pid, pipe_handle)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\multiprocessing
\reduction.py", line 87, in steal_handle
    _winapi.DUPLICATE_SAME_ACCESS | _winapi.DUPLICATE_CLOSE_SOURCE)
PermissionError: [WinError 5] Access is denied

根据 Linux 上的测试代码，它可以工作。这是否意味着 Windows Python 不能在函数中支持多处理？

Answer 1

您的代码是正确的。您不应该将if __name__ == '__main__': 保留在函数中。在此处阅读更多信息why name=="main"

尝试如下，

from multiprocessing import Process
import time

def test_multiprocessing():
    def func1():
        print('test1')
        time.sleep(10)  
    def func2():
        print('test2')
        time.sleep(5)

    p_func1 = Process(target=func1)
    p_func2 = Process(target=func2)
    p_func1.start()
    p_func2.start()
    p_func1.join()
    p_func2.join()  
    print('done')

test_multiprocessing()

Answer 2

@Prakash 答案中的一些更正。你需要从if __name__== "__main__"里面调用函数

Here，解释清楚！！

from multiprocessing import Process
import time


def func1():
    print('test1')
    time.sleep(10)

def func2():
    print('test2')
    time.sleep(5)

def test_multiprocessing():
    p_func1 = Process(target=func1)
    p_func2 = Process(target=func2)
    p_func1.start()
    p_func2.start()
    p_func1.join()
    p_func2.join()
    print('done')

if __name__== "__main__":
    test_multiprocessing()

另一种方法是您可以将方法绑定到一个类，因为functions are only picklable if they are defined at the top-level of a module. 如下：

from multiprocessing import Process
import time

class Foo:

    def func1(self):
        print('test1')
        time.sleep(10)

    def func2(self):
        print('test2')
        time.sleep(5)

    def test_multiprocessing(self):
        p_func1 = Process(target=self.func1)
        p_func2 = Process(target=self.func2)
        p_func1.start()
        p_func2.start()
        p_func1.join()
        p_func2.join()
        print('done')

if __name__== "__main__":
    f=Foo()
    f.test_multiprocessing()