【问题标题】:Python Multiprocessing: Execute code serially before and after parallel executionPython多处理:在并行执行之前和之后串行执行代码
【发布时间】:2018-08-05 23:37:56
【问题描述】:

这里的新手:我正在尝试串行执行一些代码,然后创建一个线程池并并行执行一些代码。并行执行完成后,我想串行执行更多代码。

例如...

import time
from multiprocessing import Pool

print("I only want to print this statement once")



def worker(i):
    """worker function"""
    now = time.time()
    time.sleep(i)
    then = time.time()
    print(now, then)

if __name__ == '__main__':
    with Pool(3) as p:
        p.map(worker, [1, 1, 1])
        p.close()

print("Only print this once as well")

我希望这个返回...

I only want to print this statement once
1533511478.0619314 1533511479.0620182
1533511478.0789354 1533511479.0791905
1533511478.0979397 1533511479.098235
Only print this once as well

然而它返回的是这样的:

I only want to print this statement once
I only want to print this statement once
Only print this once as well
I only want to print this statement once
Only print this once as well
I only want to print this statement once
Only print this once as well
I only want to print this statement once
Only print this once as well
I only want to print this statement once
Only print this once as well
1533511478.0619314 1533511479.0620182
1533511478.0789354 1533511479.0791905
1533511478.0979397 1533511479.098235
Only print this once as well

所以它似乎为每个池额外运行了一次打印语句。

任何帮助将不胜感激!

【问题讨论】:

    标签: python multiprocessing python-multiprocessing pool


    【解决方案1】:

    根据观察到的行为,我假设您使用的是 NT/Windows 操作系统。

    您看到所有这些打印的原因是因为在 Windows 上使用了spawn 启动策略。当“生成”一个新进程时,会启动一个新的 Python 解释器并接收模块和它应该执行的函数。当新解释器导入模块时,将执行顶级print 函数。因此重复打印。

    只需在__main__ 中移动这些打印语句,您就不会再看到它们了。

    【讨论】:

      猜你喜欢
      • 2021-10-05
      • 2016-12-25
      • 2013-05-31
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多