Python multiprocessing+savefig 导致错误或系统死机

Question

我有一个大的 3d numpy 数组，我想写出一个类似 imshow 的图形（即值的热图）的每个切片（2d 数组）。作为一个具体的例子，假设数组的形状为 3x3x3000，所以我想要 3000 张图像，每张图像代表一个 3x3 矩阵。用单个线程循环它有点慢。由于迭代是完全独立的，我想使用多处理模块来加快速度。代码如下。

def write_tensor_image(t_slice_wrapper):

    idx = t_slice_wrapper['idx']
    t_slice = t_slice_wrapper['t_slice']
    folder_path=t_slice_wrapper['folder_path']

    fig = matplotlib.pyplot.figure()
    ax = fig.add_subplot(111)    
    ax.imshow(t_slice,interpolation='none')
    fig.tight_layout()

    fname_ = os.path.join(folder_path,'tmp_%s.png'%str(idx))
    fig.savefig(fname_, bbox_inches="tight")    

def write_tensor_image_sequence(tensor, folder_path='/home/foo/numpy_cache'):

    os.system('mkdir -p %s'%folder_path)
    os.system('rm -rf %s/*'%folder_path)

    slices = [None]*tensor.shape[2]
    for i in range(0,tensor.shape[2]):
        slices[i] = {'t_slice':tensor[:,:,i], 'idx':i, 'folder_path':folder_path}

    pool = multiprocessing.Pool(processes=4)
    pool.map(write_tensor_image, slices)
    pool.close()
    pool.join()

但是这不起作用 - 单线程情况可以正常工作（只是在 for 循环中调用 write_tensor_image()），但使用池会导致机器完全锁定或出现类似以下错误：

XIO:  fatal IO error 11 (Resource temporarily unavailable) on X server ":0"
  after 849 requests (849 known processed) with 28 events remaining.
XIO:  fatal IO error 11 (Resource temporarily unavailable) on X server ":0"
XIO:  fatal IO error 11 (Resource temporarily unavailable) on X server ":0"
  after 849 requests (849 known processed) with 28 events remaining.
  after 849 requests (849 known processed) with 28 events remaining.
X Error of failed request:  BadPixmap (invalid Pixmap parameter)
Major opcode of failed request:  54 (X_FreePixmap)
Resource id in failed request:  0x4e0001e
Serial number of failed request:  851
Current serial number in output stream:  851

我认为我在正确的轨道上（根据例如 How to fix the python multiprocessing matplotlib savefig() issue? 和 Matplotlib: simultaneous plotting in multiple threads），但我一定错过了一些东西。

Answer 1

在脚本的开头写一个matplotlib.use('agg')。 Matplotlib 似乎试图在每个子进程中建立一个 GUI，这些子进程相互冲突。

更一般地说，您可能不想使用 pyplot 接口，而是使用 OOP 接口，以防您不进行标准交互式绘图。