Numba：双重释放或损坏（！prev）中止（核心转储）答案

【问题标题】：Numba: double free or corruption (!prev) Aborted (core dumped)Numba：双重释放或损坏（！prev）中止（核心转储）
【发布时间】：2020-02-13 12:16:40
【问题描述】：

我正在尝试使用 numba 加速以下功能。

import numpy as np
from numba import jit, prange

@jit(nopython=True, parallel=True)
def find_reg_numba(states):
    reg = []    
    states_sum = np.sum(states, axis=1)
    for i in prange(states.shape[0]):
        if states_sum[i] > 0 and states_sum[i] < 5:
            reg.append(states[i])
    return reg

states 是使用以下函数生成的

def generate_states(size):
    # size is a natural number
    states = np.array(list(map(list, itertools.product([0., 1.], repeat = size))))
    return states

当我尝试使用find_reg 函数时，我得到以下错误跟踪。

double free or corruption (!prev)
Aborted (core dumped)

我的 numba 版本是0.48.0。

如何解决这个问题？

【问题讨论】：

你有reg = []; reg = []。你想要：reg = []; reg_size = []。 find_reg_numba 也不返回任何东西。你想要：return reg, reg_size？
我已经编辑了我的代码，我只想要reg，因为reg_size可以稍后计算。
奇怪的是，大多数代码运行都可以，但偶尔我会得到：错误double free or corruption (!prev) repl process died unexpectedly: signal: aborted (core dumped)。这在非 Numba 版本中不会发生（即不使用装饰器）。
我得到double free or corruption (!prev)Segmentation fault (core dumped) 的所有运行。我的 numba 版本是 0.48.0。

标签： python numpy numba

【解决方案1】：

不确定您的代码为何会产生错误。相关错误帖有：

然而，事实证明这些都无济于事。

这是 find_reg_numba 的另一种 Numba 版本：

运行没有错误
产生与没有 Numba 的原始代码相同的结果（即原始代码仅在使用 Numba 时产生错误）。

代码重构

import numpy as np
from numba import jit
import itertools

@jit(nopython=True, parallel=True)
def find_reg_numba(states):
    states_sum = np.sum(states, axis=1)

    # Find indexes satisfying condition using np.where as described https://www.geeksforgeeks.org/numpy-where-in-python/
    indexes = np.where((states_sum > 0) & (states_sum < 5))
    return states[indexes]

def generate_states(size):
    # size is a natural number
    states = np.array(list(map(list, itertools.product([0., 1.], repeat = size))))
    return states

测试

for size in range(10):
  s = generate_states(size)
  r  = find_reg_numba(s)
  print(f'Size: {size}\n Result: \n{r}')

结果

 Size: 0
 Result:
[]
Size: 1
 Result:
[[1.]]
Size: 2
 Result:
[[0. 1.]
 [1. 0.]
 [1. 1.]]
Size: 3
 Result:
[[0. 0. 1.]
 [0. 1. 0.]
 [0. 1. 1.]
 [1. 0. 0.]
 [1. 0. 1.]
 [1. 1. 0.]
 [1. 1. 1.]]
Size: 4
 Result:
[[0. 0. 0. 1.]
 [0. 0. 1. 0.]
 [0. 0. 1. 1.]
 [0. 1. 0. 0.]
 [0. 1. 0. 1.]
 [0. 1. 1. 0.]
 [0. 1. 1. 1.]
 [1. 0. 0. 0.]
 [1. 0. 0. 1.]
 [1. 0. 1. 0.]
 [1. 0. 1. 1.]
 [1. 1. 0. 0.]
 [1. 1. 0. 1.]
 [1. 1. 1. 0.]
 [1. 1. 1. 1.]]
Size: 5
 Result:
[[0. 0. 0. 0. 1.]
 [0. 0. 0. 1. 0.]
 [0. 0. 0. 1. 1.]
 [0. 0. 1. 0. 0.]
 [0. 0. 1. 0. 1.]
 [0. 0. 1. 1. 0.]
 [0. 0. 1. 1. 1.]
 [0. 1. 0. 0. 0.]
 [0. 1. 0. 0. 1.]
 [0. 1. 0. 1. 0.]
 [0. 1. 0. 1. 1.]
 [0. 1. 1. 0. 0.]
 [0. 1. 1. 0. 1.]
 [0. 1. 1. 1. 0.]
 [0. 1. 1. 1. 1.]
 [1. 0. 0. 0. 0.]
 [1. 0. 0. 0. 1.]
 [1. 0. 0. 1. 0.]
 [1. 0. 0. 1. 1.]
 [1. 0. 1. 0. 0.]
 [1. 0. 1. 0. 1.]
 [1. 0. 1. 1. 0.]
 [1. 0. 1. 1. 1.]
 [1. 1. 0. 0. 0.]
 [1. 1. 0. 0. 1.]
 [1. 1. 0. 1. 0.]
 [1. 1. 0. 1. 1.]
 [1. 1. 1. 0. 0.]
 [1. 1. 1. 0. 1.]
 [1. 1. 1. 1. 0.]]
Size: 6
 Result:
[[0. 0. 0. 0. 0. 1.]
 [0. 0. 0. 0. 1. 0.]
 [0. 0. 0. 0. 1. 1.]
 [0. 0. 0. 1. 0. 0.]
 [0. 0. 0. 1. 0. 1.]
 [0. 0. 0. 1. 1. 0.]
 [0. 0. 0. 1. 1. 1.]
 [0. 0. 1. 0. 0. 0.]
 [0. 0. 1. 0. 0. 1.]
 [0. 0. 1. 0. 1. 0.]
 [0. 0. 1. 0. 1. 1.]
 [0. 0. 1. 1. 0. 0.]
 [0. 0. 1. 1. 0. 1.]
 [0. 0. 1. 1. 1. 0.]
 [0. 0. 1. 1. 1. 1.]
 [0. 1. 0. 0. 0. 0.]
 [0. 1. 0. 0. 0. 1.]
 [0. 1. 0. 0. 1. 0.]
 [0. 1. 0. 0. 1. 1.]
 [0. 1. 0. 1. 0. 0.]
 [0. 1. 0. 1. 0. 1.]
 [0. 1. 0. 1. 1. 0.]
 [0. 1. 0. 1. 1. 1.]
 [0. 1. 1. 0. 0. 0.]
 [0. 1. 1. 0. 0. 1.]
 [0. 1. 1. 0. 1. 0.]
 [0. 1. 1. 0. 1. 1.]
 [0. 1. 1. 1. 0. 0.]
 [0. 1. 1. 1. 0. 1.]
 [0. 1. 1. 1. 1. 0.]
 [1. 0. 0. 0. 0. 0.]
 [1. 0. 0. 0. 0. 1.]
 [1. 0. 0. 0. 1. 0.]
 [1. 0. 0. 0. 1. 1.]
 [1. 0. 0. 1. 0. 0.]
 [1. 0. 0. 1. 0. 1.]
 [1. 0. 0. 1. 1. 0.]
 [1. 0. 0. 1. 1. 1.]
 [1. 0. 1. 0. 0. 0.]
 [1. 0. 1. 0. 0. 1.]
 [1. 0. 1. 0. 1. 0.]
 [1. 0. 1. 0. 1. 1.]
 [1. 0. 1. 1. 0. 0.]
 [1. 0. 1. 1. 0. 1.]
 [1. 0. 1. 1. 1. 0.]
 [1. 1. 0. 0. 0. 0.]
 [1. 1. 0. 0. 0. 1.]
 [1. 1. 0. 0. 1. 0.]
 [1. 1. 0. 0. 1. 1.]
 [1. 1. 0. 1. 0. 0.]
 [1. 1. 0. 1. 0. 1.]
 [1. 1. 0. 1. 1. 0.]
 [1. 1. 1. 0. 0. 0.]
 [1. 1. 1. 0. 0. 1.]
 [1. 1. 1. 0. 1. 0.]
 [1. 1. 1. 1. 0. 0.]]
Size: 7
 Result:
[[0. 0. 0. 0. 0. 0. 1.]
 [0. 0. 0. 0. 0. 1. 0.]
 [0. 0. 0. 0. 0. 1. 1.]
 [0. 0. 0. 0. 1. 0. 0.]
 [0. 0. 0. 0. 1. 0. 1.]
 [0. 0. 0. 0. 1. 1. 0.]
 [0. 0. 0. 0. 1. 1. 1.]
 [0. 0. 0. 1. 0. 0. 0.]
 [0. 0. 0. 1. 0. 0. 1.]
 [0. 0. 0. 1. 0. 1. 0.]
 [0. 0. 0. 1. 0. 1. 1.]
 [0. 0. 0. 1. 1. 0. 0.]
 [0. 0. 0. 1. 1. 0. 1.]
 [0. 0. 0. 1. 1. 1. 0.]
 [0. 0. 0. 1. 1. 1. 1.]
 [0. 0. 1. 0. 0. 0. 0.]
 [0. 0. 1. 0. 0. 0. 1.]
 [0. 0. 1. 0. 0. 1. 0.]
 [0. 0. 1. 0. 0. 1. 1.]
 [0. 0. 1. 0. 1. 0. 0.]
 [0. 0. 1. 0. 1. 0. 1.]
 [0. 0. 1. 0. 1. 1. 0.]
 [0. 0. 1. 0. 1. 1. 1.]
 [0. 0. 1. 1. 0. 0. 0.]
 [0. 0. 1. 1. 0. 0. 1.]
 [0. 0. 1. 1. 0. 1. 0.]
 [0. 0. 1. 1. 0. 1. 1.]
 [0. 0. 1. 1. 1. 0. 0.]
 [0. 0. 1. 1. 1. 0. 1.]
 [0. 0. 1. 1. 1. 1. 0.]
 [0. 1. 0. 0. 0. 0. 0.]
 [0. 1. 0. 0. 0. 0. 1.]
 [0. 1. 0. 0. 0. 1. 0.]
 [0. 1. 0. 0. 0. 1. 1.]
 [0. 1. 0. 0. 1. 0. 0.]
 [0. 1. 0. 0. 1. 0. 1.]
 [0. 1. 0. 0. 1. 1. 0.]
 [0. 1. 0. 0. 1. 1. 1.]
 [0. 1. 0. 1. 0. 0. 0.]
 [0. 1. 0. 1. 0. 0. 1.]
 [0. 1. 0. 1. 0. 1. 0.]
 [0. 1. 0. 1. 0. 1. 1.]
 [0. 1. 0. 1. 1. 0. 0.]
 [0. 1. 0. 1. 1. 0. 1.]
 [0. 1. 0. 1. 1. 1. 0.]
 [0. 1. 1. 0. 0. 0. 0.]
 [0. 1. 1. 0. 0. 0. 1.]
 [0. 1. 1. 0. 0. 1. 0.]
 [0. 1. 1. 0. 0. 1. 1.]
 [0. 1. 1. 0. 1. 0. 0.]
 [0. 1. 1. 0. 1. 0. 1.]
 [0. 1. 1. 0. 1. 1. 0.]
 [0. 1. 1. 1. 0. 0. 0.]
 [0. 1. 1. 1. 0. 0. 1.]
 [0. 1. 1. 1. 0. 1. 0.]
 [0. 1. 1. 1. 1. 0. 0.]
 [1. 0. 0. 0. 0. 0. 0.]
 [1. 0. 0. 0. 0. 0. 1.]
 [1. 0. 0. 0. 0. 1. 0.]
 [1. 0. 0. 0. 0. 1. 1.]
 [1. 0. 0. 0. 1. 0. 0.]
 [1. 0. 0. 0. 1. 0. 1.]
 [1. 0. 0. 0. 1. 1. 0.]
 [1. 0. 0. 0. 1. 1. 1.]
 [1. 0. 0. 1. 0. 0. 0.]
 [1. 0. 0. 1. 0. 0. 1.]
 [1. 0. 0. 1. 0. 1. 0.]
 [1. 0. 0. 1. 0. 1. 1.]
 [1. 0. 0. 1. 1. 0. 0.]
 [1. 0. 0. 1. 1. 0. 1.]
 [1. 0. 0. 1. 1. 1. 0.]
 [1. 0. 1. 0. 0. 0. 0.]
 [1. 0. 1. 0. 0. 0. 1.]
 [1. 0. 1. 0. 0. 1. 0.]
 [1. 0. 1. 0. 0. 1. 1.]
 [1. 0. 1. 0. 1. 0. 0.]
 [1. 0. 1. 0. 1. 0. 1.]
 [1. 0. 1. 0. 1. 1. 0.]
 [1. 0. 1. 1. 0. 0. 0.]
 [1. 0. 1. 1. 0. 0. 1.]
 [1. 0. 1. 1. 0. 1. 0.]
 [1. 0. 1. 1. 1. 0. 0.]
 [1. 1. 0. 0. 0. 0. 0.]
 [1. 1. 0. 0. 0. 0. 1.]
 [1. 1. 0. 0. 0. 1. 0.]
 [1. 1. 0. 0. 0. 1. 1.]
 [1. 1. 0. 0. 1. 0. 0.]
 [1. 1. 0. 0. 1. 0. 1.]
 [1. 1. 0. 0. 1. 1. 0.]
 [1. 1. 0. 1. 0. 0. 0.]
 [1. 1. 0. 1. 0. 0. 1.]
 [1. 1. 0. 1. 0. 1. 0.]
 [1. 1. 0. 1. 1. 0. 0.]
 [1. 1. 1. 0. 0. 0. 0.]
 [1. 1. 1. 0. 0. 0. 1.]
 [1. 1. 1. 0. 0. 1. 0.]
 [1. 1. 1. 0. 1. 0. 0.]
 [1. 1. 1. 1. 0. 0. 0.]]
Size: 8
 Result:
[[0. 0. 0. ... 0. 0. 1.]
 [0. 0. 0. ... 0. 1. 0.]
 [0. 0. 0. ... 0. 1. 1.]
 ...
 [1. 1. 1. ... 1. 0. 0.]
 [1. 1. 1. ... 0. 0. 0.]
 [1. 1. 1. ... 0. 0. 0.]]
Size: 9
 Result:
[[0. 0. 0. ... 0. 0. 1.]
 [0. 0. 0. ... 0. 1. 0.]
 [0. 0. 0. ... 0. 1. 1.]
 ...
 [1. 1. 1. ... 0. 0. 0.]
 [1. 1. 1. ... 0. 0. 0.]
 [1. 1. 1. ... 0. 0. 0.]]

【讨论】：