TypeError：列表索引必须是整数或切片，而不是 cupy.core.core.ndarray

Question

在对象检测算法中，非最大抑制（NMS）用于丢弃对象的额外检测结果，例如车辆。

通常，水平边界框用于对象检测算法，水平 NMS 的 GPU 实现已经存在，但我希望 GPU 实现旋转边界框。

CPU 实现已经完成，但我正在努力使用 CuPy 包将 CPU 版本转换为 GPU 版本。这是我写的代码。在代码部分之后，您可以看到错误。

我的问题是 TypeError 的原因是什么：列表索引必须是整数或切片，而不是 cupy.core.core.ndarray？

    from shapely.geometry import Polygon as shpoly
    import time
    
    #### CPU implementation
    import numpy as np   
    
    def polygon_iou(poly1, poly2):
      """
      Intersection over union between two shapely polygons.
      """
      if not poly1.intersects(poly2): # this test is fast and can accelerate calculation
        iou = 0
      else:
        try:
          inter_area = poly1.intersection(poly2).area
          union_area = poly1.area + poly2.area - inter_area
          iou = float(inter_area) / float(union_area)
        except shapely.geos.TopologicalError:
          warnings.warn("'shapely.geos.TopologicalError occured, iou set to 0'", UserWarning)
          iou = 0
        except ZeroDivisionError:
          iou = 0
      return iou
    
    def polygon_from_array(poly_):
      """
      Create a shapely polygon object from gt or dt line.
      """
      polygon_points = np.array(poly_).reshape(4, 2)
      polygon = shpoly(polygon_points).convex_hull
      return polygon
    
    def nms(dets, thresh):
        scores = dets[:, 8]
        order = scores.argsort()[::-1]
        polys = []
        areas = []
        for i in range(len(dets)):
            tm_polygon = polygon_from_array(dets[i,:8])
            polys.append(tm_polygon)
        keep = []
        while order.size > 0:
            ovr = []
            i = order[0]
            keep.append(i)
            for j in range(order.size - 1):
                iou = polygon_iou(polys[i], polys[order[j + 1]])
                ovr.append(iou)
            ovr = np.array(ovr)
            inds = np.where(ovr <= thresh)[0]
            order = order[inds + 1]
        return keep
    
    
    #### GPU implementation
    import cupy as cp  
      
    def polygon_iou_gpu(poly1, poly2):
      """
      Intersection over union between two shapely polygons.
      """
      if not poly1.intersects(poly2): # this test is fast and can accelerate calculation
        iou = 0
      else:
        try:
          inter_area = poly1.intersection(poly2).area
          union_area = poly1.area + poly2.area - inter_area
          iou = float(inter_area) / float(union_area)
        except shapely.geos.TopologicalError:
          warnings.warn("'shapely.geos.TopologicalError occured, iou set to 0'", UserWarning)
          iou = 0
        except ZeroDivisionError:
          iou = 0
      return iou
    
    def polygon_from_array_gpu(poly_):
      """
      Create a shapely polygon object from gt or dt line.
      """
      polygon_points = cp.array(poly_).reshape(4, 2)
      polygon = shpoly(polygon_points).convex_hull
      return polygon
    
    def nms_gpu(dets, thresh):
        scores = dets[:, 8]
        order = scores.argsort()[::-1]
        polys = []
        areas = []
        for i in range(len(dets)):
            tm_polygon = polygon_from_array_gpu(dets[i,:8])
            polys.append(tm_polygon)
        keep = []
        while order.size > 0:
            ovr = []
            i = order[0]
            keep.append(i)
            for j in range(order.size - 1):   
                iou = polygon_iou_gpu(polys[i], polys[order[j + 1]])
                ovr.append(iou)
            ovr = np.array(ovr)
            inds = np.where(ovr <= thresh)[0]
            order = order[inds + 1]
        return keep
    
    
    if __name__ == '__main__':
        import random
        boxes = np.random.randint(0,100,(1000,8))
        scores = np.random.rand(1000, 1)
        dets = np.hstack((boxes, scores[:])).astype(np.float32)

    
        thresh = 0.1
        start = time.time()
        keep = nms(dets, thresh)
        print("CPU implementation took: {}".format(time.time() - start))
    
        cp.cuda.Device(1)
        dets_gpu = cp.array(dets)
        start = time.time()
        keep = nms_gpu(dets_gpu, thresh)
        print("GPU implementation took: {}".format(time.time() - start))

错误是

CPU 实现耗时：0.3672311305999756

Traceback（最近一次调用最后一次）：

文件“nms_rotated.py”，第 117 行，在

keep = nms_gpu(dets_gpu, thresh)

文件“nms_rotated.py”，第 97 行，在 nms_gpu 中

iou = polygon_iou_gpu(polys[i], polys[order[j + 1]])

TypeError: 列表索引必须是整数或切片，而不是 cupy.core.core.ndarray

更新：2019 年 2 月 13 日 我试过@Yuki Hashimoto的回答

将iou = polygon_iou_gpu(polys[i], polys[order[j + 1]]) 替换为 iou = polygon_iou_gpu(polys[i.get()], polys[order[j + 1].get()])。它不会抛出任何错误，但 GPU 版本比 CPU 版本慢几倍。

通过使用 100000 次随机检测：

      CPU implementation took: 47.125494956970215
      GPU implementation took: 142.08464860916138

Answer 1

简而言之：使用PFN官方non-maximum suppression。

详情： 使用cp.where，它返回一个匹配某些条件的list 对象。

---

不推荐corochann的答案，因为polys是一个列表，list也不应该被np.ndarray切片。 （并且不建议注入另一个依赖项......）

>>> polys[order.get()]  # get method returns np.ndarray
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: only integer scalar arrays can be converted to a scalar index
>>> polys[order[j + 1].get()]
### some result in some case, but this may fails depending on your env.###

Answer 2

[2019 年 2 月 13 日更新]

请参考@yuki-hashimoto的回答，比较合适。

---

如错误信息中所写

TypeError: 列表索引必须是整数或切片，而不是 cupy.core.core.ndarray

我猜order 是cupy 数组？ 在这种情况下，polys[order[j + 1]] 使用索引order[j+1] 作为可能导致问题的cupy 数组。 试试用cuda.to_cpu(array)方法把它们转成numpy数组怎么样？

from chainer import cuda
iou = polygon_iou_gpu(polys[i], polys[cuda.to_cpu(order[j + 1])])