将 KITTI velodyne 投影到图像会产生一条窄条

Question

我正在尝试将KITTI velodyne 投影到左侧摄像头图像上。我按照 KITTI devkit 中的 README 进行操作，但结果是关闭的——这些点被投影为图像顶部的窄带。乐队看起来有一些分布，所以我怀疑我对校准矩阵做错了什么。或者也许在PIL.ImageDraw.point？

我使用的投影方程是根据 KITTI devkit 文档：

• x = P2 * R0_rect * Tr_velo_to_cam * y，在哪里
  • y 是一个4xN 矩阵，N 点采用XYZL 格式（L 是发光），
  • Tr_velo_to_cam 是 3x4 velodyne 到相机的转换矩阵
  • R0_rect 是 3x3 外部相机旋转矩阵
  • P2 是 3x3 内在相机投影矩阵

下面是代码，它的STDIO，以及生成的图像。

test.py:

import numpy as np
import os
from PIL import Image, ImageDraw

DATASET_PATH = "<DATASET PATH HERE>"

vld_path = os.path.join(DATASET_PATH, "velodyne/{:06d}.bin")
img_path = os.path.join(DATASET_PATH, "image_2/{:06d}.png")
clb_path = os.path.join(DATASET_PATH, "calib/{:06d}.txt")

frame_num = 58

# Load files
img = Image.open(img_path.format(frame_num))
clb = {}
with open(clb_path.format(frame_num), 'r') as clb_f:
  for line in clb_f:
    calib_line = line.split(':')
    if len(calib_line) < 2:
      continue
    key = calib_line[0]
    value = np.array(list(map(float, calib_line[1].split())))
    value = value.reshape((3, -1))
    clb[key] = value
vld = np.fromfile(vld_path.format(frame_num), dtype=np.float32)
vld = vld.reshape((-1, 4)).T

print("img.shape:", np.shape(img))
print("P2.shape:", clb['P2'].shape)
print("R0_rect.shape:", clb['R0_rect'].shape)
print("Tr_velo_to_cam.shape:", clb['Tr_velo_to_cam'].shape)
print("vld.shape:", vld.shape)

# Reshape calibration files
P2 = clb['P2']
R0 = np.eye(4)
R0[:-1, :-1] = clb['R0_rect']
Tr = np.eye(4)
Tr[:-1, :] = clb['Tr_velo_to_cam']

# Prepare 3d points
pts3d = vld[:, vld[-1, :] > 0].copy()
pts3d[-1, :] = 1

# Project 3d points
pts3d_cam = R0 @ Tr @ pts3d
mask = pts3d_cam[2, :] >= 0  # Z >= 0
pts2d_cam = P2 @ pts3d_cam[:, mask]
pts2d = (pts2d_cam / pts2d_cam[2, :])[:-1, :]

print("pts2d.shape:", pts2d.shape)

# Draw the points
img_draw = ImageDraw.Draw(img)
img_draw.point(pts2d, fill=(255, 0, 0))
img.show()

标准输出：

$> python ./test.py 
img.shape: (370, 1224, 3)
P2.shape: (3, 4)
R0_rect.shape: (3, 3)
Tr_velo_to_cam.shape: (3, 4)
vld.shape: (4, 115052)
pts2d.shape: (2, 53119)

制作图片：

Answer 1

发现问题：注意pts2d的维度是(2, N)，也就是说总共有N个点。但是，ImageDraw 例程期望它是 Nx2 或 1x2N 行向量，具有交替的 x 和 y 值。虽然我无法让point 例程与Nx2 输入一起工作，但我将它放在for 循环中（在转置点之后），并且它起作用了。

# ...
pts2d = (pts2d_cam / pts2d_cam[2, :])[:-1, :].T

print("pts2d.shape:", pts2d.shape)

# Draw the points
img_draw = ImageDraw.Draw(img)
for point in pts2d:
  img_draw.point(point, fill=(255, 0, 0))
# ...