将纹理特征集成到基于颜色的视频帧对象检测

Question

我正在做一个项目，我必须从鸟瞰图检测彩色车辆。请参阅下面的图像框架

因为我认为在这种情况下，深度学习是多余的，所以我决定使用经典的检测管道，通过应用直方图反投影方法来检测红色车辆。

我使用的代码如下所示

#import packages
import cv2 
import numpy as np
import matplotlib.pyplot as plt

#defining disk for the convolution
disc = cv2.getStructuringElement(cv2.MORPH_RECT,(10,10))
#defining Kernel
kernel=cv2.getStructuringElement(cv2.MORPH_RECT, (3,3))
kernel1 = np.ones((5,5),np.uint8)



#preparing the object model by cropping the target object and reading it
Red_Model=cv2.imread(r'C:/Users/kjbaili/.spyder-py3/webcam_calib/red_new_uniform.png')

#convert object_model to hsv
hsv_red =  cv2.cvtColor(Red_Model, cv2.COLOR_BGR2HSV)
cv2.imshow('HSV_Model',hsv_red)

#calculating histogram of object_model
M = cv2.calcHist([hsv_red],[0,1],None,[180,256],[0,180,0,256])

#reading the image where we want to search for the target object
img=cv2.imread(r'C:/Users/kjbaili/.spyder-py3/Mean_Shift/MST_with_3D_projection/messigray1.png')

#converting the image to hsv
hsv_img =  cv2.cvtColor(img, cv2.COLOR_BGR2HSV)

#calculating histogram of image
I= cv2.calcHist([hsv_img],[0,1],None,[180,256],[0,180,0,256])

#calculating the ration histogram
R=M.copy() /I.copy()

#extracting the image channels
h,s,v = cv2.split(hsv_img.copy())


 
#backprojecting the R_red hist
B1 = R[h.ravel(),s.ravel()]

#reshaping the resulted vector into image and normalize the values between 0 and 1
B2=B1.reshape(hsv_img.copy().shape[:2])
B = np.minimum(B1.copy(),1)
B = B.reshape(hsv_img.copy().shape[:2])
 



#removing noises
open_B_red=cv2.morphologyEx(B.copy(),cv2.MORPH_OPEN,kernel)     
Filter_red=cv2.filter2D(open_B_red.copy(),-1,disc,open_B_red)

#displaying results
cv2.imshow('Filter',Filter_red)



cv2.waitKey(0)
cv2.destroyAllWindows()

该算法适用于图示（见结果蒙版图像）

但是，当对象的颜色与背景颜色相似时如下图_它也会检测到我的 T 恤，它会同时检测到这两种颜色，因此无法仅区分目标对象

因此，我想到了使用附加特征和颜色特征来增强检测，使其仍然检测到目标对象，尽管颜色相似度很小，结果如此称为“Local Binary Pattern-LBP”，它计算对象的纹理。事实上，我可以计算物体和图像的 LBP，但是不知道将它与上述方法集成，以便通过颜色纹理检测来检测红色车辆。

因此，我真的很想知道如何在这种情况下使用 LBP 的 T纹理特征甚至其他附加特征来改进检测。

有关 LBP 的信息，请参阅：https://towardsdatascience.com/face-recognition-how-lbph-works-90ec258c3d6b

有关直方图反投影如何工作的信息：https://theailearner.com/2019/04/18/histogram-backprojection/

这是原图： 这是结果面具 提前谢谢

Answer 1

具有少量控制参数的模板匹配（如（Size ..etc）可以适用于您的情况：

import cv2
import numpy as np
from matplotlib import pyplot as plt

img = cv2.imread('Path_To_\\target.jpg',0)
img2 = img.copy()
imgDisp = cv2.cvtColor(img2 , cv2.COLOR_GRAY2RGB)
template = cv2.imread('Path_t0\\template.jpg',0)
w, h = template.shape[::-1]

# All the 6 methods for comparison in a list
methods = ['cv2.TM_CCOEFF', 'cv2.TM_CCOEFF_NORMED', 'cv2.TM_CCORR',
        'cv2.TM_CCORR_NORMED', 'cv2.TM_SQDIFF', 'cv2.TM_SQDIFF_NORMED']

for meth in methods:
    img = img2.copy()
    method = eval(meth)

    # Apply template Matching
    res = cv2.matchTemplate(img,template,method)
    min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(res)

    # If the method is TM_SQDIFF or TM_SQDIFF_NORMED, take minimum
    if method in [cv2.TM_SQDIFF, cv2.TM_SQDIFF_NORMED]:
       top_left = min_loc
    else:
       top_left = max_loc
    bottom_right = (top_left[0] + w, top_left[1] + h)

    cv2.rectangle(imgDisp,top_left, bottom_right, 255, 2)

    plt.subplot(121),plt.imshow(res,cmap = 'gray')
    plt.title('Matching Result'), plt.xticks([]), plt.yticks([])
    plt.subplot(122),plt.imshow(imgDisp,cmap = 'gray')
    plt.title('Detected Point'), plt.xticks([]), plt.yticks([])
    plt.suptitle(meth)

    plt.show()

结果：

你可以罚款代码HERE

Answer 2

这是在 Python/OpenCV 中执行此操作的一种方法。颜色的第一个阈值。然后应用一些形态来清洁无关区域。然后获取轮廓并遍历每个轮廓。计算每个旋转的矩形及其面积和纵横比（最大尺寸/最小尺寸）。过滤两者范围内的轮廓，并在输入图像上绘制白色轮廓和白色旋转矩形。

输入：

import cv2
import numpy as np

image = cv2.imread("red_car.png")
hh, ww = image.shape[:2]

# convert to HSV
hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)

# create a binary thresholded image
lower = (80,100,180)
upper = (255,255,255)
thresh = cv2.inRange(hsv, lower, upper)


# apply morphology
kernel = np.ones((5,5), np.uint8)
clean = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel)

# get external contours
contours = cv2.findContours(clean, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]

area_thresh1 = 500
area_thresh2 = 1000
aspect_thresh1 = 2
aspect_thresh2 = 4
result1 = image.copy()
result2 = image.copy()
for c in contours:
    
    # get rotated rectangle from contour
    # get its dimensions
    # get angle relative to horizontal from rotated rectangle
    rotrect = cv2.minAreaRect(c)
    box = cv2.boxPoints(rotrect)
    box = np.int0(box)
    (center), (dim1,dim2), angle = rotrect
    maxdim = max(dim1,dim2)
    mindim = min(dim1,dim2)
    area = dim1 * dim2
    if area > 0:
        aspect = maxdim / mindim
        #print(area, aspect)

    if area > area_thresh1 and area < area_thresh2 and aspect > aspect_thresh1 and aspect < aspect_thresh2:
        # draw contour on input
        cv2.drawContours(result1,[c],0,(255,255,255),1)
        # draw rectangle on input
        cv2.drawContours(result2,[box],0,(255,255,255),1)
        print(area, aspect)

# save result
cv2.imwrite("red_car_thresh.png",thresh)
cv2.imwrite("red_car_clean.png",clean)
cv2.imwrite("red_car_thresh_result1.png",result1)
cv2.imwrite("red_car_thresh_result2.png",result2)

# display result
cv2.imshow("thresh", thresh)
cv2.imshow("clean", clean)
cv2.imshow("result1", result1)
cv2.imshow("result2", result2)
cv2.waitKey(0)
cv2.destroyAllWindows()

阈值图像：

形态清洁图像：

汽车轮廓：

汽车旋转边界框：