去歪斜图像中的单个字符

Question

我试图在一个程序中破坏一个反机器人功能，在该功能中，用户必须按字母数字顺序单击图像中的字母。我已经设法使用预处理从背景中提取了一些文本，但仍然需要使用 Tesseract 对每个单独的字符进行去偏斜以获得最佳结果。

使用霍夫线之前的图像，只是预处理

带有由 HoughLinesP 检测到的线条绘制的原始图像 我尝试使用 Canny Edge Detector + Hough Lines 来尝试找到每个字符下方的行。但是，它认为不一致并且过于依赖线条的质量，我无法区分底线和在字符本身上检测到的线条。

这是我尝试过的代码：

# -*- coding:utf-8 -*-
import cv2, numpy as np, time
img_roi = [48, 191, 980, 656]  # x1, y1, x2, y2
src_img_dir = "images/source/9.png"
bg_img = cv2.imread("images/background.png", cv2.IMREAD_COLOR)[img_roi[1]:img_roi[3], img_roi[0]:img_roi[2]]
# The background of the area is constant. So I have used a reference background image and removed pixels which have a similar H value as the background

bg_hsv = cv2.cvtColor(bg_img, cv2.COLOR_BGR2HSV)
src_img = cv2.imread(src_img_dir, cv2.IMREAD_COLOR)[img_roi[1]:img_roi[3], img_roi[0]:img_roi[2]]
# This image is the image where letters are placed on top of the background image

src_hsv = cv2.cvtColor(src_img, cv2.COLOR_BGR2HSV)
mask = np.zeros([src_img.shape[0], src_img.shape[1], 3], dtype=np.uint8)

offset = 3
start_time = time.time()
for y in range(src_img.shape[0]):
    for x in range(src_img.shape[1]):
        sp = src_hsv[y][x]
        bp = bg_hsv[y][x]

        if bp[0]-offset <= sp[0] <= bp[0]+offset:
            if sp[1] >= 109:
                mask[y][x] = src_img[y][x]
        elif sp[1] <= 90:
            if sp[0] >= 67:
                mask[y][x] = src_img[y][x]
            elif sp[2] >= 125 and sp[1] >= 20:
                mask[y][x] = src_img[y][x]
        else:
            mask[y][x] = src_img[y][x]
        """if sp[1] >= 60 and sp[2] >= 60:
            mask[y][x] = src_img[y][x]
            #mask[y][x] = conv"""

print("duration", time.time()-start_time)
mask = cv2.cvtColor(mask, cv2.COLOR_BGR2HSV)
#mask[:,:,2] = 255
mask = cv2.cvtColor(mask, cv2.COLOR_HSV2BGR)
mask_gray = cv2.cvtColor(mask, cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(mask_gray, 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)
opened = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, (3,3))
opened = cv2.morphologyEx(opened, cv2.MORPH_OPEN, (3,3))
opened = cv2.erode(opened, (3,3))
opened = cv2.dilate(opened, (3,3))
opened = cv2.dilate(opened, (5, 5))
opened = cv2.morphologyEx(opened, cv2.MORPH_CLOSE, (3,3))
opened = cv2.erode(opened, (3,3))
opened = cv2.erode(opened, (3,3))
final_img = opened
#edges = cv2.Canny(final_img, 0, 255)
lines = cv2.HoughLinesP(final_img, 1, np.pi / 180, 20, minLineLength=10, maxLineGap=3)
for line in lines:
        coords = line[0]
        cv2.line(src_img, (coords[0], coords[1]), (coords[2], coords[3]), [255,255,255], 2)
#cv2.imshow("can", edges)


#cv2.drawContours(src_img, fixed_contours, -1, (0,255,0), 2)
cv2.imshow("src", src_img)
cv2.imshow("", final_img)

cv2.waitKey(0)
cv2.destroyAllWindows()

Answer 1

乍一看，歪斜不是很强烈，而且人物之间的距离也很远。

我会对您过滤后的图像执行多步骤方法（已经很不错了）

• 首先检测包含两个非常大的斑点（字母/数字 + 下划线）的感兴趣区域，滤除噪声像素
• 然后将下划线明确检测为两者中的长平线（字母“I”和数字“1”在这方面可能存在问题）
• 使用与局部感兴趣区域相关的下划线方向（下划线 + 字符）来确定哪一侧朝下
• 启发式确定倾斜角度：假设 x 度（在 x 的狭窄范围内循环），有多少局部感兴趣区域的信号位于下划线上方的四边形内，使得底部（下划线）与左侧之间的角度为 x。李>
• 使用 image-unwarp 函数，以便将下划线映射到具有适当宽高比的矩形的底部边缘
• 利润