如何按一个因子缩放轮廓的高度？

Question

我正在尝试使用 OpenCV 使用手机的摄像头扫描护照页面。

在上图中，红色标记的轮廓是我的 ROI（需要俯视图）。执行分割我可以detect the MRZ area。并且页面应该有一个固定的纵横比。有没有办法使用纵横比来缩放绿色轮廓以接近红色轮廓？我尝试使用approxPolyDP 找到绿色矩形的角，然后缩放该矩形，最后进行透视扭曲以获得顶视图。问题是在进行矩形缩放时没有考虑透视旋转，因此最终的矩形通常是错误的。

我经常得到如下图所示的输出

更新：添加更多解释

关于第一张图片（假设红色矩形始终具有恒定的纵横比），

• 我的目标是：裁剪掉红色标记的部分，然后得到一个俯视图
• 我的方法：检测 MRZ/绿色矩形 -> 现在假设绿色矩形的底部边缘与红色矩形相同（足够接近）-> 所以我得到了矩形的宽度和两个角 -> 计算其他两个角使用高度/纵横比
• 问题：我上面的计算没有输出红色矩形，而是在第二张图像中输出绿色矩形（可能是因为那些四边形不是矩形，边缘之间的角度不是 0 度或 90 度）

Answer 1

据我了解，您的主要目标是在从任意角度拍摄照片时获得护照页面的顶视图。 另外据我了解，您的方法如下：

1. 查找机读区及其环绕多边形
2. 将 MRZ 多边形扩展到顶部 - 这将为您提供页面多边形
3. 扭曲透视以获得顶视图。

目前的主要障碍是扩展多边形。

如果对目标的理解有误，请纠正我。

从数学角度来看，扩展多边形非常容易。多边形每一边的点形成一条边线。如果你把线画得更远，你可以在那里放一个新点。以编程方式它可能看起来像这样

new_left_top_x = old_left_bottom_x + (old_left_top_x - old_left_bottom_x) * pass_height_to_MRZ_height_ratio
new_left_top_y = old_left_bottom_y + (old_left_top_y - old_left_bottom_y) * pass_height_to_MRZ_height_ratio

右侧部分也可以这样做。这种方法也适用于高达 45 度的旋转。

但是，我担心这种方法不会给出准确的结果。我建议检测护照页面本身而不是 MRZ。原因是页面本身是照片上安静的醒目对象，可以通过findContours函数轻松找到。

我写了一些代码来说明检测机读区并不是真正必要的想法。

import os
import imutils
import numpy as np
import argparse
import cv2


# Thresholds
passport_page_aspect_ratio = 1.44
passport_page_coverage_ratio_threshold = 0.6
morph_size = (4, 4)


def pre_process_image(image):
    # Let's get rid of color first
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

    # Then apply Otsu threshold to reveal important areas
    ret, thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)

    # erode white areas to "disconnect" them
    # and dilate back to restore their original shape
    morph_struct = cv2.getStructuringElement(cv2.MORPH_RECT, morph_size)
    thresh = cv2.erode(thresh, morph_struct, anchor=(-1, -1), iterations=1)
    thresh = cv2.dilate(thresh, morph_struct, anchor=(-1, -1), iterations=1)

    return thresh


def find_passport_page_polygon(image):
    cnts = cv2.findContours(image, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    cnts = imutils.grab_contours(cnts)
    cnts = sorted(cnts, key=cv2.contourArea, reverse=True)

    for cnt in cnts:
        # compute the aspect ratio and coverage ratio of the bounding box
        # width to the width of the image
        (x, y, w, h) = cv2.boundingRect(cnt)
        ar = w / float(h)
        cr_width = w / float(image.shape[1])

        # check to see if the aspect ratio and coverage width are within thresholds
        if ar > passport_page_aspect_ratio and cr_width > passport_page_coverage_ratio_threshold:
            # approximate the contour with a polygon with 4 points
            epsilon = 0.02 * cv2.arcLength(cnt, True)
            approx = cv2.approxPolyDP(cnt, epsilon, True)
            return approx

    return None


def order_points(pts):
    # initialize a list of coordinates that will be ordered in the order:
    # top-left, top-right, bottom-right, bottom-left
    rect = np.zeros((4, 2), dtype="float32")
    pts = pts.reshape(4, 2)

    # the top-left point will have the smallest sum, whereas
    # the bottom-right point will have the largest sum
    s = pts.sum(axis=1)
    rect[0] = pts[np.argmin(s)]
    rect[2] = pts[np.argmax(s)]

    # now, compute the difference between the points, the
    # top-right point will have the smallest difference,
    # whereas the bottom-left will have the largest difference
    diff = np.diff(pts, axis=1)
    rect[1] = pts[np.argmin(diff)]
    rect[3] = pts[np.argmax(diff)]

    return rect


def get_passport_top_vew(image, pts):
    rect = order_points(pts)
    (tl, tr, br, bl) = rect

    # compute the height of the new image, which will be the
    # maximum distance between the top-right and bottom-right
    # y-coordinates or the top-left and bottom-left y-coordinates
    height_a = np.sqrt(((tr[0] - br[0]) ** 2) + ((tr[1] - br[1]) ** 2))
    height_b = np.sqrt(((tl[0] - bl[0]) ** 2) + ((tl[1] - bl[1]) ** 2))
    max_height = max(int(height_a), int(height_b))

    # compute the width using standard passport page aspect ratio
    max_width = int(max_height * passport_page_aspect_ratio)

    # construct the set of destination points to obtain the top view, specifying points
    # in the top-left, top-right, bottom-right, and bottom-left order
    dst = np.array([
        [0, 0],
        [max_width - 1, 0],
        [max_width - 1, max_height - 1],
        [0, max_height - 1]], dtype="float32")

    # compute the perspective transform matrix and apply it
    M = cv2.getPerspectiveTransform(rect, dst)
    warped = cv2.warpPerspective(image, M, (max_width, max_height))

    return warped


if __name__ == "__main__":
    ap = argparse.ArgumentParser()
    ap.add_argument("-i", "--image", required=True, help="path to images directory")
    args = vars(ap.parse_args())

    in_file = args["image"]
    filename_base = in_file.replace(os.path.splitext(in_file)[1], "")

    img = cv2.imread(in_file)

    pre_processed = pre_process_image(img)

    # Visualizing pre-processed image
    cv2.imwrite(filename_base + ".pre.png", pre_processed)

    page_polygon = find_passport_page_polygon(pre_processed)

    if page_polygon is not None:
        # Visualizing found page polygon
        vis = img.copy()
        cv2.polylines(vis, [page_polygon], True, (0, 255, 0), 2)
        cv2.imwrite(filename_base + ".bounds.png", vis)

        # Visualizing the warped top view of the passport page
        top_view_page = get_passport_top_vew(img, page_polygon)
        cv2.imwrite(filename_base + ".top.png", top_view_page)

我得到的结果：

为了获得更好的效果，补偿相机光圈失真也很好。