OpenCV：如何将图像垂直拼接成长图像而不是全景图/水平

Question

我尝试创建一个可以垂直拼接 iPhone 屏幕截图的应用程序，如下图所示。它会忽略图像数组的重叠部分，并以正确的顺序缝合独特的部分。

我在互联网上查看了一些教程，我认为openCV 框架可以完成这项工作，因为它支持拼接图像全景/水平。所以我将这个 pod 集成到我的项目中，并使其适用于全景图像拼接。

但是我在垂直拼接图像时遇到了麻烦。从CVWrapper.mm 我可以看到processWithArray 这个方法会旋转并从图像数组创建新的拼接图像。目前，我在数组中的所有图像的方向都是up。

我的问题：是否可以改变这个方向，所以让openCV 从上到下而不是从左到右缝合图像？我有点迷失在那些方向上，现在不知道我应该从哪里开始。任何提示/建议将不胜感激。

CVWrapper.mm

+ (UIImage*) processWithArray:(NSArray*)imageArray
{
    if ([imageArray count]==0){
        NSLog (@"imageArray is empty");
        return 0;
        }
    std::vector<cv::Mat> matImages;

    for (id image in imageArray) {
        NSLog(@"image orientation: %ld", (long)[image imageOrientation]);
        
        if ([image isKindOfClass: [UIImage class]]) {
            /*
             All images taken with the iPhone/iPa cameras are LANDSCAPE LEFT orientation. The  UIImage imageOrientation flag is an instruction to the OS to transform the image during display only. When we feed images into openCV, they need to be the actual orientation that we expect them to be for stitching. So we rotate the actual pixel matrix here if required.
             */
            UIImage* rotatedImage = [image rotateToImageOrientation];
            cv::Mat matImage = [rotatedImage CVMat3];
            NSLog (@"matImage: %@",image);
            matImages.push_back(matImage);
        }
    }
    NSLog (@"stitching...");
    cv::Mat stitchedMat = stitch (matImages);
    UIImage* result =  [UIImage imageWithCVMat:stitchedMat];
    return result;
}

UIImage+Rotate.m

- (UIImage *)rotateToImageOrientation {

    // No-op if the orientation is already correct
    if (self.imageOrientation == UIImageOrientationUp) return self;
    
    // We need to calculate the proper transformation to make the image upright.
    // We do it in 2 steps: Rotate if Left/Right/Down, and then flip if Mirrored.
    CGAffineTransform transform = CGAffineTransformIdentity;
    
    switch (self.imageOrientation) {
        case UIImageOrientationDown:
        case UIImageOrientationDownMirrored:
            transform = CGAffineTransformTranslate(transform, self.size.width, self.size.height);
            transform = CGAffineTransformRotate(transform, M_PI);
            break;
            
        case UIImageOrientationLeft:
        case UIImageOrientationLeftMirrored:
            transform = CGAffineTransformTranslate(transform, self.size.width, 0);
            transform = CGAffineTransformRotate(transform, M_PI_2);
            break;
            
        case UIImageOrientationRight:
        case UIImageOrientationRightMirrored:
            transform = CGAffineTransformTranslate(transform, 0, self.size.height);
            transform = CGAffineTransformRotate(transform, -M_PI_2);
            break;
        case UIImageOrientationUp:
        case UIImageOrientationUpMirrored:
            break;
    }
    
    switch (self.imageOrientation) {
        case UIImageOrientationUpMirrored:
        case UIImageOrientationDownMirrored:
            transform = CGAffineTransformTranslate(transform, self.size.width, 0);
            transform = CGAffineTransformScale(transform, -1, 1);
            break;
            
        case UIImageOrientationLeftMirrored:
        case UIImageOrientationRightMirrored:
            transform = CGAffineTransformTranslate(transform, self.size.height, 0);
            transform = CGAffineTransformScale(transform, -1, 1);
            break;
        case UIImageOrientationUp:
        case UIImageOrientationDown:
        case UIImageOrientationLeft:
        case UIImageOrientationRight:
            break;
    }
    
    // Now we draw the underlying CGImage into a new context, applying the transform
    // calculated above.
    CGContextRef ctx = CGBitmapContextCreate(NULL, self.size.width, self.size.height,
                                             CGImageGetBitsPerComponent(self.CGImage), 0,
                                             CGImageGetColorSpace(self.CGImage),
                                             CGImageGetBitmapInfo(self.CGImage));
    CGContextConcatCTM(ctx, transform);
    switch (self.imageOrientation) {
        case UIImageOrientationLeft:
        case UIImageOrientationLeftMirrored:
        case UIImageOrientationRight:
        case UIImageOrientationRightMirrored:
            // Grr...
            CGContextDrawImage(ctx, CGRectMake(0,0,self.size.height,self.size.width), self.CGImage);
            break;
            
        default:
            CGContextDrawImage(ctx, CGRectMake(0,0,self.size.width,self.size.height), self.CGImage);
            break;
    }
    
    // And now we just create a new UIImage from the drawing context
    CGImageRef cgimg = CGBitmapContextCreateImage(ctx);
    UIImage *img = [UIImage imageWithCGImage:cgimg];
    CGContextRelease(ctx);
    CGImageRelease(cgimg);
    return img;
}

视图控制器

    @objc func stitchButtonTapped(sender: UIButton) {
        for imageView in [self.imageView1, self.imageView2, self.imageView3] {
            imageView.isHidden = true
        }

        DispatchQueue.global().async { [weak self] in
            guard let images = self?.images else { return }
            let stitchedImage = CVWrapper.process(with: images)

            DispatchQueue.main.async {
                guard let imageView = self?.longImageView else { return }
                UIView.transition(with: imageView,
                                  duration: 1.6,
                                  options: .transitionCrossDissolve,
                                  animations: {
                                    self?.longImageView.image = stitchedImage
                                    self?.longImageView.isHidden = false
                                  },
                                  completion: nil)
            }
        }
    }

原3截图

拼接新的长图

Answer 1

好吧，我已经解决了这个问题。简而言之，在处理它们之前，我将每个图像逆时针旋转 90 度。所以我从不在UIImage+Rotate.m中使用方法rotateToImageOrientation。

现在，这个拼接器正在处理大多数应用内屏幕截图，但仍有人无法工作。详情请至my blog