【问题标题】:How to create CIImage from AVCaptureStillImageOutput in swift?如何快速从 AVCaptureStillImageOutput 创建 CIImage?
【发布时间】:2017-03-01 18:37:00
【问题描述】:

所以我正在使用一些在 Objective C 中执行此操作的代码,我一直在将其转换为 swift,我正在努力从 AVCaptureStillImageOutput 创建一个 CIImage。因此,如果有人可以查看这段代码并告诉我哪里出错了,那就太好了。

这是 Objective C 代码

- (void)captureImageWithCompletionHander:(void(^)(NSString *fullPath))completionHandler
{ 
dispatch_suspend(_captureQueue); 

AVCaptureConnection *videoConnection = nil;
for (AVCaptureConnection *connection in self.stillImageOutput.connections)
{
    for (AVCaptureInputPort *port in connection.inputPorts)
    {
        if ([port.mediaType isEqual:AVMediaTypeVideo] )
        {
            videoConnection = connection;
            break;
        }
    }
    if (videoConnection) break;
}

__weak typeof(self) weakSelf = self;

[self.stillImageOutput captureStillImageAsynchronouslyFromConnection:videoConnection completionHandler: ^(CMSampleBufferRef imageSampleBuffer, NSError *error)
 {
     if (error)
     {
         dispatch_resume(_captureQueue);
         return;
     }

     __block NSArray *filePath = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES); //create an array and store result of our search for the documents directory in it

     NSString *documentsDirectory = [filePath objectAtIndex:0]; //create NSString object, that holds our exact path to the documents directory

     NSString *fullPath = [documentsDirectory stringByAppendingPathComponent:[NSString stringWithFormat:@"/iScan_img_%i.pdf",(int)[NSDate date].timeIntervalSince1970]];


     @autoreleasepool
     {
         NSData *imageData = [AVCaptureStillImageOutput jpegStillImageNSDataRepresentation:imageSampleBuffer];
         CIImage *enhancedImage = [[CIImage alloc] initWithData:imageData options:@{kCIImageColorSpace:[NSNull null]}];
         imageData = nil;

         if (weakSelf.cameraViewType == DocScannerCameraViewTypeBlackAndWhite)
         {
             enhancedImage = [self filteredImageUsingEnhanceFilterOnImage:enhancedImage];
         }
         else
         {
             enhancedImage = [self filteredImageUsingContrastFilterOnImage:enhancedImage];
         }

         if (weakSelf.isBorderDetectionEnabled && rectangleDetectionConfidenceHighEnough(_imageDedectionConfidence))
         {
             CIRectangleFeature *rectangleFeature = [self biggestRectangleInRectangles:[[self highAccuracyRectangleDetector] featuresInImage:enhancedImage]];

             if (rectangleFeature)
             {
                 enhancedImage = [self correctPerspectiveForImage:enhancedImage withFeatures:rectangleFeature];
             }
         }

         CIFilter *transform = [CIFilter filterWithName:@"CIAffineTransform"];
         [transform setValue:enhancedImage forKey:kCIInputImageKey];
         NSValue *rotation = [NSValue valueWithCGAffineTransform:CGAffineTransformMakeRotation(-90 * (M_PI/180))];
         [transform setValue:rotation forKey:@"inputTransform"];
         enhancedImage = transform.outputImage;

         if (!enhancedImage || CGRectIsEmpty(enhancedImage.extent)) return;

         static CIContext *ctx = nil;
         if (!ctx)
         {
             ctx = [CIContext contextWithOptions:@{kCIContextWorkingColorSpace:[NSNull null]}];
         }

         CGSize bounds = enhancedImage.extent.size;
         bounds = CGSizeMake(floorf(bounds.width / 4) * 4,floorf(bounds.height / 4) * 4);
         CGRect extent = CGRectMake(enhancedImage.extent.origin.x, enhancedImage.extent.origin.y, bounds.width, bounds.height);

         static int bytesPerPixel = 8;
         uint rowBytes = bytesPerPixel * bounds.width;
         uint totalBytes = rowBytes * bounds.height;
         uint8_t *byteBuffer = malloc(totalBytes);

         CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();

         [ctx render:enhancedImage toBitmap:byteBuffer rowBytes:rowBytes bounds:extent format:kCIFormatRGBA8 colorSpace:colorSpace];

         CGContextRef bitmapContext = CGBitmapContextCreate(byteBuffer,bounds.width,bounds.height,bytesPerPixel,rowBytes,colorSpace,kCGImageAlphaNoneSkipLast);
         CGImageRef imgRef = CGBitmapContextCreateImage(bitmapContext);
         CGColorSpaceRelease(colorSpace);
         CGContextRelease(bitmapContext);
         free(byteBuffer);

         if (imgRef == NULL)
         {
             CFRelease(imgRef);
             return;
         }
         saveCGImageAsJPEGToFilePath(imgRef, fullPath);



         CFRelease(imgRef);

         dispatch_async(dispatch_get_main_queue(), ^
                        {
                            completionHandler(fullPath);

                            dispatch_resume(_captureQueue);
                        });

         _imageDedectionConfidence = 0.0f;
     }
 }];

}

现在基本上它会捕获内容,如果某些if 语句为真,那么它会捕获显示的CIRectangleFeature 中的内容,然后将CIImage 转换为CGImage 以在保存函数中调用。

我已经把它翻译成这样了。

func captureImage(completionHandler: @escaping (_ imageFilePath: String) -> Void) {

    self.captureQueue?.suspend()
    var videoConnection: AVCaptureConnection!
    for connection in self.stillImageOutput.connections{
        for port in (connection as! AVCaptureConnection).inputPorts {
            if (port as! AVCaptureInputPort).mediaType.isEqual(AVMediaTypeVideo) {
                videoConnection = connection as! AVCaptureConnection
                break
            }
        }
        if videoConnection != nil {
            break
        }
    }
    weak var weakSelf = self
    self.stillImageOutput.captureStillImageAsynchronously(from: videoConnection) { (sampleBuffer, error) -> Void in
        if error != nil {
            self.captureQueue?.resume()
            return
        }
        let filePath = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true)
        let documentsDirectory: String = filePath[0]
        let fullPath: String = URL(fileURLWithPath: documentsDirectory).appendingPathComponent("iScan_img_\(Int(Date().timeIntervalSince1970)).pdf").absoluteString
        autoreleasepool {
            let imageData = Data(AVCaptureStillImageOutput.jpegStillImageNSDataRepresentation(sampleBuffer))
            var enhancedImage = CIImage(data: imageData, options: [kCIImageColorSpace: NSNull()])


            if weakSelf?.cameraViewType == DocScannerCameraViewType.blackAndWhite {
                enhancedImage = self.filteredImageUsingEnhanceFilter(on: enhancedImage!)
            }
            else {
                enhancedImage = self.filteredImageUsingContrastFilter(on: enhancedImage!)
            }
            if (weakSelf?.isEnableBorderDetection == true) && self.rectangleDetectionConfidenceHighEnough(confidence: self.imageDedectionConfidence) {
                let rectangleFeature: CIRectangleFeature? = self.biggestRectangles(rectangles: self.highAccuracyRectangleDetector().features(in: enhancedImage!))
                if rectangleFeature != nil {
                    enhancedImage = self.correctPerspective(for: enhancedImage!, withFeatures: rectangleFeature!)
                }
            }
            let transform = CIFilter(name: "CIAffineTransform")
            let rotation = NSValue(cgAffineTransform: CGAffineTransform(rotationAngle: -90 * (.pi / 180)))
            transform?.setValue(rotation, forKey: "inputTransform")
            enhancedImage = transform?.outputImage
            if (enhancedImage == nil) || (enhancedImage?.extent.isEmpty)! {
                return
            }
            var ctx: CIContext?
            if (ctx != nil) {
                ctx = CIContext(options: [kCIContextWorkingColorSpace: NSNull()])
            }
            var bounds: CGSize = (enhancedImage?.extent.size)!
            bounds = CGSize(width: CGFloat((floorf(Float(bounds.width)) / 4) * 4), height: CGFloat((floorf(Float(bounds.height)) / 4) * 4))
            let extent = CGRect(x: CGFloat((enhancedImage?.extent.origin.x)!), y: CGFloat((enhancedImage?.extent.origin.y)!), width: CGFloat(bounds.width), height: CGFloat(bounds.height))
            let bytesPerPixel: CGFloat = 8
            let rowBytes = bytesPerPixel * bounds.width
            let totalBytes = rowBytes * bounds.height
            let byteBuffer = malloc(Int(totalBytes))
            let colorSpace = CGColorSpaceCreateDeviceRGB()
            ctx!.render(enhancedImage!, toBitmap: byteBuffer!, rowBytes: Int(rowBytes), bounds: extent, format: kCIFormatRGBA8, colorSpace: colorSpace)
            let bitmapContext = CGContext(data: byteBuffer, width: Int(bounds.width), height: Int(bounds.height), bitsPerComponent: Int(bytesPerPixel), bytesPerRow: Int(rowBytes), space: colorSpace, bitmapInfo: CGImageAlphaInfo.noneSkipLast.rawValue)
            let imgRef = bitmapContext?.makeImage()
            free(byteBuffer)

            self.saveCGImageAsJPEGToFilePath(imgRef: imgRef!, filePath: fullPath)
            DispatchQueue.main.async(execute: {() -> Void in
                completionHandler(fullPath)
                self.captureQueue?.resume()
            })
            self.imageDedectionConfidence = 0.0
        }
    }
}

所以它需要 AVCaptureStillImageOutput 将其转换为 CIImage 以用于所有需要的用途,然后将其转换为 CGImage 以进行保存。我在翻译中到底做错了什么?还是有更好的方法来做到这一点?

我真的不想问这个问题,但我似乎找不到任何像这样的问题,或者至少找不到任何涉及从AVCaptureStillImageOutput 捕获为CIImage 的问题。

感谢您的帮助!

【问题讨论】:

    标签: ios objective-c swift avcaptureoutput


    【解决方案1】:

    这是正确的 swift 翻译再次感谢 Prientus 帮助我找到我的错误

    func captureImage(completionHandler: @escaping (_ imageFilePath: String) -> Void) {
    
        self.captureQueue?.suspend()
        var videoConnection: AVCaptureConnection!
        for connection in self.stillImageOutput.connections{
            for port in (connection as! AVCaptureConnection).inputPorts {
                if (port as! AVCaptureInputPort).mediaType.isEqual(AVMediaTypeVideo) {
                    videoConnection = connection as! AVCaptureConnection
                    break
                }
            }
            if videoConnection != nil {
                break
            }
        }
        weak var weakSelf = self
        self.stillImageOutput.captureStillImageAsynchronously(from: videoConnection) { (sampleBuffer: CMSampleBuffer?, error) -> Void in
            if error != nil {
                self.captureQueue?.resume()
                return
            }
            let filePath = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true)
            let documentsDirectory: String = filePath[0]
            let fullPath: String = documentsDirectory.appending("/iScan_img_\(Int(Date().timeIntervalSince1970)).pdf")
            autoreleasepool {
    
                let imageData = Data(AVCaptureStillImageOutput.jpegStillImageNSDataRepresentation(sampleBuffer))
                var enhancedImage = CIImage(data: imageData, options: [kCIImageColorSpace: NSNull()])
    
    
                if weakSelf?.cameraViewType == DocScannerCameraViewType.blackAndWhite {
                    enhancedImage = self.filteredImageUsingEnhanceFilter(on: enhancedImage!)
                }
                else {
                    enhancedImage = self.filteredImageUsingContrastFilter(on: enhancedImage!)
                }
                if (weakSelf?.isEnableBorderDetection == true) && self.rectangleDetectionConfidenceHighEnough(confidence: self.imageDedectionConfidence) {
                    let rectangleFeature: CIRectangleFeature? = self.biggestRectangles(rectangles: self.highAccuracyRectangleDetector().features(in: enhancedImage!))
                    if rectangleFeature != nil {
                        enhancedImage = self.correctPerspective(for: enhancedImage!, withFeatures: rectangleFeature!)
                    }
                }
                let transform = CIFilter(name: "CIAffineTransform")
                transform?.setValue(enhancedImage, forKey: kCIInputImageKey)
                let rotation = NSValue(cgAffineTransform: CGAffineTransform(rotationAngle: -90 * (.pi / 180)))
                transform?.setValue(rotation, forKey: "inputTransform")
                enhancedImage = (transform?.outputImage)!
                if (enhancedImage == nil) || (enhancedImage?.extent.isEmpty)! {
                    return
                }
                var ctx: CIContext?
                if (ctx == nil) {
                    ctx = CIContext(options: [kCIContextWorkingColorSpace: NSNull()])
                }
                var bounds: CGSize = (enhancedImage!.extent.size)
                bounds = CGSize(width: CGFloat((floorf(Float(bounds.width)) / 4) * 4), height: CGFloat((floorf(Float(bounds.height)) / 4) * 4))
                let extent = CGRect(x: CGFloat((enhancedImage?.extent.origin.x)!), y: CGFloat((enhancedImage?.extent.origin.y)!), width: CGFloat(bounds.width), height: CGFloat(bounds.height))
                let bytesPerPixel: CGFloat = 8
                let rowBytes = bytesPerPixel * bounds.width
                let totalBytes = rowBytes * bounds.height
                let byteBuffer = malloc(Int(totalBytes))
                let colorSpace = CGColorSpaceCreateDeviceRGB()
                ctx!.render(enhancedImage!, toBitmap: byteBuffer!, rowBytes: Int(rowBytes), bounds: extent, format: kCIFormatRGBA8, colorSpace: colorSpace)
                let bitmapContext = CGContext(data: byteBuffer, width: Int(bounds.width), height: Int(bounds.height), bitsPerComponent: Int(bytesPerPixel), bytesPerRow: Int(rowBytes), space: colorSpace, bitmapInfo: CGImageAlphaInfo.noneSkipLast.rawValue)
                let imgRef = bitmapContext?.makeImage()
                free(byteBuffer)
                if imgRef == nil {
                    return
                }
                self.saveCGImageAsJPEGToFilePath(imgRef: imgRef!, filePath: fullPath)
                DispatchQueue.main.async(execute: {() -> Void in
                    completionHandler(fullPath)
                    self.captureQueue?.resume()
                })
                self.imageDedectionConfidence = 0.0
            }
        }
    }
    

    【讨论】:

      【解决方案2】:

      尝试用以下几行替换您的 CIImage 创建:

      guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer), let enhancedImage = CIImage(cvPixelBuffer: pixelBuffer) else {
          return
      }
      

      【讨论】:

      • @ Prientus 在实际捕获图像之前,我不应该需要CIImage 来绘制CIRectangleFeature 以及我正在使用的任何过滤器吗?我知道如何捕获为CGImage 然后转换,但我应该需要CIImage 来处理我在捕获之前对图像所做的所有事情。不过感谢您的回答!
      • @CarpenterBlood 哦,我明白了,我误解了优先级。对不起。不过我还是会考虑一下:)
      • @CarpenterBlood 请查看我的更新答案。我认为您的代码可能无法正常工作,因为您可能没有在 autoreleasepool 中正确获取 CIImage。
      • @Prientus 我会尝试将其更改为那个。但是我实际上已经在函数func captureOutput( _ captureOutput: AVCaptureOutput, didOutPutSampleBuffer sampleBuffer: CMSampleBuffer?, from connection: AVCaptureConnection) {} 中调用了它,它与Objective c 一样完美地工作,并且在swift enhancedImage 中返回nil
      • @CarpenterBlood 很奇怪,因为我在同一个地方调用它,它对我来说很好用。 :S 您的 sampleBuffer nil 是否出于某种奇怪的原因可能是该方法?我知道它的力量展开了,但谁知道......
      猜你喜欢
      • 2012-02-24
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2023-03-28
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多