使用 Vision 扫描照片库中的图像

Question

有没有一种方法可以使用Vision 框架从用户的照片库中扫描现有图像？比如，不使用相机拍摄新照片，而只是选择用户已有的图像？

Answer 1

是的，你可以。首先，获取 UIImagePickerController 的实例并呈现它。

let picker = UIImagePickerController()
picker.delegate = self
picker.sourceType = .photoLibrary
present(picker, animated: true, completion: nil)

然后实现委托方法获取想要的图片

extension YourViewController: UIImagePickerControllerDelegate {
func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [UIImagePickerController.InfoKey : Any]) {
    if let pickedImage = info[.originalImage] as? UIImage {
        
        ## here start your request & request handler
    }
    
    picker.dismiss(animated: true, completion: nil)
    
    }
}

Answer 2

是的，你可以。添加到@Zulqarnayn 的答案，这是一个检测和绘制矩形边界框的工作示例。

1。设置要显示图像的图像视图

@IBOutlet weak var imageView: UIImageView!

@IBAction func pickImage(_ sender: Any) {
    let picker = UIImagePickerController()
    picker.delegate = self
    self.present(picker, animated: true)
}

override func viewDidLoad() {
    super.viewDidLoad()
    
    imageView.layer.borderWidth = 4
    imageView.layer.borderColor = UIColor.blue.cgColor
    imageView.contentMode = .scaleAspectFill
    
    imageView.backgroundColor = UIColor.green.withAlphaComponent(0.3)
    imageView.layer.masksToBounds = false /// allow image to overflow, for testing purposes
    
}

2。从图片选择器中获取图片

extension ViewController: UIImagePickerControllerDelegate, UINavigationControllerDelegate {
    func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [UIImagePickerController.InfoKey : Any]) {
        guard let image = info[.originalImage] as? UIImage else { return }

        /// set the imageView's image
        imageView.image = image

        /// start the request & request handler
        detectCard()

        /// dismiss the picker
        dismiss(animated: true)
    }
}

3。启动视觉请求

func detectCard() {
    guard let cgImage = imageView.image?.cgImage else { return }

    /// perform on background thread, so the main screen is not frozen
    DispatchQueue.global(qos: .userInitiated).async {
        
        let request = VNDetectRectanglesRequest { request, error in

            /// this function will be called when the Vision request finishes
            self.handleDetectedRectangle(request: request, error: error)
        }
        
        request.minimumAspectRatio = 0.0
        request.maximumAspectRatio = 1.0
        request.maximumObservations = 1 /// only look for 1 rectangle
        
        let imageRequestHandler = VNImageRequestHandler(cgImage: cgImage, orientation: .up)
        
        do {
            try imageRequestHandler.perform([request])
        } catch let error {
            print("Error: \(error)")
        }
    }
}

4。从 Vision 请求中获取结果

func handleDetectedRectangle(request: VNRequest?, error: Error?) {
    if let results = request?.results {
        if let observation = results.first as? VNRectangleObservation {
            
            /// get back to the main thread
            DispatchQueue.main.async {
                guard let image = self.imageView.image else { return }
                
                let convertedRect = self.getConvertedRect(
                    boundingBox: observation.boundingBox,
                    inImage: image.size,
                    containedIn: self.imageView.bounds.size
                )
                self.drawBoundingBox(rect: convertedRect)
            }
        }
    }
}

5。将observation.boundingBox转换为图像视图的UIKit坐标，然后在检测到的矩形周围画一个边框

我在answer 中对此进行了更详细的解释。

func getConvertedRect(boundingBox: CGRect, inImage imageSize: CGSize, containedIn containerSize: CGSize) -> CGRect {
    
    let rectOfImage: CGRect
    
    let imageAspect = imageSize.width / imageSize.height
    let containerAspect = containerSize.width / containerSize.height
    
    if imageAspect > containerAspect { /// image extends left and right
        let newImageWidth = containerSize.height * imageAspect /// the width of the overflowing image
        let newX = -(newImageWidth - containerSize.width) / 2
        rectOfImage = CGRect(x: newX, y: 0, width: newImageWidth, height: containerSize.height)
        
    } else { /// image extends top and bottom
        let newImageHeight = containerSize.width * (1 / imageAspect) /// the width of the overflowing image
        let newY = -(newImageHeight - containerSize.height) / 2
        rectOfImage = CGRect(x: 0, y: newY, width: containerSize.width, height: newImageHeight)
    }
    
    let newOriginBoundingBox = CGRect(
    x: boundingBox.origin.x,
    y: 1 - boundingBox.origin.y - boundingBox.height,
    width: boundingBox.width,
    height: boundingBox.height
    )
    
    var convertedRect = VNImageRectForNormalizedRect(newOriginBoundingBox, Int(rectOfImage.width), Int(rectOfImage.height))
    
    /// add the margins
    convertedRect.origin.x += rectOfImage.origin.x
    convertedRect.origin.y += rectOfImage.origin.y
    
    return convertedRect
}

/// draw an orange frame around the detected rectangle, on top of the image view
func drawBoundingBox(rect: CGRect) {
    let uiView = UIView(frame: rect)
    imageView.addSubview(uiView)
    
    uiView.backgroundColor = UIColor.clear
    uiView.layer.borderColor = UIColor.orange.cgColor
    uiView.layer.borderWidth = 3
}

结果 | Demo repo

Input image	Result