如何将 Vision 输出显示到 UI 中？

Question

我对编码比较陌生，最近我一直在开发一个程序，该程序允许用户使用 iPhone 的后置摄像头扫描水晶，它会识别水晶的种类。我使用 CreateML 来构建模型，并使用 Vision 来识别晶体。我似乎无法弄清楚如何将结果放入我构建的 UI 中。结果正在打印到 Xcode 控制台。

这是故事板的图片：

Answer 1

我假设您想在检测到的水晶周围画一个框？

你的水晶应该是boundingBox，看起来像这样：

(0.166666666666667, 0.35, 0.66666666666667, 0.3)

这些是“标准化”坐标，这意味着它们与您发送到 Vision 的图像相关。我更详细地解释这个here...

What you are used to	What Vision returns

您需要将这些“标准化”坐标转换为您可以使用的 UIKit 坐标。为此，我有这个转换功能：

func getConvertedRect(boundingBox: CGRect, inImage imageSize: CGSize, containedIn containerSize: CGSize) -> CGRect {
    
    let rectOfImage: CGRect
    
    let imageAspect = imageSize.width / imageSize.height
    let containerAspect = containerSize.width / containerSize.height
    
    if imageAspect > containerAspect { /// image extends left and right
        let newImageWidth = containerSize.height * imageAspect /// the width of the overflowing image
        let newX = -(newImageWidth - containerSize.width) / 2
        rectOfImage = CGRect(x: newX, y: 0, width: newImageWidth, height: containerSize.height)
        
    } else { /// image extends top and bottom
        let newImageHeight = containerSize.width * (1 / imageAspect) /// the width of the overflowing image
        let newY = -(newImageHeight - containerSize.height) / 2
        rectOfImage = CGRect(x: 0, y: newY, width: containerSize.width, height: newImageHeight)
    }
    
    let newOriginBoundingBox = CGRect(
    x: boundingBox.origin.x,
    y: 1 - boundingBox.origin.y - boundingBox.height,
    width: boundingBox.width,
    height: boundingBox.height
    )
    
    var convertedRect = VNImageRectForNormalizedRect(newOriginBoundingBox, Int(rectOfImage.width), Int(rectOfImage.height))
    
    /// add the margins
    convertedRect.origin.x += rectOfImage.origin.x
    convertedRect.origin.y += rectOfImage.origin.y
    
    return convertedRect
}

你可以这样使用它：

let convertedRect = self.getConvertedRect(
    boundingBox: observation.boundingBox,
    inImage: image.size, /// image is the image that you feed into Vision
    containedIn: self.previewView.bounds.size /// the size of your camera feed's preview view
)
self.drawBoundingBox(rect: convertedRect)

/// draw the rectangle
func drawBoundingBox(rect: CGRect) {
    let uiView = UIView(frame: rect)
    previewView.addSubview(uiView)
        
    uiView.backgroundColor = UIColor.orange.withAlphaComponent(0.2)
    uiView.layer.borderColor = UIColor.orange.cgColor
    uiView.layer.borderWidth = 3
}

结果（我正在做一个VNDetectRectanglesRequest）：

如果您想在手机移动时“跟踪”检测到的物体，请在此处查看我的answer