使用带有 MTLTexture 的 CARenderer 将动画 CALayer 渲染到屏幕外

Question

我想将动画NSView（或只是底层的CALayer）渲染成一系列图像，而不在屏幕上ATALL呈现视图。我想出了如何使用CARenderer 和MTLTexture 来做到这一点，但以下方法存在一些问题。

这在操场上运行并将输出存储到您下载的Off-screen Render 文件夹：

import AppKit
import Metal
import QuartzCore
import PlaygroundSupport

let view = NSView(frame: CGRect(x: 0, y: 0, width: 600, height: 400))
let circle = NSView(frame: CGRect(x: 0, y: 0, width: 50, height: 50))

circle.wantsLayer = true
circle.layer?.backgroundColor = NSColor.red.cgColor
circle.layer?.cornerRadius = 25
view.wantsLayer = true
view.addSubview(circle)

let textureDescriptor = MTLTextureDescriptor.texture2DDescriptor(pixelFormat: .rgba8Unorm, width: 600, height: 400, mipmapped: false)
textureDescriptor.usage = [MTLTextureUsage.shaderRead, .shaderWrite, .renderTarget]

let device = MTLCreateSystemDefaultDevice()!
let texture: MTLTexture = device.makeTexture(descriptor: textureDescriptor)!
let context = CIContext(mtlDevice: device)
let renderer = CARenderer(mtlTexture: texture)

renderer.layer = view.layer
renderer.bounds = view.frame

let outputURL: URL = try! FileManager.default.url(for: .downloadsDirectory, in: .userDomainMask, appropriateFor: nil, create: false).appendingPathComponent("Off-screen Render")
try? FileManager.default.removeItem(at: outputURL)
try! FileManager.default.createDirectory(at: outputURL, withIntermediateDirectories: true, attributes: nil)

var frameNumber: Int = 0

func render() {
    Swift.print("Rendering frame #\(frameNumber)…")

    renderer.beginFrame(atTime: CACurrentMediaTime(), timeStamp: nil)
    renderer.addUpdate(renderer.bounds)
    renderer.render()
    renderer.endFrame()

    let ciImage: CIImage = CIImage(mtlTexture: texture)!
    let cgImage: CGImage = context.createCGImage(ciImage, from: ciImage.extent)!
    let url: URL = outputURL.appendingPathComponent("frame-\(frameNumber).png")
    let destination: CGImageDestination = CGImageDestinationCreateWithURL(url as CFURL, kUTTypePNG, 1, nil)!
    CGImageDestinationAddImage(destination, cgImage, nil)
    guard CGImageDestinationFinalize(destination) else { fatalError() }

    frameNumber += 1
}

var timer: Timer?

NSAnimationContext.runAnimationGroup({ context in
    context.duration = 0.25
    view.animator().frame.origin = CGPoint(x: 550, y: 350)
}, completionHandler: {
    timer?.invalidate()
    render()
    Swift.print("Finished off-screen rendering of \(frameNumber) frames in \(outputURL.path)…")
})

// Make the first render immediately after the animation start and after it completes. For the purpose
// of this demo timer is used instead of display link.

render()
timer = Timer.scheduledTimer(withTimeInterval: 1 / 30, repeats: true, block: { _ in render() })

上述代码的问题如下附件所示，分别是：

1. 纹理没有得到清理，并且每一帧都在前一个渲染之上绘制。我知道我可以使用replace(region:…)，但怀疑它与具有清晰颜色描述的渲染通道相比效率不高。这是真的？渲染通道可以和CARenderer一起使用吗？

2. 第一帧（在实际项目中是两到三帧）通常是空的。我怀疑这与CARenderer 渲染或CGImage 使用Core Image 构造期间的一些异步行为有关。如何避免这种情况？纹理上有某种等待直到渲染完成的回调吗？

Answer 1

在与 Apple 开发者技术支持交谈后，似乎：

Core Image 延迟渲染，直到客户端请求访问帧缓冲区，即CVPixelBufferLockBaseAddress。

所以，解决方法就是在调用CIContext.render之后做CVPixelBufferLockBaseAddress，如下所示：

for frameNumber in 0 ..< frameCount {
    var pixelBuffer: CVPixelBuffer?
    guard let pixelBufferPool: CVPixelBufferPool = pixelBufferAdaptor.pixelBufferPool else { preconditionFailure() }
    precondition(CVPixelBufferPoolCreatePixelBuffer(nil, pixelBufferPool, &pixelBuffer) == kCVReturnSuccess)

    let ciImage = CIImage(cgImage: frameImage)
    context.render(ciImage, to: pixelBuffer!)

    precondition(CVPixelBufferLockBaseAddress(pixelBuffer!, []) == kCVReturnSuccess)
    defer { precondition(CVPixelBufferUnlockBaseAddress(pixelBuffer!, []) == kCVReturnSuccess) }

    let bytes = UnsafeBufferPointer(start: CVPixelBufferGetBaseAddress(pixelBuffer!)!.assumingMemoryBound(to: UInt8.self), count: CVPixelBufferGetDataSize(pixelBuffer!))
    precondition(bytes.contains(where: { $0 != 0 }))

    while !input.isReadyForMoreMediaData { Thread.sleep(forTimeInterval: 10 / 1000) }
    precondition(pixelBufferAdaptor.append(pixelBuffer!, withPresentationTime: CMTime(seconds: Double(frameNumber) * frameRate, preferredTimescale: 600)))
}

附：这与Making CIContext.render(CIImage, CVPixelBuffer) work with AVAssetWriter 问题的答案相同——您可能需要查看它以更深入地了解在使用 AVFoundation 时此问题可能发生的位置和方式。虽然问题不同，但解决方案完全相同。

Answer 2

我认为您可以使用AVVideoCompositionCoreAnimationTool 来渲染带有动画的视图。