在osx上将视频帧传递给Core Image

Question

大家好，你们这些了不起的程序员！在过去的几周里，我从各种有用的资源中收集了这个东西（包括来自 stackoverflow 的很多帖子），试图创建一些可以接收网络摄像头并在微笑发生时检测到微笑的东西（不妨在周围画框面孔和微笑也是如此，一旦被检测到，这似乎并不难）。如果代码混乱，请给我一些余地，因为我还在学习。 目前我一直在尝试将图像传递给 CIImage 以便可以分析面部（我计划在克服面部障碍后处理微笑）。因为如果我在 (5) 之后注释掉该块，编译器就会成功 - 它会在窗口中显示一个简单的 AVCaptureVideoPreviewLayer。我认为这就是我所说的“rootLayer”，所以它就像显示输出的第一层，在我检测到视频帧中的人脸之后，我将在任何检测到的人脸的“边界”之后显示一个矩形新图层覆盖在这一层之上，我称该层为“previewLayer”...对吗？

但是在 (5) 之后的块中，编译器会抛出三个错误 -

架构 x86_64 的未定义符号： “_CMCopyDictionaryOfAttachments”，引用自： -[AVRecorderDocument captureOutput:didOutputSampleBuffer:fromConnection:] 在 AVRecorderDocument.o “_CMSampleBufferGetImageBuffer”，引用自： -[AVRecorderDocument captureOutput:didOutputSampleBuffer:fromConnection:] 在 AVRecorderDocument.o ld：未找到架构 x86_64 的符号 clang：错误：链接器命令失败，退出代码为 1（使用 -v 查看调用）

谁能告诉我哪里出了问题以及我的下一步是什么？

感谢您的帮助，我在这一点上卡住了几天，我无法弄清楚，我能找到的所有示例都是针对 IOS 的，在 OSX 中不起作用。

    - (id)init
{
    self = [super init];
    if (self) {

        // Move the output part to another function
        [self addVideoDataOutput];

        // Create a capture session
        session = [[AVCaptureSession alloc] init];

        // Set a session preset (resolution)
        self.session.sessionPreset = AVCaptureSessionPreset640x480;

        // Select devices if any exist
        AVCaptureDevice *videoDevice = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo];
        if (videoDevice) {
            [self setSelectedVideoDevice:videoDevice];
        } else {
            [self setSelectedVideoDevice:[AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeMuxed]];
        }
        NSError *error = nil;
        //  Add an input
        videoDeviceInput = [AVCaptureDeviceInput deviceInputWithDevice:videoDevice error:&error];
        [self.session addInput:self.videoDeviceInput];

        // Start the session (app opens slower if it is here but I think it is needed in order to send the frames for processing)
        [[self session] startRunning];


          // Initial refresh of device list
         [self refreshDevices];

    }
    return self;
}

-(void) addVideoDataOutput {
    // (1) Instantiate a new video data output object
    AVCaptureVideoDataOutput * captureOutput = [[AVCaptureVideoDataOutput alloc] init];
    captureOutput.videoSettings = @{ (NSString *) kCVPixelBufferPixelFormatTypeKey : @(kCVPixelFormatType_32BGRA) };

    // discard if the data output queue is blocked (while CI processes the still image)
    captureOutput.alwaysDiscardsLateVideoFrames = YES;

    // (2) The sample buffer delegate requires a serial dispatch queue
    dispatch_queue_t captureOutputQueue;
    captureOutputQueue = dispatch_queue_create("CaptureOutputQueue", DISPATCH_QUEUE_SERIAL);
    [captureOutput setSampleBufferDelegate:self queue:captureOutputQueue];
    dispatch_release(captureOutputQueue);  //what does this do and should it be here or after we receive the processed image back?

    // (3) Define the pixel format for the video data output 
    NSString * key = (NSString*)kCVPixelBufferPixelFormatTypeKey;
    NSNumber * value = [NSNumber numberWithUnsignedInt:kCVPixelFormatType_32BGRA];
    NSDictionary * settings = @{key:value};
    [captureOutput setVideoSettings:settings];

    // (4) Configure the output port on the captureSession property
    if ( [self.session canAddOutput:captureOutput] )
    [session addOutput:captureOutput];

}
// Implement the Sample Buffer Delegate Method
- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection {

// I *think* I have a video frame now in some sort of image format... so have to convert it into a CIImage before I can process it:

    // (5) Convert CMSampleBufferRef to CVImageBufferRef, then to a CI Image (per weichsel's answer in July '13)
    CVImageBufferRef cvFrameImage = CMSampleBufferGetImageBuffer(sampleBuffer);  // Having trouble here, prog. stops and won't recognise CMSampleBufferGetImageBuffer.
    CFDictionaryRef attachments = CMCopyDictionaryOfAttachments(kCFAllocatorDefault, sampleBuffer, kCMAttachmentMode_ShouldPropagate);
    self.ciFrameImage = [[CIImage alloc] initWithCVImageBuffer:cvFrameImage options:(__bridge NSDictionary *)attachments];
    //self.ciFrameImage = [[CIImage alloc] initWithCVImageBuffer:cvFrameImage];

    //OK so it is a CIImage. Find some way to send it to a separate CIImage function to find the faces, then smiles.  Then send it somewhere else to be displayed on top of AVCaptureVideoPreviewLayer
    //TBW

}


- (NSString *)windowNibName
{
    return @"AVRecorderDocument";
}


- (void)windowControllerDidLoadNib:(NSWindowController *) aController
{
    [super windowControllerDidLoadNib:aController];

    // Attach preview to session
    CALayer *rootLayer = self.previewView.layer;
    [rootLayer setMasksToBounds:YES]; //aaron added
    self.previewLayer = [[AVCaptureVideoPreviewLayer alloc] initWithSession:self.session];
    [self.previewLayer setBackgroundColor:CGColorGetConstantColor(kCGColorBlack)];
    [self.previewLayer setVideoGravity:AVLayerVideoGravityResizeAspect];
    [self.previewLayer setFrame:[rootLayer bounds]];
    //[newPreviewLayer setAutoresizingMask:kCALayerWidthSizable | kCALayerHeightSizable];  //don't think I need this for OSX?
    [self.previewLayer setVideoGravity:AVLayerVideoGravityResizeAspect];
    [rootLayer addSublayer:previewLayer];
//  [newPreviewLayer release];  //what's this for?


}

Answer 1

（从 cmets 部分移出）

哇。我想两天后，一篇 StackOverflow 帖子就可以确定我没有将 CoreMedia.framework 添加到我的项目中。