带有 AVCaptureOutput 的 CIDetector

Question

我正在尝试使用 AVFoundation 和 CIDetector 从视频输入中逐帧检测微笑。 CIDetector 在 captureOutput 中使用时在准确性方面非常不一致。根据光线、与相机的距离以及照片中的其他面孔，相同的面部表情可以很快地从微笑转变为不微笑。有没有办法让 CIDetector 不那么紧张而又不让它变慢？

@interface CameraViewController () {
    BOOL smiling;
}

...

- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection {

    CVPixelBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
    CFDictionaryRef attachments = CMCopyDictionaryOfAttachments(kCFAllocatorDefault, sampleBuffer, kCMAttachmentMode_ShouldPropagate);
    CIImage *ciImage = [[CIImage alloc] initWithCVPixelBuffer:pixelBuffer options:(__bridge NSDictionary *)attachments];

    if (attachments)
        CFRelease(attachments);

    //CIImage translates all the pixels -90 deg; set orientation taking this into account

    NSNumber *orientation;
    switch ([UIDevice currentDevice].orientation) {
        case UIDeviceOrientationPortrait:
            orientation = @6;
            break;
        case UIDeviceOrientationPortraitUpsideDown:
            orientation = @8;
            break;
        case UIDeviceOrientationLandscapeLeft:
            orientation = @3;
            break;
        case UIDeviceOrientationLandscapeRight:
            orientation = @1;
            break;
        default:
            orientation = @6;
            break;
    }

    NSArray *faces = [[[SmileDetector sharedDetector] detector] featuresInImage:ciImage
                                                                        options:@{CIDetectorSmile: @(YES), CIDetectorImageOrientation:orientation}];
    smiling = NO;
    for (CIFaceFeature* face in faces)
        if (face.hasSmile) {
            smiling = YES;
            break;
        }
}

SmileDetector 是一个单例，用于在其他 ViewController 中使用相同的 CIDetector 实例。我将检测器精度设置为低，因为高太慢。

- (id)init {
    if (self = [super init]) {
        detector = [CIDetector detectorOfType:CIDetectorTypeFace
                                      context:[CIContext contextWithOptions:nil]
                                      options:@{CIDetectorAccuracy:CIDetectorAccuracyLow}];
    }
    return self;
}

Answer 1

我目前正在处理同样的问题，我发现在 CIDetector 工作之前对帧进行预处理以稳定对比度和亮度效果很好。我的项目是静止帧，我不知道它对视频帧的效果如何......