CVOpenGLESTextureCacheCreateTextureFromImage 为 kCVPixelFormatType_OneComponent8答案

【问题标题】：CVOpenGLESTextureCacheCreateTextureFromImage for kCVPixelFormatType_OneComponent8CVOpenGLESTextureCacheCreateTextureFromImage 为 kCVPixelFormatType_OneComponent8
【发布时间】：2020-04-03 17:21:23
【问题描述】：

我正在尝试使用 CVOpenGLESTextureCacheCreateTextureFromImage 以便在 OpenGL 中使用引用，但没有运气：

我有一个CVPixelBufferRef pixel_bufferAlpha，它使用CVPixelBufferCreateWithBytes 更新，成功： CVReturn 成功。

然后我尝试在 1 通道纹理 (alphaMatte) 上使用CVOpenGLESTextureCacheCreateTextureFromImage 来创建可以在 OpenGL 中使用的CVOpenGLESTextureRef。

我已经初始化了我的CVOpenGLESTextureCacheRef_videoTextureAlphaCache：

err = CVOpenGLESTextureCacheCreate(
                                        kCFAllocatorDefault,
                                        nil,
                                        eaglContext,
                                        nil,
                                        &_videoTextureAlphaCache);

我的CVPixelBufferRef pixel_bufferAlpha 使用以下方法初始化：

 cvret = CVPixelBufferCreate(kCFAllocatorDefault,
                                             width,height,
                                             kCVPixelFormatType_OneComponent8,
                                             (__bridge CFDictionaryRef)cvBufferProperties,
                                             &pixel_bufferAlpha);

        if(cvret != kCVReturnSuccess)
        {
            assert(!"Failed to create shared opengl pixel_bufferAlpha");
        }

我正在使用kCVPixelFormatType_OneComponent8，因为我的MTLTexture 传入CVPixelBufferCreateWithBytes 有一个MTLPixelFormat 10 -> MTLPixelFormatR8Unorm。

当尝试使用CVOpenGLESTextureCacheCreateTextureFromImage 时，我收到一个错误“不兼容 opengl”：

CVPixelBufferLockBaseAddress(pixel_bufferAlpha, 0);

        err = noErr;
        err = CVOpenGLESTextureCacheCreateTextureFromImage(kCFAllocatorDefault,
                                                           _videoTextureAlphaCache,
                                                           pixel_bufferAlpha,
                                                           NULL,
                                                           GL_TEXTURE_2D,
                                                           // internal
                                                           // GL_RGBA,
                                                           GL_ALPHA,
                                                           width,
                                                           height,
                                                           // gl format
                                                           // GL_BGRA_EXT,
                                                           // GL_R8_EXT,
                                                           GL_ALPHA8_EXT,
                                                           // gl type
                                                           // GL_UNSIGNED_INT_8_8_8_8_REV,
                                                           GL_UNSIGNED_BYTE,
                                                           NULL,
                                                           &alphaTextureGLES);


        if (err != kCVReturnSuccess) {
            CVBufferRelease(pixel_bufferAlpha);

            if(err == kCVReturnInvalidPixelFormat){
                NSLog(@"Invalid pixel format");
            }

            if(err == kCVReturnInvalidPixelBufferAttributes){
                NSLog(@"Invalid pixel buffer attributes");
            }

            if(err == kCVReturnInvalidSize){
                NSLog(@"invalid size");
            }

            if(err == kCVReturnPixelBufferNotOpenGLCompatible){
                NSLog(@"CVOpenGLESTextureCacheCreateTextureFromImage::not opengl compatible");
            }

        }else{
            NSLog(@"ok CVOpenGLESTextureCacheCreateTextureFromImage SUCCESS");
        }

        // ================================================================================ //

        // clear texture cache
        CVOpenGLESTextureCacheFlush(_videoTextureAlphaCache, 0);
        CVPixelBufferUnlockBaseAddress(pixel_bufferAlpha, 0);

我不确定我在这里做错了什么。

另外我将不胜感激任何指针，因为我不是超级精通 iOS 和纹理转换/格式...

最好的，

代码的完整相关部分：

alphaTexture = [matteDepthTexture  generateMatteFromFrame:_session.currentFrame commandBuffer:commandBuffer];

        // ===============================================================



        NSUInteger texBytesPerRow = alphaTexture.bufferBytesPerRow;
        NSUInteger texArrayLength = alphaTexture.arrayLength;




        int width = (int) alphaTexture.width;
        int height = (int) alphaTexture.height;
        MTLPixelFormat texPixelFormat = alphaTexture.pixelFormat;
        MTLTextureType texType = alphaTexture.textureType;
        int bytesPerPixel = 8;

        // MTLPixelFormatR8Unorm Ordinary format with one 8-bit normalized unsigned integer component.
        NSLog(@" texPixelFormat of the texture is : %d", texPixelFormat);
        NSLog(@" texType of the texture is : %d", texType);

        CVReturn err = noErr;
        err = CVPixelBufferCreateWithBytes(kCFAllocatorDefault,
                                     width,
                                     height,
                                     kCVPixelFormatType_OneComponent8,
                                     alphaTexture,
                                     bytesPerPixel * width,
                                     stillImageDataReleaseCallback,
                                     alphaTexture,
                                     NULL,
                                     &pixel_bufferAlpha);

        if (err != kCVReturnSuccess) {

               if(err == kCVReturnInvalidPixelFormat){
                   NSLog(@"Invalid pixel format");
               }

               if(err == kCVReturnInvalidPixelBufferAttributes){
                   NSLog(@"Invalid pixel buffer attributes");
               }

               if(err == kCVReturnInvalidSize){
                   NSLog(@"invalid size");
               }

               if(err == kCVReturnPixelBufferNotOpenGLCompatible){
                   NSLog(@"CVPixelBufferCreateWithBytes::not opengl compatible");
               }

        }else{
            NSLog(@"ok CVPixelBufferCreateWithBytes SUCCESS");
        }

        OSType sourcePixelFormat = CVPixelBufferGetPixelFormatType(pixel_bufferAlpha);
        if (kCVPixelFormatType_OneComponent8 == sourcePixelFormat) {
           NSLog(@" got format kCVPixelFormatType_OneComponent8");
        } else{
           NSLog(@" Unknown CoreVideo pixel format : %a", sourcePixelFormat);
        }




        // ================================================================================ //
        CVPixelBufferLockBaseAddress(pixel_bufferAlpha, 0);

        err = noErr;
        err = CVOpenGLESTextureCacheCreateTextureFromImage(kCFAllocatorDefault,
                                                           _videoTextureAlphaCache,
                                                           pixel_bufferAlpha,
                                                           NULL,
                                                           GL_TEXTURE_2D,
                                                           // internal
                                                           // GL_RGBA,
                                                           GL_RED_EXT,
                                                           width,
                                                           height,
                                                           // gl format
                                                           // GL_BGRA_EXT,
                                                           // GL_R8_EXT,
                                                           GL_R8_EXT,
                                                           // gl type
                                                           // GL_UNSIGNED_INT_8_8_8_8_REV,
                                                           GL_UNSIGNED_BYTE,
                                                           NULL,
                                                           &alphaTextureGLES);


        if (err != kCVReturnSuccess) {
            CVBufferRelease(pixel_bufferAlpha);

            if(err == kCVReturnInvalidPixelFormat){
                NSLog(@"Invalid pixel format");
            }

            if(err == kCVReturnInvalidPixelBufferAttributes){
                NSLog(@"Invalid pixel buffer attributes");
            }

            if(err == kCVReturnInvalidSize){
                NSLog(@"invalid size");
            }

            if(err == kCVReturnPixelBufferNotOpenGLCompatible){
                NSLog(@"CVOpenGLESTextureCacheCreateTextureFromImage::not opengl compatible");
            }

        }else{
            NSLog(@"ok CVOpenGLESTextureCacheCreateTextureFromImage SUCCESS");
        }

        // ================================================================================ //

        // clear texture cache
        CVOpenGLESTextureCacheFlush(_videoTextureAlphaCache, 0);
        CVPixelBufferUnlockBaseAddress(pixel_bufferAlpha, 0);

【问题讨论】：

标签： ios objective-c opengl-es arkit objective-c-runtime

【解决方案1】：

经过一番挖掘，我从Alloy 找到了 MTLtexture 的扩展 - 那里做得很好 - ：

here

getBytes 将纹理放入 CVPixelBufferRef 然后
CVOpenGLESTextureCacheCreateTextureFromImage 将其存储在 CVOpenGLESTextureRef 中。

所以在 Objective - C 中它看起来像这样 - 对 pixel_bufferAlpha 和 _videoTextureAlphaCache 进行相同的初始化 - ：

        alphaTexture = [matteDepthTexture  generateMatteFromFrame:_session.currentFrame commandBuffer:commandBuffer];

        int width = (int) alphaTexture.width;
        int height = (int) alphaTexture.height;
        MTLPixelFormat texPixelFormat = alphaTexture.pixelFormat;


        CVPixelBufferLockBaseAddress(pixel_bufferAlpha, 0);
        void * CV_NULLABLE pixelBufferBaseAdress = CVPixelBufferGetBaseAddress(pixel_bufferAlpha);
        size_t bytesPerRow = CVPixelBufferGetBytesPerRow(pixel_bufferAlpha);

         [alphaTexture getBytes:pixelBufferBaseAdress
                            bytesPerRow:bytesPerRow
                            fromRegion:MTLRegionMake2D(0, 0, width, height)
                            mipmapLevel:0];


        size_t w = CVPixelBufferGetWidth(pixel_bufferAlpha);
        size_t h = CVPixelBufferGetHeight(pixel_bufferAlpha);

        CVReturn err = noErr;
        err = CVOpenGLESTextureCacheCreateTextureFromImage(kCFAllocatorDefault,
                                                        _videoTextureAlphaCache,
                                                        pixel_bufferAlpha,
                                                        nil,
                                                        GLenum(GL_TEXTURE_2D),
                                                        GLint(GL_LUMINANCE),
                                                        w,
                                                        h,
                                                        GLenum(GL_LUMINANCE),
                                                        GLenum(GL_UNSIGNED_BYTE),
                                                        0,
                                                        &alphaTextureGLES);


        if (err != kCVReturnSuccess) {
            CVBufferRelease(pixel_bufferAlpha);
            NSLog(@"error on CVOpenGLESTextureCacheCreateTextureFromImage");
        }

        CVPixelBufferUnlockBaseAddress(pixel_bufferAlpha, 0);

希望这可以帮助其他人。

【讨论】：