WinRT C++ (Win10) 从 SoftwareBitmap / BitmapBuffer 访问字节

Question

要在 OpenCV 中处理我的相机 previewFrames，我需要访问原始像素数据/字节。所以，有新的 SoftwareBitmap，它应该正好提供这个。

c# 有一个 example，但在 Visual c++ 中我无法获得 IMemoryBufferByteAccess（见备注）接口工作。

异常代码：

// Capture the preview frame
return create_task(_mediaCapture->GetPreviewFrameAsync(videoFrame))
    .then([this](VideoFrame^ currentFrame)
{
    // Collect the resulting frame
    auto previewFrame = currentFrame->SoftwareBitmap;

    auto buffer = previewFrame->LockBuffer(Windows::Graphics::Imaging::BitmapBufferAccessMode::ReadWrite);
    auto reference = buffer->CreateReference();

    // Get a pointer to the pixel buffer
    byte* pData = nullptr;
    UINT capacity = 0;

    // Obtain ByteAccess
    ComPtr<IUnknown> inspectable = reinterpret_cast<IUnknown*>(buffer);
    // Query the IBufferByteAccess interface.
    Microsoft::WRL::ComPtr<IMemoryBufferByteAccess> bufferByteAccess;
    ThrowIfFailed(inspectable.As(&bufferByteAccess));     // ERROR ---> Throws HRESULT = E_NOINTERFACE

    // Retrieve the buffer data.
    ThrowIfFailed(bufferByteAccess->GetBuffer(_Out_ &pData, _Out_ &capacity));      // ERROR ---> Throws HRESULT = E_NOINTERFACE, because bufferByteAccess is null

我也试过这个：

    HRESULT hr = ((IMemoryBufferByteAccess*)reference)->GetBuffer(&pData, &capacity);

HRESULT 没问题，但我无法访问 pData -> 读取内存时访问冲突。

感谢您的帮助。

Answer 1

您应该在 reinterpret_cast 中使用引用而不是缓冲区。

#include "pch.h"
#include <wrl\wrappers\corewrappers.h>
#include <wrl\client.h>

MIDL_INTERFACE("5b0d3235-4dba-4d44-865e-8f1d0e4fd04d") 
IMemoryBufferByteAccess : IUnknown
{
    virtual HRESULT STDMETHODCALLTYPE GetBuffer(
        BYTE   **value,
        UINT32 *capacity
        );
};

        auto previewFrame = currentFrame->SoftwareBitmap;

        auto buffer = previewFrame->LockBuffer(BitmapBufferAccessMode::ReadWrite);

        auto reference = buffer->CreateReference();

        ComPtr<IMemoryBufferByteAccess> bufferByteAccess;

        HRESULT result = reinterpret_cast<IInspectable*>(reference)->QueryInterface(IID_PPV_ARGS(&bufferByteAccess));

        if (result == S_OK)
        {
            WriteLine("Get interface successfully");

            BYTE* data = nullptr;

            UINT32 capacity = 0;

            result = bufferByteAccess->GetBuffer(&data, &capacity);

            if (result == S_OK)
            {
                WriteLine("get data access successfully, capacity: " + capacity);
            }
        }

Answer 2

我目前正在从我在MediaFrameReader::FrameArrived 事件中获得的每一帧中访问原始unsigned char* data，而不使用 WRL 和 COM...

方法如下：

void MainPage::OnFrameArrived(MediaFrameReader ^reader, MediaFrameArrivedEventArgs ^args)
{
    MediaFrameReference ^mfr = reader->TryAcquireLatestFrame();
    VideoMediaFrame ^vmf = mfr->VideoMediaFrame;
    VideoFrame ^vf = vmf->GetVideoFrame();
    SoftwareBitmap ^sb = vf->SoftwareBitmap; 
    Buffer ^buff = ref new Buffer(sb->PixelHeight * sb->PixelWidth * 2);
    sb->CopyToBuffer(buff);
    DataReader ^dataReader = DataReader::FromBuffer(buffer);
    Platform::Array<unsigned char, 1> ^arr = ref new Platform::Array<unsigned char, 1>(buffer->Length);
    dataReader->ReadBytes(arr);
    // here arr->Data is a pointer to the raw pixel data
}

注意：MediaCapture 对象需要配置MediaCaptureMemoryPreference::Cpu 才能拥有有效的SoftwareBitmap

希望以上内容对某人有所帮助

Answer 3

基于来自@jeffrey-chen 的answer 和来自@kennykerr 的example，我组装了一个更清洁的解决方案：

#include <wrl/client.h>
// other includes, as required by your project

MIDL_INTERFACE("5b0d3235-4dba-4d44-865e-8f1d0e4fd04d")
IMemoryBufferByteAccess : ::IUnknown
{
    virtual HRESULT __stdcall GetBuffer(BYTE **value, UINT32 *capacity) = 0;
};

// your code:
    auto previewFrame = currentFrame->SoftwareBitmap;

    auto buffer = previewFrame->LockBuffer(BitmapBufferAccessMode::ReadWrite);

    auto bufferByteAccess= buffer->CreateReference().as<IMemoryBufferByteAccess>();
    WriteLine("Get interface successfully"); // otherwise - exception is thrown

    BYTE* data = nullptr;
    UINT32 capacity = 0;

    winrt::check_hresult(bufferByteAccess->GetBuffer(&data, &capacity));
    WriteLine("get data access successfully, capacity: " + capacity);