UWP:- 如何在非托管中更新 MediaStreamSample.Timestamp 而无需一次又一次地创建媒体流样本

Question

我一直在以 30 fps 从混合现实 webrtc 获取 I420 视频帧。我正在使用以下代码创建媒体示例并将框架从 webrtc 复制到此媒体示例，并将其附加到从 MediaStreamSource.SampleRequested 事件生成的示例中。但是，UWP 应用中的媒体播放器越多，每次都创建媒体流示例，渲染体验看起来并不好。

据说可以在非托管代码中更新媒体流示例时间戳，而无需使用更新的时间戳创建新的媒体流示例，从而节省 GC 花费的时间。但我不确定如何。有人可以说明如何在非托管代码中实现这一点吗？

public class StreamSamplePool
{         
      /// <summary>
      /// Queue of buffers in use by a sample.
      /// </summary>
      /// <remarks>
      /// Newly used buffers are added on the back of the queue, and removed
        /// from the front once the <see cref="Windows.Media.Core.MediaStreamSample.Processed"/>
        /// signal is fired. Because some users report out-of-order or missing
        /// calls, all earlier buffers are also removed, so order of the buffers
        /// in the queue reflects the chronology and matters.
        /// </remarks>

        Queue<Buffer> _usedBuffers;

        /// <summary>
        /// Stack of free buffers available for recycling by a new sample.
        /// </summary>
        /// <remarks>
        /// Since buffer resize shall correspond to video resize and thus be infrequent,
        /// favor reusing the last released buffer, which is most likely to have the same
        /// capacity as the new frame, by using a stack.
        /// </remarks>
        Stack<Buffer> _freeBuffers;

        /// <summary>
        /// Construct a new pool of buffers.
        /// </summary>
        /// <param name="capacity">Initial capacity of both the used and free collections of buffers</param>
        public StreamSamplePool(int capacity)
        {
            this._usedBuffers = new Queue<Buffer>(capacity);
            this._freeBuffers = new Stack<Buffer>(capacity);
        }

        /// <summary>
        /// Get a sample from the pool which has a buffer with a given capacity
        /// and with the associated timestamp.
        /// </summary>
        /// <param name="byteSize">The exact size in bytes that the sample buffer needs to accomodate.</param>
        /// <param name="timestamp">The sample presentation timestamp.</param>
        /// <returns>The newly created sample</returns>
        /// <remarks>
        /// The returned sample's buffer has a <see cref="Windows.Storage.Streams.Buffer.Length"/> property
        /// set to the input <see cref="byteSize"/>. This is required to be set before creating the sample,
        /// and should not be modified once the sample was created.
        /// </remarks>
        public MediaStreamSample Pop(uint byteSize, System.TimeSpan timestamp)
        {
            Buffer buffer;
            lock (this)
            {
                if (_freeBuffers.Count > 0)
                {
                    buffer = _freeBuffers.Pop();
                    if (buffer.Capacity < byteSize)
                    {
                        buffer = new Buffer(byteSize);
                    }
                }
                else
                {
                    buffer = new Buffer(byteSize);
                }
                _usedBuffers.Enqueue(buffer);

                // This must be set before calling CreateFromBuffer() below otherwise
                // the Media Foundation pipeline throws an exception.
                buffer.Length = byteSize;
            }

            // Because the managed wrapper does not allow modifying the timestamp,
            // need to recreate the sample each time with the correct timestamp.
            var sample = MediaStreamSample.CreateFromBuffer(buffer, timestamp);
            sample.Processed += OnSampleProcessed;
            return sample;
        }

        /// <summary>
        /// Callback fired by MediaFoundation when a <see cref="Windows.Media.Core.MediaStreamSample"/>
        /// has been processed by the pipeline and its buffer can be reused.
        /// </summary>
        /// <param name="sample">The sample which has been processed.</param>
        /// <param name="args"></param>
        private void OnSampleProcessed(MediaStreamSample sample, object args)
        {
            lock (this)
            {
                // This does a linear search from front, which generally finds
                // the first object (oldest) or at worse one very close to front,
                // so is optimal anyway.
                // Remove this sample and all earlier ones too. Some users report that
                // the Processed event is not always reported for earlier samples, which
                // would result in memory leaks. This may be due to out-of-order reporting.
                while (_usedBuffers.TryDequeue(out Buffer buffer))
                {
                    // Save the buffer for later reuse
                    _freeBuffers.Push(buffer);

                    if (buffer == sample.Buffer)
                    {
                        break;
                    }
                }
            }
        }
    }

Answer 1

我已就此咨询过团队。我收到了其中一家媒体 SME 的回复。所以答案是不幸的是没有办法实现你想要做的事情，特别是因为 Timestamp 属性是只读的。您正在做的事情是您将获得的最佳效果，并且您无法使用 MediaStreamSample 使其变得更好。