如何处理“AWS Kinesis Video Stream(kvs) -> gstreamer+opencv image processing -> kvs”流程中的gstreamer？答案

【问题标题】：How to handle the gstreamer in "AWS Kinesis Video Stream(kvs) -> gstreamer+opencv image processing -> kvs " process?如何处理“AWS Kinesis Video Stream(kvs) -> gstreamer+opencv image processing -> kvs”流程中的gstreamer？
【发布时间】：2021-12-23 14:42:45
【问题描述】：

我想创建如下图所示的管道；

https://i.stack.imgur.com/Nb17P.png

GSTREAMER_OUT = ' ! '.join([
        'appsrc',
        'queue',
        f'video/x-raw,format=BGR,width={frame_width},height={frame_height}',
        'videoconvert',
        'video/x-raw,format=YV12',
        'x264enc byte-stream=true',
        'video/x-h264,stream-format=avc,alignment=au,profile=baseline',
        ' '.join([
            'kvssink',
            f'stream-name={kvs_out_stream_name}',
            'storage-size=512',
            f'access-key={access_key}',
            f'secret-key={secret_key}',
            'aws-region=ap-northeast-1',
            # 'buffer-duration=60',
            f'framerate=1',
            # 'max-latency=10',
        ]),
    ])

cap = cv2.VideoCapture(in_stream_url)
out = cv2.VideoWriter(GSTREAMER_OUT, cv2.CAP_GSTREAMER, 0, target_fps, (frame_width, frame_height), True)

while True:
    ret, frame = cap.read()
    result = some_image_process(frame)
    out.write(result)

但是，管理控制台经常显示如下所示的断帧；

https://i.stack.imgur.com/oecrR.png

整个视频：https://youtu.be/vBAgECHYJjU

我想知道如何修复这个 GStreamer 管道。我对 Gstreamer 不是很熟悉，如果您详细介绍 GStreamer 管道，我们将不胜感激。

【问题讨论】：

标签： opencv gstreamer amazon-kinesis-video-streams

【解决方案1】：

当我删除queue 并向x264enc 添加选项时，我发现此问题已修复，如下面的命令；

GSTREAMER_OUT = ' ! '.join([
    'appsrc',
    'clockoverlay halignment=right valignment=top font-desc="Sans bold 60px"',
    'videoconvert',
    'video/x-raw,format=YV12',
    'x264enc byte-stream=true noise-reduction=10000 speed-preset=ultrafast tune=zerolatency ',
    'video/x-h264,stream-format=avc,alignment=au,profile=baseline',
    ' '.join([
        'kvssink',
        f'stream-name={kvs_out_stream_name}',
        'storage-size=512',
        f'access-key={access_key}',
        f'secret-key={secret_key}',
        'aws-region=ap-northeast-1',
        # 'buffer-duration=10',
        # 'connection-staleness=60',
        # "fragment-acks=true",
        "framerate=1",
        # "key-frame-fragmentation=false",
        # "max-latency=10"
    ]),
])

更长的示例代码：https://gist.github.com/LittleWat/c93394bb4f6dc125d1c4a66f1a7aee9d

损坏的图像已修复，但存在延迟问题。

所以我会问另一个问题。 How to reduce the gstreamer pipeline latency in "AWS Kinesis Video Stream(kvs) -> gstreamer+opencv image processing -> kvs " process?

【讨论】：