索尼相机远程 API，Liveview 流速度慢

Question

我正在尝试在台式机上流式传输 SONY FDR-X1000V 摄像机的实时视图。我使用python调用API并下载包并使用opencv解码jpeg。 当我运行它时，它几乎无法每秒捕捉一帧。后来我发现一个 jpeg 的有效载荷大小可以是 8MB。但是，FDR-X1000V 不支持更改实时取景大小。但是当我在 iPhone 上使用该应用程序进行实时取景时，它可以流畅地播放。所以这是我的问题： 1. 一个jpeg的payload可以大到8MB是正常的吗？ 2. 如果是这样，我怎样才能流畅地进行实时取景？

这是我的代码：

try:
    result = api.do('startLiveview')
    url = result['result'][0]
except KeyError:
    print result
f = urllib2.urlopen(url)
buff = ''
chunk_size = 32768
for i in xrange(3000):
    if len(buff) < chunk_size:
        time_s = time.time()
        buff = buff + f.read(chunk_size)
        print "Download Speed %f KB/s"%(chunk_size/1000/(time.time() - time_s))
    time_s = time.time()
    start_code = ''.join(buff).find('$5hy')
    # print "LCS time cost", time.time() - time_s

    if start_code < 0:
        buff = buff[-12:]
        print "skip", len(buff)-12
    elif start_code < 8:
        buff = buff[8:]
    else:
        if start_code > len(buff) - 129:
            buff = buff + f.read(chunk_size)
        payload_type = ord(buff[start_code-7])
        payload_size, = struct.unpack('<I', buff[start_code+4:start_code+8].ljust(4,'\0'))
        padding_size = ord(buff[start_code+8])
        print "Type:%d\tPayload:%d\tPadding:%d\t"%(payload_type,payload_size,padding_size)

        buff = buff[start_code+128:]
        if payload_type == 1:
            if payload_size + padding_size > len(buff):
                time_s = time.time()
                download_size = payload_size+padding_size-len(buff)
                buff = buff + f.read(download_size)
                print "Download Speed %f KB/s"%(download_size/1000/(time.time() - time_s))
            img_data = buff[:payload_size]
            buff = buff[payload_size:]

            time_s = time.time()
            d = np.asarray(bytearray(img_data), dtype='uint8')
            img = cv2.imdecode(d,cv2.IMREAD_COLOR)
            cv2.imshow('postview',img)
            cv2.waitKey(30)
            # print "Decode time cost", time.time() - time_s

一些输出：

Type:1  Payload:8410624 Padding:0   
Download Speed 679.626326 KB/s

Answer 1

有点无耻的促销 - 但它可能会帮助你。您是否尝试过使用 pysony 代码？

您必须手动设置 QX_ADDR，但这应该会为您提供“更快”的实时取景流... https://github.com/Bloodevil/sony_camera_api/issues/22

pygameLiveView 示例具有所需的构建块，您只需调整 jpeg 加载以拉入 openCV 而不是 pygame

Answer 2

我认为问题可能是您将“LiveView 的有效负载标头”中的“有效负载数据大小没有填充大小”解析为 4 字节数据而不是 3 字节数据。所以在这种情况下，有效载荷大小看起来比预期的要大。这在“实时取景数据 JPEG 容器的格式”下的文档中进行了解释。尝试更改此设置，如果它仍然不适合您，请告诉我。

Answer 3

@Simon Wood 和@Robert - Sony 的回答都非常有用！ 我在代码中犯了两个错误。

一个是有效载荷大小是3个字节而不是4个字节。

另一个是我假设字节顺序是小端而不是大端。

@Simon Wood 提到的项目是一个不错的项目。虽然它没有在 4 字节起始码上实现重新同步。

为了做到这一点，我用两种方法编写自己的代码来检查和重新同步它：

try:
    result = api.do('startLiveview')
    url = result['result'][0]
except KeyError:
    print result
f = urllib2.urlopen(url)

#method 1
buff = ''
chunk_size = 8192
for i in xrange(300):
    if len(buff) < chunk_size:
        time_s = time.time()
        buff = buff + f.read(chunk_size)
        # print "Download Speed %f KB/s"%(chunk_size/1000/(time.time() - time_s))
    time_s = time.time()
    start_code = ''.join(buff).find('$5hy')
    # print "LCS time cost", time.time() - time_s

    if start_code < 0:
        buff = buff[-12:]
        print "skip", len(buff)-12
    elif start_code < 8:
        buff = buff[8:]
        print "skip a header"
    else:
        if start_code > len(buff) - 129:
            buff = buff + f.read(chunk_size)
        start_byte = ord(buff[start_code - 8])
        payload_type = ord(buff[start_code - 7])
        sequence_num, = struct.unpack('>I', buff[start_code - 6:start_code - 4].rjust(4,'\0'))
        time_stamp, = struct.unpack('>I', buff[start_code - 4:start_code].rjust(4,'\0'))
        payload_size, = struct.unpack('>I', buff[start_code+4:start_code+7].rjust(4,'\0'))
        padding_size = ord(buff[start_code+8])
        print "StartByte:%d\t sequenceNum:%d\t timeStamp:%d\t Type:%d\t Payload:%d\t Padding:%d\t"%(
            start_byte,sequence_num,time_stamp,payload_type,payload_size,padding_size)

        buff = buff[start_code+128:]
        if payload_type == 1:
            if payload_size + padding_size > len(buff):
                time_s = time.time()
                download_size = payload_size+padding_size-len(buff)
                buff = buff + f.read(download_size)
                # print "Download Speed %f KB/s"%(download_size/1000/(time.time() - time_s))
            img_data = buff[:payload_size]
            buff = buff[payload_size:]

            time_s = time.time()
            d = np.asarray(bytearray(img_data), dtype='uint8')
            img = cv2.imdecode(d,cv2.IMREAD_COLOR)
            cv2.imshow('postview',img)
            cv2.waitKey(10)
            # print "Decode time cost", time.time() - time_s

#method 2
def checkbyte(f):
    if f.read(4) == '$5hy':
        return
    state = 0
    i = 1
    while 1:
        i+=1
        if state == 0 :
            if f.read(1) == '$':
                state = 1
            else:
                state = 0
        if state == 1 :
            if f.read(1) == '5':
                state = 2
            else:
                state = 0
        if state == 2 :
            if f.read(1) == 'h':
                state = 3
            else:
                state = 0
        if state == 3 :
            if f.read(1) == 'y':
                state = 4
            else:
                state = 0
        if state == 4 :
            print 'skip', i
            return
for i in xrange(300):
    buff = f.read(8)
    start_byte ord(buff[0])
    payload_type, = struct.unpack('>I',buff[1].rjust(4,'\0'))
    sequence_num, = struct.unpack('>I',buff[2:4].rjust(4,'\0'))
    time_stamp, = struct.unpack('>I',buff[4:8])

    #payload header
    checkbyte(f)
    buff = f.read(124)
    payload_size, = struct.unpack('>I',buff[0:3].rjust(4,'\0'))
    padding_size= ord(buff[3])

    print "StartByte:%d\t sequenceNum:%d\t timeStamp:%d\t Type:%d\t Payload:%d\t Padding:%d\t"%(
            start_byte,sequence_num,time_stamp,payload_type,payload_size,padding_size)
    d = f.read(payload_size)
    if padding_size > 0:
        f.read(padding_size)

    if payload_type == 1:
        # Type = 0x01
        d = np.asarray(bytearray(d), dtype='uint8')
        img = cv2.imdecode(d,cv2.IMREAD_COLOR)
        cv2.imshow('postview',img)
        cv2.waitKey(1)

print api.do('stopLiveview')